Here are the steps for creating an intent in a skill.
To create an intent:
Click Intents in the left navbar.
If you already have defined your intents in a CSV file, click Import
Intents. Import Intents from a CSV File describes this file's format. Otherwise, click Add
Intent. Your skill needs at least two intents.
Click to enter a descriptive name or phrase for the intent in the Conversation Name field. For example, if the intent name is callAgent, the conversation name would be Talk to a customer representative. When the skill can't resolve a message to an intent, it outputs the user-friendly names and phrases that you enter into the Conversation Name field as the options that are listed in the Do you want to disambiguation messages described in How Confidence Win Margin Works and Configure the Intent and Q&A Routing.
Add the intent name in the Name field. If you don't
enter a conversation name, then the Name field value is
used instead. Keep in mind that a short name with no end punctuation might not
contribute to the user experience. The intent name displays in the
Conversation Name field for skills built with prior
versions of Digital Assistant.
Note
In naming your intents, do not use system. as a prefix.
system. is a namespace that's reserved for the intents
that we provide. Because intents with this prefix are handled differently by
Trainer Tm, using it may cause your intents to resolve in unexpected
ways.
Add a description of the intent. Your description should focus on what makes
the intent unique and the task or actions it performs.
If this is an answer intent, add a short answer to the Answer
field.
Optionally, in the Annotations field, add one or more
tags for the intent to categorize it in a way that is useful for you. You can
use any words of your choosing.
Tip:
On the
Intents page, you can filter the display of
intents by annotation.
Start building the training corpus by adding example utterances that illustrate the meaning
behind the intent. To ensure optimal intent resolution, use terms, wording, and
phrasing specific to the individual intent. Ideally, you should base your
training data on real-world phrases. You can save your utterances by clicking
Enter or by clicking outside of the input field. To manage the training set,
select a row to access the Edit () and Delete () functions.
To allow your skill to cleanly distinguish between intents, create an intent that
resolves inappropriate user input or gibberish.
While utterances can be added to an existing intent manually or by importing a
CSV, they can also be assigned to intents through data manufacturing jobs and the Insights retrainer.
In the Auto-Complete Suggestions field, enter a set of suggested phrases that
help the user enter an appropriately worded request. Do not add the entire set
of training data. Add a set of phrases that represent ideal user requests
instead. Adding too broad a set of utterances may not only confuse users, but
may also result in unexpected behavior.
This is an optional step. This function is only supported by the Oracle Web Channel.
Add an entity if the intent needs one to resolve the user input. To find out how, see Add Entities to Intents.
To teach your skill how to comprehend user input using the set of utterances that you’ve provided so far, click Train, choose a model and then click Submit.
As described in Which Training Model Should I Use?, we provide two models
that learn from your corpus: Trainer Ht and Trainer Tm. Each uses a different
algorithm to reconcile the user input against your intents. Trainer Ht uses
pattern matching while Trainer Tm a machine learning algorithm which uses word
vectors. Both skills that use Digital Assistant's native language support and skills with answer intents require Trainer
TM.
You’d typically follow this process:
Create the initial training corpus.
Train with Trainer Ht. You should start with Trainer Ht
because it doesn’t require a large set of utterances. As long as
there are enough utterances to disambiguate the intents, your skill
will be able to resolve user input.
If you get a Something’s gone wrong message when
you try to train your skill, then you may not have added a
sufficient number of utterances to support training. First off, make
sure that you have at least two intents with at least two (or
preferable more) utterances each. If you haven’t added enough
utterances, add a few more then train your skill.
Refine your corpus, retrain with Trainer Ht. Repeat as
necessary—training is an iterative process.
Train with Trainer Tm. Use this trainer when you’ve
accumulated a robust set of intents.
The Training Needed displays
whenever you add an intent or when you update an intent by adding, changing,
or deleting its utterances. To bring the training up to date, choose a
training model and then click Train. The model
displays an exclamation point whenever it needs training. When its training
is current, it displays a check mark.
Click Test Utterances (located at the upper left) to
open the Utterance Tester. Select the target language, then enter utterances
similar to those in your training set. The Utterance Tester returns the
confidence level for this utterance and enables you to assign the utterance to
an intent, or add it as a test case.
To log your intent testing results, enable the conversation
intent logging (Settings > General > Enable
Insights) .
Click Validate and review the validation messages for errors such as too few utterances
and for guidance on applying best practices like adding an
unresolvedIntent intent.
Add Entities to Intents 🔗
Some intents require entities—both built-in and custom— to complete an action within
the dialog flow or make a REST call to a backend API. The system uses only these
entities, which are known as intent entities, to fulfill the intent that’s associated
with them. You can associate an entity to an intent when you click Add New
Entity and then select from the custom () or built-in () entities. If you're assigning a built-in entity, leave Value
Agnostic enabled (the default) if specific entity values do not factor
into intent classification (which is generally the case). If the intent requires a
specific entity value, switch this feature off.
Note
Value Agnostic
applies to built-in entities only. You cannot apply it to custom
entities.
Alternatively, you can click New Entity to add an
intent-specific entity.
Tip:
Only intent
entities that are included in the JSON payloads are sent to, and returned by, the
Component Service. The ones that aren’t associated with an intent won’t be included,
even if they contribute to the intent resolution by recognizing user input. If your
custom component accesses entities through entity matches, then be sure to add the
entity to your intent.
Value Agnostic Intent
Entities 🔗
The Value Agnostic feature allows you to adjust how
entity values affect intent classification. When you enable this feature, the specific
values for an associated built-in entity do not have bearing on the intent
classification. However, when you disable this feature, you allow the entity value to
play a key role in resolving the input.
In general, you can leave this feature in its default setting (enabled)
because a specific entity value seldom factors into intent classification. The training
utterances for an account balances intent, for example, may include specific dates
(What was my balance on October 5?) but these values are not the deciding
factor in resolving the input to the intent. Leaving Value
Agnostic enabled will, in most cases, improve intent resolution because
it prevents the values from affecting confidence scores or even signaling an unintended
intent. However, whenever specific values play a key role in intent resolution, you
should switch this feature off. For example, you would disable the feature if the value
for a DATE is central to distinguishing an intent for checking past vacation balances
from an intent that checks for future vacation balances. If these intents were date
agnostic, then the model would ignore past and present and would not resolve input
correctly.
Example Intents
Associated Entity
Training Utterances
Enable Value Agnostic?
Account Balance
DATE
Can you tell me my account balance
yesterday?
How much money do I have in checking?
What was my balance on October 5th?
What was my credit card balance last
week?
What is my bank balance today?
What was my savings account balance on
5/3?
Yes – The specific date values do not signal the
intent. The various date values in these utterances can be ignored
because a user can ask for an account balance on any day.
Holiday Store Hours
DATE
Are you open on January 1st?
Are you open on Thanksgiving?
Hours for New Year's Day
What are the store hours for July
4th?
What are your holiday hours?
Will you be open on Christmas?
No – The intent classification is based on a specific
(and limited) set of values and users are inquiring about
holidays.
Check Past Vacation Balance
Check Future Vacation Balance
DATE
Check Past Vacation Balance
Did I take any time off last
month?
Check Future Vacation Balance
Any planned vacation in next
month?
No – Disable Value Agnostic
for both intents. Agnostic DATE values in this case would mean that
the model would not consider a value as past or future. A "last
month" value, which should signal the Check Past Vacation Balance
intent, would be ignored. As a result, similarly worded input like
"Did I take any time off next month" may resolve incorrectly
to this intent.
Import Intents from a CSV File 🔗
You can add your intents manually, or import them from a CSV file. You can create
this file from a CSV of exported intents, or by creating it from
scratch in a spreadsheet program or a text file.
The CSV file has six columns for skills that use the Natively-Supported language mode and five columns for those that don't. Here are the column names and what they represent:
query: An example utterance.
topIntent: The intent that the utterance should match to.
conversationName: The conversation name for the intent.
answer: For answer intents, the static answer for the intent.
enabled: If true, the intent is enabled in the skill.
nativeLanguageTag: (For skills with native-language support only) the language of the utterance. For values, use two-character language tags (fr, en, etc,).
For skills with Digital Assistant's native language support, this column is required.
For skills without the native language support, you can't import a CSV that has this column.
Here's an excerpt from a CSV file for a skill that does not have native language support and which doesn't use answer intents.
query,topIntent,conversationName,answer,enabled
I want to order a pizza,OrderPizza,Order a Pizza.,,true
I want a pizza,OrderPizza,Order a Pizza.,,true
I want a pizaa,OrderPizza,Order a Pizza.,,true
I want a pizzaz,OrderPizza,Order a Pizza.,,true
I'm hungry,OrderPizza,Order a Pizza.,,true
Make me a pizza,OrderPizza,Order a Pizza.,,true
I feel like eating a pizza,OrderPizza,Order a Pizza.,,true
Gimme a pie,OrderPizza,Order a Pizza.,,true
Give me a pizza,OrderPizza,Order a Pizza.,,true
pizza I want,OrderPizza,Order a Pizza.,,true
I do not want to order a pizza,CancelPizza,Cancel your order.,,true
I do not want this,CancelPizza,Cancel your order.,,true
I don't want to order this pizza,CancelPizza,Cancel your order.,,true
Cancel this order,CancelPizza,Cancel your order.,,true
Can I cancel this order?,CancelPizza,Cancel your order.,,true
Cancel my pizza,CancelPizza,Cancel your order.,,true
Cancel my pizaa,CancelPizza,Cancel your order.,,true
Cancel my pizzaz,CancelPizza,Cancel your order.,,true
I'm not hungry anymore,CancelPizza,Cancel your order.,,true
don't cancel my pizza,unresolvedIntent,unresolvedIntent,,true
Why is a cheese pizza called Margherita,unresolvedIntent,unresolvedIntent,,true
Here's an excerpt from a CSV file for a skill with native-language support that uses answer intents.
query,topIntent,conversationName,answer,enabled,nativeLanguageTag
Do you sell pasta,Products,Our Products,We sell only pizzas. No salads. No pasta. No burgers. Only pizza,true,en
Vendez-vous des salades,Products,Our Products,Nous ne vendons que des pizzas. Pas de salades. Pas de pâtes. Pas de hamburgers. Seulement pizza,fr
do you sell burgers,Products,Our Products,We sell only pizzas. No salads. No pasta. No burgers. Only pizza,true,en
Do you sell salads,Products,Our Products,We sell only pizzas. No salads. No pasta. No burgers. Only pizza,true,en
Vendez des hamburgers,Products,Our Products,Nous ne vendons que des pizzas. Pas de salades. Pas de pâtes. Pas de hamburgers. Seulement pizza,true,fr
To import a CSV file:
Click Intents () in the left navbar.
Click More, and then choose Import intents.
Select the .csv file and then click Open.
Train your skill.
Export Intents to a CSV File 🔗
You can reuse your training corpus by exporting it to CSV. You can then import this file to another skill.
We provide a duo of training models that mold your skill’s cognition, Trainer Tm and
Trainer Ht. You can use either of these models, each of which uses a different approach
to machine learning. In general, you train your with Trainer Tm before you put your
skills into production. Because of its shorter training time, you can use Ht for
prototyping or for skills.
Note
You can't
use Trainer Ht for skills that use answer intents, use native language support, or
have a large number of intents. Use Trainer Tm for these skills.
Trainer Ht is
the default model, but you can change this by clicking Settings >
General and then by choosing another model from the list. The default
model displays in the tile in the skill catalog.
Trainer Tm 🔗
Trainer Tm (Tm) achieves highly accurate intent
classification even when a skill has hundreds, or even thousands, of intents. Even
though the intents in these large data sets are often closely related and are sometimes
"unbalanced" in quantity of utterances, Tm can still differentiate between them. In
general, you would apply Tm to any skill before you put it into production.
You don't need to bulk up your training data with utterances
that accommodate case sensitivity (Tm recognizes BlacK Friday as Black Friday, for
example), punctuation, similar verbs and nouns, or misspellings. In the latter case,
Trainer Tm uses context to resolve a phrase even when a user enters a key word
incorrectly. Here are some general guidelines for building a training corpus when you're
developing your skill with this model.
Trainer Tm enhances the skill's cognition by
Recognizing the irrelevant content. For I'm really excited about the
coming Black Friday deals, and can't wait for the deals. Can you tell me
what's going to be on sale for Black Friday?, Trainer Tm:
Discards the extraneous content (I'm really excited about
the coming Black Friday deals...)
Resolves the relevant content (Can you tell me what's
going to be on sale for Black Friday?) to an intent. In this
case, an intent called Black Friday Deals.
Trainer Tm can also distinguish between the relevant and
irrelevant content in a message even when the irrelevant content can
potentially be resolved to an intent. I bought the new 80 inch TV on
Black Friday for $2200, but now I see that the same set is available
online for $2100. Do you offer price match? for example, could be
matched to the Black Friday Deals intent and to a Price Matching intent,
which is appropriate for this message. In this case Trainer Tm:
Recognizes that I bought the new 80 inch TV on Black
Friday for $2200, but now I see that the same set is available
online for $2100 is extraneous content.
Resolves Do you offer price match?
Resolving intents when a single word or a name matches an entity.
For example, Trainer Tm can resolve a message consisting of only Black
Friday to an intent that's associated with a entity for Black
Friday.
Distinguishing between similar utterances (Cancel my order vs. Why
did you cancel my order?).
Recognizing out-of-scope utterances, such as Show me pizza recipes or
How many calories in a Meat Feast for a skill for fulfilling a pizza
order and nothing else.
Recognizing out-of-domain utterances, such as What's the weather like
today for a pizza ordering skill.
Tip:
While Trainer Tm can easily
distinguish when a user message is unclassifiable because it's clearly
dissimilar from the training data, you might still want to define an
unresolvedIntent with utterances that represent the
phrases that you want to make sure do not resolve to any of your skill's
intents. These phrases can be within the domain of your skill, but are still
out of scope, even though they may share some of the same words as the
training data. For example, I want to order a car for a pizza skill,
which has also been trained with I want to order a pizza.
Distinguishing between similar entities – For example, Tm recognizes that
mail is not same as email in the context of an intent called Sign Up for Email
Deals. Because it recognizes that an entity called regular mail would be out of
scope, it would resolve the phrase I want to sign up for deals through
regular mail at a lower confidence than it would for I want to sign
up for email deals.
Trainer Ht 🔗
Trainer Ht is the default training model. It needs only a small training corpus, so
use it as you develop the entities, intents, and the training
corpus. When the training corpus has matured to the point where
tests reveal highly accurate intent resolution, you’re ready to
add a deeper dimension to your skill’s cognition by training
Trainer Tm.
You can get a general understanding of how Trainer Ht resolves intents just from the training corpus itself. It forms matching rules from the sample sentences by tagging parts of speech and entities (both custom and built-in) and by detecting words that have the same meaning within the context of the intent. If an intent called SendMoney has both Send $500 to Mom and Pay Cleo $500, for example, Trainer Ht interprets pay as the equivalent to send . After training, Trainer Ht’s tagging reduces these sentences to templates (Send Currency to person, Pay person Currency) that it applies to the user input.
Because Trainer Ht draws on the sentences that you provide, you can predict its
behavior: it will be highly accurate when tested with sentences similar to the ones that
make up the training corpus (the user input that follows the rules, so to speak), but
may fare less well when confronted with esoteric user input.
Build Your Training
Corpus 🔗
When you define an intent, you first give it a name that illustrates some user
action and then follow up by compiling a set of real-life user statements, or
utterances. Collectively, your intents, and the utterances that belong to them, make up
a training corpus. The term corpus is just a quick way of saying “all of the
intents and sample phrases that I came up with to make this skill smart”. The corpus is
the key to your skill’s intelligence. By training a model with your corpus, you
essentially turn that model into a reference tool for resolving user input to a single
intent. Because your training corpus ultimately plays the key role in deciding which
route the skill-human conversation will take, you need to choose your words carefully
when building it.
Generally speaking, a large and varied set of sample phrases increases a model’s ability to resolve intents accurately. But building a robust training corpus doesn’t just begin with well-crafted sample phrases; it actually begins with intents that are clearly delineated. Not only should they clearly reflect your use case, but their relationship to their sample sentences should be equally clear. If you’re not sure where a sample sentence belongs, then your intents aren’t distinct from one another.
You probably have sample utterances in mind when you create your intents, but you
can expand upon them by using these guidelines.
Guidelines for Trainer Tm 🔗
Use a minimum confidence threshold of 0.7 for any skill that you plan
to put into production.
Use good naming conventions for your intent names so it's easy to
review related intents.
As a general rule, create at least 80 to 100 utterances for each
intent. Per the corpus size and shape guidelines, the minimum (through not recommended)
number of utterances for an intent is two. The total number of utterances in
your training set should not exceed 25,000.
If possible, use unmodified, real-word phrases that include:
vernacular
standard abbreviations that a user might enter ("opty" for
opportunity, for example)
non-standard names, such a product names
spelling variants ("check" and "cheque", for example)
If you don't have any actual data, incorporate these in your own training
data. Here are some pointers:
Create fully formed sentences that mention both the action
and the entity on which the action is performed.
Try to keep the utterance length between 3 and 30 words.
Utterances that are too short and lacking context can cause the model to
generalize in unpredictable ways. Utterances that are too long may
prevent the model from identifying the pertinent words and phrases.
There can be exceptions, however, for one- or two-word utterances when
they're commonly used phrases. If you expect two-word messages like
order status, price check, membership info, or
ship internationally) that specify both the entity and
action, add them to your training data. Be sure that your sample phrases
have both an action and an entity.
Be specific. For example, What is your store phone
number? is better than What is your phone number? because
it enables Trainer Tm to associate a phone number with a store. As a
result of this learning, it will resolve What's your mom's phone
number? to a lower confidence score.
While Trainer Tm detects out-of-scope utterances, you can
still improve confidence and accuracy by creating an
unresolvedIntent for utterances that are in domain
but still out of scope for the skill's intents. This enables Trainer Tm
to learn the boundary of domain intents. You can define an
unresolvedIntent for phrases that you do not want
resolved to any of your skill's intents. You may only want to define an
unresolvedIntent when user messages have been
resolved to a skill's intents even when they don't apply to any of
them.
Vary the words and phrases that surround the significant content as much
as possible. For example, I'd like a pizza, please", "Can you get me a
pizza?", "A pizza, please"
Some practices to avoid:
Do not associate a single word or phrase with a
specific intent unless that word or phrase indicates the intent.
Repeated phrases can skew the intent resolution. For example,
starting each OrderPizza utterance with "I want to …" and each
ShowMenu intent with "Can you help me to …" may increase the
likelihood of the model resolving any user input that begins
with "Can you help me to" with OrderPizza and "I want to" with
ShowMenu.
A high occurrence of one-word utterances in your
intents. One-word utterances are an exception. Use them
sparingly, if at all.
Open-ended utterances that can easily apply to
other domains or out-of-domain topics.
Your corpus doesn't need to repeat the same
utterance with different casing or with different word forms
that have same lemma. For example, because Trainer Tm can
distinguish between manage, manages, and manager, it not only
differentiates between "Who does Sam manage?" and "Who manages
Sam?", but also understands that these words are related to one
another.
Note
You may
be tempted to add misspellings of words. But before you do,
use those misspellings in the utterance tester to see if the
model recognizes them. You might be surprised at how well it
handles them. Also, by not adding misspellings you run less
risk of skewing your model in unexpected ways.
Create test cases to ensure the integrity of the intent
resolution.
When you deploy your skill, you can continuously improve the
training data by:
Reviewing the Conversation Logs, summaries of all
conversations that have occurred for a specified period. You enable the
logging by switching Enable Insights on in
Settings.
Running Skill Quality Reports and by assigning (or reassigning) actual user messages
to your intents with the Insights Retrainer. If these reports indicate
that unresolvedIntent has a lot of misclassified
utterances within the domain intents:
Move the in-scope utterances from
unresolvedIntent to the domain
intents.
Move the out-of-scope utterances from the domain
intents to unresolvedIntent.
Guidelines for Trainer Ht 🔗
Create 12 to 24 sample phrases per intent, if possible. Use unmodified,
real-word phrases that include:
vernacular
common misspellings
standard abbreviations that a user might enter ("opty" for
"opportunity", for example)
non-standard names, such a product names
spelling variants ("check" and "cheque", for example)
If you don't have any actual data, incorporate these in your own training data. Here
are some pointers:
Vary the vocabulary and sentence structure in these starter
phrases by one or two permutations using:
slang words (moolah, lucre, dough)
standard abbreviations that a user might enter ("opty"
for opportunity, for example)
non-standard names, such a product names
common expressions (Am I broke? for an intent
called AccountBalance)
alternate wording (Send cash to savings,
Send funds to savings, Send money to savings,
Transfer cash to savings.)
different categories of objects (I want to order a
pizza, I want to order some food).
alternate spellings (check, cheque)
common misspellings ("buisness" for business)
unusual word order (To checking, $20 send)
Use different concepts to express the same intent, like I am
hungry and Make me a pizza
Do not associate a single word or phrase with a specific intent
unless that word or phrase indicates the intent. Repeated phrases can skew the
intent resolution. For example, starting each OrderPizza utterance with "I want
to …" and each ShowMenu intent with "Can you help me to …" may increase the
likelihood of the model resolving any user input that begins with "I want to"
with OrderPizza and "Can you help me to" with ShowMenu.
Avoid sentence fragments and single words. Instead, use complete
sentences (which can be up to 255 characters) that include the action and
the entity. If you must use single key word examples, choose them
carefully.
Create test cases to ensure the integrity of the test the intent
resolution. Because adding a new intent examples can cause regressions, you
might end up adding several test phrases to stabilize the intent resolution
behavior.
Limits for Training Data Shape and
Size 🔗
Regarding training data and shape, here are the limits to the number of intents
and utterances.
Intents:
Minimum number of intents per skill: 2
Maximum number of intents per skill: 2,500
Utterances:
Maximum number of utterances per skill: 25,000
Minimum number of utterances per intent: 2
Utterance word length: Between 3 and 30 words. Per the
guidelines for Trainer Tm, there are exceptions where one or
two-word utterances can be appropriate if they are commonly
used.
Note
These are technical limits, not
recommendations. See Guidelines for Trainer Tm for practical recommendations for shaping your skills
and providing robust training data.
Export Intent Data 🔗
To log conversations, be sure to enable Enable Insights in Settings > General before you test your intents.
To export data for a skill:
Click to open the side menu and select Development > Skills.
In the tile for the skill, click and select Export Conversations.
Choose Intent Conversation Log, set the logging period, and then click
Export.
Review the user input by opening the CSV
files in a spreadsheet program.