• Total number of conversations—The total number of conversations, which is comprised of completed, incomplete, and in-progress conversations. Regardless of status, a conversation can be comprised of one or more dialog turns. Each turn is a single exchange between the user and the skill.
  Note
  
  Conversations are not the same as metered requests. To find out more about metering, refer to Oracle PaaS and IaaS Universal Credits Service Descriptions.
• Completed conversations – Conversations that have ended by answering a user's query successfully. Conversations are counted as complete when the traversal through the dialog flow ends with a return transition or at a state with the insightsEndConversation property.
• Incomplete conversations – Conversations that users didn't complete, because they abandoned the skill, or couldn't complete it because of system-level errors, timeouts, or infinite loops.
• In progress conversations – "In-flight" conversations (conversations that have not yet completed nor timed-out). This metric tracks multi-turn conversations. An in-progress conversation becomes an timeout after a session expires.
• Average time spent on conversations – The average length for all of the skill’s conversations.
• Total number of users and Number of unique users – User base metrics that indicate how many users a skill has and how many of these users are returning users.

• Average time spent on conversations – The average length of time of the voice conversations.
• Average Real Time Factor (RTF) – The ratio of the time taken to process the audio input relative to the CPU time. For example, if it takes one second of CPU time to process one second of audio, then the RTF is 1 (1/1). The RTF for 500 milliseconds to process one second of audio is .5 or ½ . Ideally, RTF should be below 1 to ensure that the processing does not lag behind the audio input. If the RTF is above 1, contact Oracle Support.
• Average Voice Latency – The delay, in milliseconds, between detecting the end of the utterance and the generation of the final result (or transcription). If you observe latency, contact Oracle Support.
• Average Audio Time – The average duration, in seconds, for all voice conversations.
• Switched Conversations – The percentage of the skill's conversations that began with voice commands, but needed to be switched to text to complete the interaction. This metric indicates that there were multiple execution paths involved in switching from voice to text.
  
  
  
  
  

• Timeouts – Timeouts are triggered when an in-progress conversation is idle for more than an hour, causing the session to expire.
• System-Handled Errors – System-handled errors are handled by the system, not the skill. These errors occur when the dialog flow definition is not equipped with error handling, either globally in the defaultTransitions node, or at the state level with error transitions.
• Infinite Loop – Infinite loops can occur because of flaws in the dialog flow definition, such as incorrectly defined transitions.
• Canceled - The number of times that users exited a skill by explicitly canceling the conversation.

• Number of users – A running total of all types of users who have interacted with the skill: users with channel-assigned IDs that persist across sessions (the unique users), and users whose automatically assigned IDs last for only one session.
• Number of unique users – The number of users who have accessed the skill as identified by their unique user IDs. Each channel has a different method of assigning an ID to a user: users chatting with the skill through the Web channel are identified by the value defined for userId field, for example. The Skill Tester's test channel assigns you a new user ID each time you end a chat session by clicking Reset.
  Once assigned, these unique IDs persist across chat sessions so that the unique user count tallied by this metric does not increase when a user revisits the skill. The count only increases when another user assigned with a unique ID is added to the user pool.

  Tip:

  Because the user IDs are only unique within a channel (a user with identical IDs on two different channels will be counted as two users, not one), you can get a better idea of the user base by filtering the report by channel.

• Completed – The conversations that users have successfully completed. These conversations include the ones where traversal through the dialog flow ended with the triggering of a return action, or ended at a state with the insightsEndConversation property.
• Incomplete – Conversations that users didn't complete, because they abandoned the skill, or couldn't complete because of system-level errors, timeouts, or flaws in the skill's design.
• In Progress – "In-flight" conversations (conversations that have not yet completed nor timed out). This metric tracks multi-turn conversations.

• You can trace the execution path for a selected intent by clicking View Path, which opens the Paths report filtered by completed conversations for the intent. To improve focus on the execution paths, you can filter out the states that you're not interested in.
  
  

  
  

  
  
• You can read transcripts of the completed conversations for an intent by clicking View Conversations, which opens the Conversations report filtered by completed conversations for the intent.
  
  

  
  

  
  

• Click View Path opens the Paths report filtered for incomplete conversations for the selected intent. The terminal states on this path may include states defined in the dialog or an internal state that marks the end of a conversation, such as System.EndSession, System.ExpiredSession, System.MaxStatesExceededHandler, and System.DefaultErrorHandler.
  
  
  
  
  
• You can access transcripts of conversations that lead to the failure by clicking View Conversations. This option opens the Conversations report filtered for incomplete conversations for the selected intent. You can narrow the results further by applying a filter. For example, you can filter the report by error conditions.
  
  

  
  

  
  

{"orderAction":"confirm""system.state":"orderSummary"}

states:
  intent:
    component: "System.Intent"
    properties:
      variable: "iResult"
    transitions:
      actions:
        Balances: "startBalances"
        Transactions: "startTxns"
        Send Money: "startPayments"
        Track Spending: "startTrackSpending"
        Dispute: "setDate"
        unresolvedIntent: "unresolved"

  startUnresolved:
    component: "System.Output"
    properties:
      text: "I can only order pizza for you today. Let me know what kind of pizza you'd like?"
      keepTurn: false
    transitions:
      return: "startUnresolved"

1. Click Settings > General.
2. Switch on Enable PII Anonymization.
3. Click Add Entity to select the entity values that you want to anonymize in the Insights reports and the logs.
   Note
   
   Anonymized values are persisted to the database only after you enable anonymization for PII values for the selected entities. They are not applied to prior conversations. Depending on the date range selected for the Insights reports or export files, the PII values might appear in both their actual and anonymized forms. You can apply anonymization to any non-anonymized PII value (including those in conversations that occurred before you enabled anonymization in the skill or digital assistant settings) when you create an export task. These anonyms apply only to the exported file and are not persisted in the database.
   If you want to discontinue the anonymization for a PII value, or if you don't want an anonym to be used at all, select the corresponding entity and then click Delete Entity. Once you delete an entity, the actual PII value appears throughout the Insights reports for subsequent conversations. Its anonymized form, however, will remain for prior conversations.
   Note
   
   Anonymization is permanent (the export task-applied anonymization notwithstanding). You can't recover PII values after you enable anonymization.

• The PII values recognized for the selected entities are replaced with anonyms. These anonyms get persisted to the database and replace the PII values in the logs and Insights reports. This anonymization is applied to the conversations that occur after – not prior to – your enabling of anonymization in Settings.
• The Enable PII anonymization for the file option for the export task is enabled by default to ensure that the PII values recognized for the entities selected in Settings are applied to conversations that occurred before PII anonymization had been set. The anonyms applied during the export to conversations that predate the PII anonymization exist in the export file only. The original PII values remain in the database, Insights logs, and in the Insights reports).
• If you switch off Enable PII anonymization for the file, only the PII values recognized for the entities that were selected in Settings will be anonymized. The log files will contain the anonyms for conversations that occurred after anonymization settings have been enabled for the skill or digital assistant. Prior conversations will appear as original, unmodified utterances with their PII values intact. Consequently, the export file may include both anonymized and non-anonymized conversations if part of the export task's date range predates anonymization.
  Note
  
  If your export task includes anonymized conversations that occurred prior to Release 22.04, the anonyms applied to the pre-22.04 conversations will be changed, or re-anonymized, in the export files when you select Enable PII anonymization for the file for the export task. The anonyms in the exported file will not match either the anonyms in pre-22.04 export files or the anonyms that appear in the Insights reports.

• The Enable PII anonymization for the file option will be disabled by default for the export task so that the exported file will contain all the original unmodified utterances, including the PII values.
• If you select Enable PII anonymization for the file, the PII values will be anonymized in the exported file only for the default entities, PERSON, EMAIL, URL, and NUMBER. The PII values will remain in the database, logs, and Insights reports.

  cancelOrder:
    component: "System.Output"
    properties:
      text: "Your order is canceled."
      insightsEndConversation: true 
    transitions:
      next: "intent"

...
  resolveSize:
    component: "System.SetVariable"
    properties:
      variable: "crust"
      value: "${iResult.value.entityMatches['PizzaSize'][0]}" 
      insightsInclude: false      
    transitions:
      ...
...

• You can only add user input to the training corpus that belongs to a draft version of a skill, not a published version.
• You can’t add any user input that’s already present as an utterance in the training corpus, or that you have already added using the Retrainer.

1. Because you cannot update a published skill, you must create a draft version before you can add new data to the corpus.

   Tip:

   Click Compare All Versions or switch off the Show Only Latest toggle to access both the draft and published versions of the skill.
   If you're reviewing a published version of the skill, select the draft version of the skill.
   
   
   
   
2. In the draft version of the skill, apply a filter, if needed, then click Search.
3. Select the user message, then choose the target intent from the Select Intent menu. If your skill supports more than one native language, then you can add it to the language-appropriate training set by choosing from among the languages in the Select Language menu.

   Tip:

   You can add utterances to an intent on an individual basis, or you can select multiple intents and then select the target intent and if needed, a language from the Add To menus that's located at the upper left of the table. If you want to add all of returned requests to an intent, select Utterances (located at the upper right of the table) and then choose the intent and language from the Add To menu.
4. Click Add Example.
5. Retrain the skill.
6. Republish the skill.
7. Update the digital assistant with the new skill.
8. Monitor the Overview report for changes to the metrics over time and also compare different versions of the skill to find out if new versions have actually added to the skill's overall success. Repeating the retraining process improves the skill's responsiveness for each new version. For skills integrated with Oracle Service Cloud Chat, for example, retraining should result in a downward trend in escalations, which is indicated by a downward trend in the usage of agent handoff intents.

1. Open the Exports page and then click + Export.
2. Enter a name for the report and then enter a date range.
3. Click Enable PII anonymization for the exported file to replace Personally Identifiable Information (PII) values with anonyms in the exported file. These anonyms exist only in the exported file if PII is not enabled in the skill settings. In this case, the PII values, not their anonym equivalents, still get stored in database and appear in the exported Insights logs and throughout the Insights reports, including the Conversations report, the Retrainer, and the key phrases in the word cloud. If PII has been enabled in the skill settings, then logs and Insights reports will contain anonyms.
   Note
   
   The PII anonymization that's enabled for the skill or digital assistant settings factors into how PII values that get anonymized in the export file and also contributes to the consistency of the anonymization in the export file.
4. Click Export.
5. When the task succeeds, click Completed to download a ZIP of the CSV (or CSVs for large exports). The name of the skill-level export CSV begins with B_. File names for digital assistant-level exports begin with D_.

• Total number of conversations – The total number of conversations for the selected time period and channel that include both conversations that requested a live agent and conversations where no live agent was requested.
• Conversations handled by live agent – The total number of conversations with live agent requests.
• Conversations handled by skills – The total number of conversations (either complete or incomplete) with no live agent requests.
• Conversations resolved by skill – The number of conversations that completed (that is, the dialog traversed to the exit state) with no live agent requests.
• Conversations abandoned while waiting for live agent - The number of conversations where users were never handed off to a live agent, despite having requested one. Conversations can be considered abandoned when users never connect with live agents, possibly because they've left the conversation or were timed out.
• Deflection Rate – The percentage of conversations, which is calculated as the tally of Conversations Resolved by Skill divided by the tally for the Total Number of Conversations.
• Number of users that were transferred to a human agent – The total number of users (unique and otherwise) who were transferred to a user agent.
• Number of unique users that were transferred to a human agent – The total number of unique users (a group that may include returning users) who were transferred to a live agent. To gauge skill usability, you can compare this metric, which may include returning users, to the number tallied by the Total number of conversations.
  
  
  
  
  

• Average Duration of Skill Conversations – The average number of seconds that users have spent having conversations as calculated by adding up the total amount of time from the start to the end of each conversation by the total number of conversations.
• Average Duration of Live Agent – The average number of seconds that users spent on conversations that were routed to a live agent. This amount of time, which is typically longer than the Average Duration of Skill Conversations, is calculated by adding up the total amount of time spent on all live agent conversations divided by the Conversations Handled by Live Agent tally.
• Average Wait Time for the Live Agent – The average number of seconds that the users had to wait in the queue before they were eventually connected to an agent.