Natural Language Understanding topic discovery logic in Virtual Agent

  • Release version: Xanadu
  • Updated August 1, 2024
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Natural Language Understanding topic discovery logic in Virtual Agent

    This document explains how the Virtual Agent in ServiceNow uses Natural Language Understanding (NLU) to discover and select topics based on user input. It details the conditions required for topic discovery, how Virtual Agent interacts with NLU models to predict intents, and how it determines which topic to present to the user. Additionally, it covers the handling of topic switching during an active conversation.

    Show full answer Show less

    Prerequisites for Topic Discovery

    • The topic must be published, active, and marked as discoverable in the Topics [syscstopic] table.
    • Topic conditions must evaluate to true at runtime.
    • If roles are assigned to the topic, the user must have at least one of those roles.
    • The NLU model for the topic must be bound to the session's language.

    Note: The Is Topic Visible setting does not affect the discovery process.

    NLU Topic Discovery Logic

    When a user inputs an utterance, Virtual Agent sends a request to the NLU provider with the utterance and all relevant NLU model IDs for the session language. The NLU returns matching intents linked to topics. Virtual Agent then:

    • Automatically selects a topic if one intent has a sufficiently high confidence score.
    • Prompts the user to choose from multiple matched topics if auto-selection criteria are not met.
    • Falls back to a keyword-based search if no intents match and backup keyword search is enabled via system properties.

    NLU Confidence Scores and Configuration

    Confidence scores determine how well an intent matches the user’s input. ServiceNow administrators can configure the minimum confidence threshold via the ServiceNow NLU driver settings. The logic for intent selection is:

    • If one intent exceeds the confidence threshold and is clearly the best match, it is auto-selected.
    • If multiple intents meet the threshold without a clear winner, a list is presented to the user for selection. The maximum number of topics shown is governed by a system property.
    • If no intents meet the threshold, Virtual Agent uses keyword search if enabled.

    Mid-Topic NLU Discovery Logic

    While a conversation is active on one topic, if the user enters an utterance that matches an intent bound to a different topic, Virtual Agent can switch to that new topic mid-conversation. For example, if the user is asked for their date of birth but instead requests to view incidents, Virtual Agent recognizes the intent and switches to the appropriate incident-related topic.

    Understand how Virtual Agent returns intents and how it selects which intents to show to the user.

    Prerequisites for topic discovery

    Virtual Agent can discover a topic when the following conditions are met:
    • The topic must be published and active.

      In the Topics [sys_cs_topic] table, the Active, Published, and Is Topic Discoverable values are set to true.

      Note:
      The Is Topic Visible column does not affect topic discovery.
    • The topic must be discoverable.
    • Topic conditions must evaluate to true at runtime.
    • If any roles are configured for the topic, the requestor must have at least one of those roles.
    • The topic's NLU model must have a binding for the session language.

    NLU topic discovery logic

    Virtual Agent sends a prediction request to the NLU provider. The request contains the utterance and all registered NLU model IDs that are bound to the session language. Matches return NLU intents that correspond with a topic. Virtual Agent then responds in one of the following ways:
    • Automatically selects a topic for the requester, based on the confidence score.
    • Prompts the requester to pick a topic from the returned list of matches.
    • Finds no matching topics.

    If no matches are found but backup keywords are enabled (the com.glide.cs.nlu.keywords.enabled property is true), Virtual Agent searches for a topic based on keywords.

    Figure 1. Virtual Agent topic discovery logic
    Topic discovery flow chart showing the process Virtual Agent uses to deliver a topic to the user.

    Virtual Agent NLU confidence scores

    Virtual Agent uses confidence scores to return predicted intents. To set the confidence value:

    1. Navigate to Conversational Interfaces > External NLU Integrations > Drivers.
    2. Select ServiceNow NLU. You can also select All and enter open_nlu_driver.list.
    3. In the Intent Confidence Threshold field, enter the confidence threshold.

    If an intent's confidence score is greater than or equal to the configured threshold, Virtual Agent considers it a good match.

    Virtual Agent uses the following logic when selecting intents:
    • Auto-selects the highest predicted intent.
      This occurs when only one intent is matched, or in the event of a tie-breaker, when the next closest match is a distant second.
      Note:
      If ServiceNow NLU is used and the Intent Confidence Delta field in the ServiceNow NLU driver table (open_nlu_driver.list) is set to 0, there can be no tiebreaker.
    • Returns a topic list for the requester to choose from.

      This occurs if auto-select is not applicable. The length of the list is determined by the com.glide.cs.max_number_display_topics system property.

    • No intents are matched.

      When zero NLU Intents are predicted with a confidence score greater than or equal to the configured threshold, Virtual Agent falls back to a keyword search if configured. (The com.glide.cs.nlu.keywords.enabled and com.glide.cs.nlu.keywords.include_topics_bound_to_lang system properties are true).

    Mid-topic NLU topic discovery logic

    While a topic is running, the requester can enter an utterance or phrase that results in a topic switch. For example:

    1. The requester is in a Virtual Agent conversation, and Topic A is running.
    2. Topic A prompts the user to enter their date of birth.
    3. Instead of choosing a date, the requester types, "I want to view my Incidents."
    4. Virtual Agent can't resolve this phrase to a date, so it issues an NLU prediction request.
    5. The NLU predictor returns Intent B, and Virtual Agent sees that Topic B is bound to Intent B.
    6. Virtual Agent switches the conversation to Topic B, which then presents information to the requester about their incidents.
    Figure 2. Virtual Agent mid-topic discovery logic
    Topic discovery flow chart that shows the process Virtual Agent uses to change topics in the middle of a conversation.