NLU vocabulary

  • Release version: Yokohama
  • Updated January 30, 2025
  • 4 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of NLU vocabulary

    NLU vocabulary items in ServiceNow help your system recognize the different expressions users employ when making requests. They enhance intent prediction by mapping various words, phrases, or patterns to standardized synonyms, improving the accuracy and confidence of natural language understanding models.

    Show full answer Show less

    Vocabulary Usage and Types

    • Regular vocabulary items: Words or phrases, including business-specific terms or acronyms, that are case insensitive. For example, defining an acronym explicitly raises model confidence.
    • Pattern vocabulary items: Use regular expressions (regex) to identify specific formats like email addresses, phone numbers, or capitalized terms, enabling more precise intent prediction.

    Vocabulary Sources

    Vocabulary sources reference sets of values, such as ServiceNow tables or static lists (e.g., conference rooms, catalog items). Unlike individual vocabulary items, sources allow mapping entire lists of related terms to a single synonym, which can be reused across multiple NLU models and specified in training utterances using the @ symbol.

    Practical Application with Intents

    Vocabulary items and sources work with intents to improve recognition. For example, adding "sfcrm" as a regular vocabulary item with a synonym "CRM software" helps the system understand acronyms in user requests. Pattern vocabulary can distinguish terms like "Word" as a software product by matching capitalization. Annotating entities in utterances allows extraction of specific values for Virtual Agent topics.

    Regex Use in Pattern Vocabulary Items

    Regex patterns enable precise matching, such as using \bIT\b to match the acronym "IT" but exclude words like ITSM. Patterns are case sensitive by default, but can be modified with regex flags for case insensitivity.

    Creating and Managing Vocabulary

    • Create regular vocabulary items: Define words or phrases mapped to synonyms for intent prediction.
    • Create pattern vocabulary items: Use regex to capture specific formats and improve recognition.
    • Create list vocabulary sources: Define static lists of terms replaced by synonyms in utterances.
    • Create table vocabulary sources: Use values from ServiceNow tables as vocabulary sources, syncing regularly to keep data current.

    Additional Resources

    ServiceNow provides pre-built vocabulary for common software and hardware terms to cover multiple variations in user utterances, simplifying setup. For detailed instructions, refer to creating NLU intents and vocabulary items.

    Use NLU vocabulary items to help the system recognize the various ways your users express their requests. Use vocabulary sources to help the system recognize objects in tables or lists, such as names of conference rooms or catalog items.

    This image shows a robot serving a platter of linked molecules. The robot and everything else is saying 'NLU Vocabulary' in 6 different languages.

    Vocabulary usage and context

    Vocabulary helps your model with the various words and phrases that it may encounter from your users. Since humans are not computers, they may come up with different ways to say the same thing. For example, if one user inputs an acronym rather than the full phrase, the system might be able to predict the correct intent by using the context in the utterance. However, by defining the acronym with a vocabulary item, you raise the model's confidence level and capability to predict intents correctly.

    The Build and train your model phase with the Vocabulary tab highlighted.

    Vocabulary items cover the varying words or phrases that might appear in utterances. The vocabulary items are mapped to the synonyms that you provide, for intent prediction. This helps the system recognize the various ways your users express their requests, while at the same time ensuring the consistency, confidence, and accuracy of predictions.

    Vocabulary item types

    NLU provides the following vocabulary types that you can use to create and configure a vocabulary item.
    Type Definition
    Regular

    A word or phrase that is not commonly known, such as a business or industry specific term or acronym. Regular vocabulary is case insensitive, so all case variations will be captured by the vocabulary you create.

    See Create a regular vocabulary item.
    Pattern

    A regular expression (regex) that can capture specific formats such as email addresses and phone numbers.

    See Create a pattern vocabulary item.

    Vocabulary sources

    You can also use vocabulary sources to cover various objects that your users might request. Vocabulary items and vocabulary sources differ in their usage:
    • Use a vocabulary item for an individual word, phrase, or pattern that can easily be mapped to a single synonym for the model to use.
    • Use a vocabulary source to reference a ServiceNow table or list so that the values can all be replaced by the single synonym you define. Vocabulary sources can be reused across all your NLU models.

    Use vocabulary sources when your user utterances refer to objects in a set. The sources can be referenced from a list of values from a ServiceNow table or a static list that you define, such as a list of catalog items or conference rooms. Once you create and save these sources, you can use the @ symbol to specify them in training utterances. You can also use these sources as entity values.

    For example, say you already have a list of all the conference rooms for your office. You can create a vocabulary source to reference that list rather than typing each conference room name into your intent's training utterances.

    To create vocabulary sources, refer to Create a table vocabulary source and Create a list vocabulary source.

    Vocabulary usage in relation to an intent

    Here's an example of how an intent can interact with the vocabulary in its utterance examples.
    Note:
    In NLU vocabulary, the synonym replaces the vocabulary that appears in the utterance.
    Table 1. Intent: OrderSoftware
    Utterance Issue and Solution
    "I need to access sfcrm"
    • Issue: the system doesn't recognize the acronym sfcrm, so can't accurately predict the intent.
    • Solution: Add sfcrm as a regular vocabulary item, and provide a synonym such as CRM software.
    "I need to install Word"
    • Issue: The term word is very common and does not necessarily indicate a software product. The term install may provide helpful context, but the confidence score may be too low to predict the OrderSoftware intent.
    • Solution: Create a pattern vocabulary item with a regex for capitalized Word, so that the system can recognize it as a software product. This added constraint makes it more likely that the OrderSoftware intent will be predicted.
      Note:
      To extract the specific software name to use in a Virtual Agent conversation topic, annotate it as an entity in the utterance.

    Use a single word or short phrase as a synonym for best results. You can map multiple vocabulary items to one synonym. Do not map one vocabulary item to multiple synonyms.

    For more context and examples, see Create an NLU intent.

    A Regex example for a pattern vocabulary item

    Let's say that you want to use a vocabulary item to identify the acronym IT, and map it to the synonym information technology.

    Regular vocabulary items are case-insensitive by default. This means that a regular vocabulary item would match both the acronym IT and the common word it. So you decide to use a pattern vocabulary item.

    The regex pattern that you can use is \bIT\b. The \b is a word boundary marker. This ensures that the pattern would not match ITSM or JIT, for example. The default case sensitivity of pattern vocabulary items means that \bIT\b would not match the common word it.

    Regex details: You can turn off case sensitivity in a pattern by using (?i). You can end that mode by using a minus sign as in (?-i). For example, (?i)te(?-i)st should match both test and TEst, but not teST or TEST.

    For more information, see Using regular expressions in entities.