NLU Workbench properties

  • Release version: Xanadu
  • Updated August 1, 2024
  • 3 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of NLU Workbench properties

    The NLU Workbench properties enable ServiceNow customers to configure and optimize the Natural Language Understanding (NLU) application for improved intent classification, vocabulary management, and model training. These system properties are accessible by users with theadminornluadminrole via the path:All > NLU Workbench > Settingsin the application navigator.

    Show full answer Show less

    Key Features

    • Model Settings: Control parameters such as the maximum number of utterances per intent (default 200, recommended less than 200 for balanced intent sizes), and limits on vocabulary sources:
      • Table vocabulary source records capped at 100,000
      • List vocabulary source values capped at 1,000
    • Pre-built Vocabulary: Enable recognition of software and hardware names by toggling pre-built vocabulary features.
    • Advanced Settings:
      • Intent Discovery classification record limits (minimum 10,000, recommended below 500,000 for quality results)
      • Minimum records for NLU performance analysis set at 5,000 for reliable insights
      • Conflict detection thresholds for NLU model tuning with moderate (.85) and critical (.95) levels
      • Batch test import file row limits set at 10,000 to ensure manageable testing sizes
      • Limits on the number of utterances for feedback display and model tuning in the Expert Feedback Loop, including a minimum of 100 utterances reviewed before tuning
      • Maximum fetch size for Virtual Agent chat logs is 3,000 by default, which can be increased up to 50,000 for high NLU usage scenarios
      • Size limits on label candidate and labeled data tables capped at 10,000 entries to maintain performance during pruning
      • Option to schedule NLU model training during off-peak hours to avoid instance blocking, with notification upon completion

    Practical Guidance

    • Maintain utterances per intent under 200 to keep models balanced.
    • Keep vocabulary source sizes within recommended limits to ensure performance.
    • Use sufficient data volumes (at least 10,000 records) for Intent Discovery to achieve high-quality classification results.
    • Leverage pre-built vocabularies to improve entity recognition for software and hardware terms.
    • Monitor NLU conflict detection thresholds to fine-tune model accuracy.
    • Use the Expert Feedback Loop to collect and review at least 100 user utterances before tuning to enhance model precision.
    • Consider increasing Virtual Agent chat log data fetch size if your instance has heavy NLU usage to improve feedback quality.
    • Enable scheduled model training to minimize impact on instance performance during peak times.

    Refer to these system properties for the Natural Language Understanding (NLU) application.

    NLU Workbench properties and their usage

    To access your system properties, use the admin or nlu_admin role and the following path in the application navigator: All > NLU Workbench > Settings.

    Table 1. Model Settings
    Label and Name Default value Plugin Recommended usage
    Maximum number of utterances per intent

    glide.nlu.utterances_per_intent.value_limit

    		 200 NLU Workbench Use fewer than 200 utterances per intent to keep your model well balanced in terms of intent size.
    Note:
    Value must be greater than 5 and less than or equal to 300.
    Maximum number of records in a Table vocabulary source

    glide.platform_ml.api.max_nlu_lookupsource_records

    		 100,000 NLU Workbench Keep the value under 100,000.
    Maximum number of values in a List vocabulary source

    glide.nlu.static_lookup.value_limit

    		 1,000 NLU Workbench Keep the value under 1,000.
    Enable pre-built vocabulary for software names

    glide.mlpredictor.option.nlu.@LookupSources:software

    		 enabled NLU Workbench Enable pre-built vocabulary so the system can recognize software names.
    Enable pre-built vocabulary for hardware names

    glide.mlpredictor.option.nlu.@LookupSources:hardware

    		 enabled NLU Workbench Enable pre-built vocabulary so the system can recognize hardware names.
    Table 2. Advanced Settings
    Label and Name Default value Plugin Recommended usage
    Maximum number of records for Intent Discovery classification

    sn_nlu_discovery.intent_discovery_max_classification_limit

    		 300,000 Intent Discovery Keep the number of records less than 500,000.
    Minimum number of records for Intent Discovery classification

    sn_nlu_discovery.intent_discovery_min_classification_limit

    		 10,000 Intent Discovery Use at least 10,000 records to get high quality results.
    Minimum number of records for NLU performance analysis

    sn_nlu_workbench.glide.nlu.performance.min_clustering_records

    		 5,000 NLU Workbench - Advanced Features Use at least 5,000 records to get high quality results.
    NLU Conflict Detection - Moderate Threshold

    sn_nlu_workbench.glide.nlu.conflict.moderate_threshold

    		 .85 NLU Workbench - Advanced Features Must be a decimal between 0 and 1. Keep this threshold less than the Critical Threshold.
    NLU Conflict Detection - Critical Threshold

    sn_nlu_workbench.glide.nlu.conflict.critical_threshold

    		 .95 NLU Workbench - Advanced Features Must be a decimal between 0 and 1. Keep this threshold greater than the Moderate Threshold.
    The maximum number of rows in a batch test import file

    sn_nlu_workbench.glide.nlu.batch_test.max_import_rows

    		 10,000 NLU Workbench - Advanced Features Make sure your batch test import file has no more than 10,000 rows.
    The maximum number of utterances to display for feedback in the expert feedback loop

    glide.mlpredictor.option.nlu.activeLearning.label_candidate_table.max_response_size

    		 300 NLU Workbench - Advanced Features Pull no more than 300 utterances from your users' Virtual Agent chat logs to display for feedback in the Expert Feedback Loop application.The minimum umber of utterances a user should review before tuning the model
    The minimum number of utterances a user should review before tuning the model

    sn_nlu_workbench.glide.nlu.optimize.min_labeled_data

    		 100 NLU Workbench - Advanced Features Provide and save feedback for at least 100 utterances from your users' Virtual Agent chat logs so you can execute the Tune Model feature in the Expert Feedback Loop application.
    The maximum number of records to fetch from Virtual Agent chat logs

    glide.mlpredictor.option.nlu.activeLearning.va_chat_logs.max_row_limit - 3000

    		 3,000 NLU Workbench - Advanced Features If there is high NLU usage, increasing the default value to a maximum of 50,000 records will increase the data available for the active learning job to filter up on and display in the Expert Feedback Loop application to give feedback on.
    Size limit on Label Candidate Table (used for pruning the table)

    glide.mlpredictor.option.nlu.activeLearning.label_candidate_table.max_data_size - 10000

    		 10,000 NLU Workbench - Advanced Features The recommended usage for this property is the same as the property above.
    Size limit on Labeled Data Table (used for pruning the table)

    glide.mlpredictor.option.nlu.activeLearning.label_table.max_data_size - 10000

    		 10,000 NLU Workbench - Advanced Features The recommended usage for this property is the same as the property above.
    Enable this property to unblock your instance during NLU model training. The training will be scheduled for an off-peak time, and we will notify you when it's done.

    glide.mlpredictor.scheduled.nlu.model.training

    		 False NLU Workbench - Advanced Features False

    To get more feedback data from Virtual Agent (VA) chat logs, refer to the Procuring additional VA feedback data on demand section in the Expert Feedback Loop documentation.