Model management
Summarize
Summary of Model management
The Model management feature in the NLU Workbench allows you to effectively oversee the entire life cycle of your Natural Language Understanding (NLU) models. It supports an iterative process of building, testing, publishing, and tuning models tailored for applications like Virtual Agent, AI Search, and Issue Auto Resolution. The available phases and functionalities adapt based on the chosen model application.
Show less
To utilize these phases, ensure you have the necessary NLU plugins installed, including the NLU Workbench - Advanced Features and Intent Discovery, which are available from the ServiceNow Store and required for model testing and performance monitoring.
Creating and Building Models
Create NLU models via the NLU Workbench under the appropriate application tab (Virtual Agent or AI Search). You can start from a prebuilt model, import training data via CSV, or build a model from scratch. Once created, build your model by adding and managing:
- Intents: Define user requests the model can understand.
- Entities: Extract contextual details from user inputs.
- Vocabulary: Include business-specific terms and acronyms for better understanding.
- Test set: Add utterances and expected intents for validation.
Training the model with relevant utterances prepares it for real user interactions.
Testing and Publishing Models
Test your model’s performance to identify improvements using the NLU Workbench - Advanced Features. This testing phase helps ensure your model meets accuracy and confidence requirements. Once satisfied with the test results, publish the model to make it available for use by applications like Virtual Agent.
Tuning Models
If the NLU Workbench - Advanced Features plugin is installed and your model is for Virtual Agent, you can use the Tune your model phase. This phase leverages the Expert Feedback Loop to incorporate real user utterances, continuously improving model accuracy. For Issue Auto Resolution models, tuning occurs through the IAR Tuning interface.
Model Settings
Modify your model’s name, description, and confidence threshold via the Settings page on the model overview. The confidence threshold controls how certain the model must be before predicting an intent, allowing you to fine-tune prediction sensitivity.
Additional Features
Irrelevance Detection: This feature enables your NLU model to identify and avoid responding to utterances that are irrelevant, helping keep Virtual Agent conversations focused and accurate.
Manage your NLU model's life cycle in the NLU Workbench. Model management phases guide you through the iterative process of building, testing, and publishing your model.
Bringing your NLU model from creation to deployment requires multiple steps, separated into phases. You can return to earlier phases when you want to adjust and maintain your model.
The phases available for your model depend on the model's application. The system will display a phase, button, or function only when it applies to your model's application.
Create a model
To create a model for Virtual Agent or AI Search, navigate to . The Virtual Agent tab opens by default. Select the appropriate tab for the model you want to create.
- Use prebuilt model: Copy one of the included read-only models, and add content specific to your business.
- Import data from CSV: Upload a CSV file that contains training utterances and matched intents.
- Start from blank: Go through the process of setting up a new model from scratch.
To get started, see Creating models.
Model management phases
After creating a model, access its management phases by navigating to . Select the tab for your model's application, then the name of the model to open the Model details page on the model overview.
There are three phases on a Virtual Agent model's overview page: Build and train your model, Test and publish your model, and Tune your model. These phases guide you as you build and improve your model.
Build and train your model
Build the model by adding and managing content:- Intents: Add more intents to broaden the range of user requests that your model can understand.
- Entities: Add more entities so that your model can extract more contextual details from your users' requests.
- Vocabulary: Add vocabulary to enable the model to better understand words and phrases that are specific to your business, such as industry terms and acronyms.
- Test set: Add test utterances and their expected intents to your model's default test set.
To learn more, see Build and train your model.
Train your model using utterances that the model is likely to encounter from your users. To learn more, see Train and try your NLU model.
Test and publish your model
Test your model to gauge the performance and identify areas for improvement.
For more information on testing and thresholds, see Test and publish your model.
When you're satisfied with the results of testing, publish your model to make it available for use by other applications. For more information, see Publish your NLU model.
Tune your model
If NLU Workbench - Advanced Features is installed, and your model is created for Virtual Agent, the Tune your model phase is enabled. With this phase, you can use Expert Feedback Loop to incorporate actual user utterances into your model.
For more information, see Tune your model.
If your model is created for Issue Auto Resolution, you will be taken to IAR Tuning by selecting the name of your model in the IAR tab of the NLU Workbench homepage. For more information, see Issue Auto Resolution Tuning in NLU.
Model settings
Use the Settings page of the model overview to change the name and description of the model. You can also modify the confidence threshold of the model. The confidence threshold determines how confident the model must be to predict an intent.
For more information, see NLU model settings.