---
sourceDocument: Yokohama Enable AI
sourceDocumentLink: https://servicenow-prod.fluidtopics.net/r/yokohama/intelligent-experiences

 Release :

    - yokohama

ft:locale :

    - en-US

ft:publication_title :

    - Yokohama Enable AI

ft:clusterId :

    - platai

bundleId :

    - platai

workflow :

    - Platform


---

# Test set creation and management

# Test set creation and management {#ariaid-title1}

Release version: Yokohama  
Updated January 30, 2025  
![](https://www.servicenow.com/docs/portal-asset/ico-clock) 4 minutes to read
Summarize  
![AI sparkle icon](https://servicenow.com/docs/portal-asset/ai-sparkle-icon) Summarized using AI  
This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.  

## Summary of Test set creation and management

ServiceNow's NLU models, used in Virtual Agent and AI Search, come with a default test set to evaluate model performance and accuracy.
This test set initially is empty and must be populated with utterances and expected intents to effectively test the model.
Managing the test set over time helps maintain and improve the model's accuracy.
Show full answer Show less  
To enable testing capabilities, customers must install the **NLU Workbench - Advanced Features** application from the ServiceNow Store.

## Accessing and Managing the Default Test Set

Customers can access the default test set through the NLU Workbench by navigating to their model's overview page and selecting the appropriate tabs or tiles such as *Build and Train your model* or *Test Coverage*. Alternatively, test sets are accessible via the Multi-model Batch Testing interface.

The test set can be populated in three ways:

* **Manual entry:** Add utterances and expected intents directly via the interface.
* **Import:** Upload utterances and intents from CSV files or other models.
* **Expert Feedback Loop:** Import real user utterances from Virtual Agent chat logs to enhance test relevance.

## Test Coverage and Quality

Test Coverage indicates the percentage of enabled intents covered by test utterances. A minimum of 60% coverage with at least 5 utterances per intent is required for reliable confidence threshold settings during batch testing. Higher coverage improves the accuracy of performance results.

It is recommended to include about 10% of utterances marked as "not relevant" to test the model's ability to handle irrelevant inputs without predicting intents.

## Using the Test Set

The default test set supports model testing during the *Test and publish your model* phase and Multi-model Batch Testing. Proper use of the test set facilitates better confidence in model readiness and deployment decisions.

## Characteristics and Maintenance

* Default test sets are automatically created upon instance upgrades for existing models lacking them.
* When duplicating a model, the original test set is copied to the new model.
* Test sets must be in the same language as the model and should not share utterances with the training set.
* Default test sets cannot be deleted independently of their associated models.
* Test sets are available only for Virtual Agent and AI Search NLU models.

## Downloading and Moving Test Sets

Customers can download default test sets as CSV files (containing utterances and expected intents but not source information) from the model overview page. Additionally, test sets can be moved between instances using update sets, which include all test utterances, intents, and sources when the NLU model is added to an update set.

Note that exporting models as CSV does not include the test set content.  
Use the default test set of your NLU model to test the model's
performance and accuracy. Manage your test set over time by building or updating its content in
the NLU Workbench.
Note:  
To test your model, install the ServiceNow® Store application NLU Workbench - Advanced Features. For more information, see [Install NLU Workbench - Advanced Features](https://servicenow-prod.fluidtopics.net/e6w1em~rJVUJ6NXkr4vAbg "You can install the NLU Workbench - Advanced Features application (com.snc.nlu.workbench.advanced) if you have the admin role.").
When you create an NLU model for Virtual Agent or AI Search, a default test set is created and associated to the model. You can use the default test set to evaluate the model's performance. Initially the test set is empty, ready to be populated with your content.

## Access your default test set {#nlu-test-set-creation-management__section_qyl_v5r_dwb}

Access your default test set with one of the following methods.

* Navigate to AllNLU WorkbenchModels. Select the tab for your model's application, then the name of your model from the list. On the model's overview page, find the Build and Train your model card and select its View phase button. Then select the Test set tab.
* Navigate to AllNLU WorkbenchModels. Select the tab for your model's application, then the name of your model from the list. On the model's overview page, select the Test Coverage tile.
* Navigate to AllMulti-model Batch TestingTest sets tab. Find the name of your model. Default test sets are labeled as Default.
{#nlu-test-set-creation-management__ul_vnh_drr_dwb}

## Add content to your default test set {#nlu-test-set-creation-management__section_ccp_1vr_dwb}

Add utterances and their expected intents to build and manage your test set over time. You can add
content to the default test set with the following methods:

* Add test utterances and their expected intents manually. From the model's overview page navigate to Build and train your modelTest set tab. Type your input into the Type a test utterance here field, select an appropriate intent, then select the Add button.

  These test
  utterances are assigned a source of Manual.
* Import test utterances and their expected intents from a CSV file or from other models. To import content to a default test set, from the model's overview page navigate to Build and train your modelTest set tab. Select Import test utterances.

  Imported test
  utterances are assigned a source of Manual.
* The Expert Feedback Loop feature lets you add actual user utterances from Virtual Agent chat logs to the test set.

  These test utterances are assigned a source of Expert Feedback. For more information, see [NLU Expert Feedback Loop](https://servicenow-prod.fluidtopics.net/QePBlbg_TxiRCdagCueKpA "Provide feedback on Virtual Agent chat log utterances to help the system continuously learn and to better predict user input.").
{#nlu-test-set-creation-management__ul_usv_2gh_vvb}

## Test Coverage {#nlu-test-set-creation-management__section_drd_pwr_dwb}

The Test Coverage score is the percentage of a model's enabled intents that have test
utterances in the default test set. Before testing your model, ensure that there is at least
60% coverage. The higher the Test Coverage score, the more accurate the performance testing
results.

Your test coverage needs to be at least 60%, with at least 5 test utterances per intent, in
order for the system to provide an optimal confidence threshold during batch testing. For
more information about the confidence threshold, see [NLU model settings](https://servicenow-prod.fluidtopics.net/9DCue~pqIqM_bEihwOdQIQ "Change your NLU model's name, description, or confidence threshold on the Settings page of the model overview.").


Aim to have about 10 percent of a model's test utterances marked as "not relevant", meaning
that there is no intent associated. This helps assess how the model handles irrelevant
utterances which should not have any intent predicted. For more information about irrelevant
utterances, see [Irrelevance detection in NLU](https://servicenow-prod.fluidtopics.net/NwbaT5ofR5JghXSGZ~0gYg "Keep Virtual Agent chats focused with Irrelevance detection. Use the Irrelevance detection feature to train your NLU model to avoid making predictions for utterances that are not relevant.").

## Use the test set {#nlu-test-set-creation-management__section_dn1_jlt_dwb}

To use the default test set from the Test and publish your model
phase, see [Test and publish your model](https://servicenow-prod.fluidtopics.net/DPaBLeE6TiKefL~QJbRX8Q "Assess the performance of your NLU model to identify areas for improvement. Then publish your model to make it available to other applications such as Virtual Agent.").

To use the test set in Multi-model Batch Testing, see [Multi-model Batch Testing](https://servicenow-prod.fluidtopics.net/4P~K156R15CxwKnHcMcKpw "Test multiple Natural Language Understanding (NLU) models against a large set of utterances to evaluate the performance of the models. Add test sets, test multiple models, and see test results.").

## Characteristics of default test sets {#nlu-test-set-creation-management__section_zyj_2zr_dwb}

When an instance is upgraded, default test sets are created for any existing models that
don't already have them.

When you copy a model using Duplicate this model, the original's
default test set is copied into the new model. For more information, see [Duplicate an NLU model](https://servicenow-prod.fluidtopics.net/neXAXjGY~zWHg7r5Fzn6Mg "Duplicate an existing Natural Language Understanding (NLU) model to create a new one. Duplicating a model copies the settings and contents of the original model, including its default test set.").

The utterances in the test set shouldn't be the same as the utterances in the training
set.

Default test sets can't be deleted separately from their models.

Test set utterances should be in the same language as their model.

Test sets are available for Virtual Agent
or AI Search models.

## Downloading or moving default test sets {#nlu-test-set-creation-management__section_pgc_5zr_dwb}

Default test sets can be downloaded or moved as follows.

* Default test sets can be separately downloaded in CSV format. To download the test set, from the model's overview page navigate to Build and train your modelTest set tab. Select Download test set.  
  Note:  
  Test sets that are downloaded from Download test set contain test utterances and their expected intents, but not the sources.
* Default test sets can be moved with update sets. When you add an NLU model to an update set, its default test set is added, including test utterances, expected intents, and sources. For more information, see [Add an NLU model to an update set](https://servicenow-prod.fluidtopics.net/vFSQ_JaPec8y5PpidN_7kw "Use update sets to move your Natural Language Understanding (NLU) models from one instance to another. Update sets include all records needed for your model to function on the target instance.").
* When using the Export model as CSV function in the All existing models table, the default test set is not included. For more information, see [Export an NLU model](https://servicenow-prod.fluidtopics.net/HULf6BiGwoOuLHMmZmPaIQ "Export a Natural Language Understanding (NLU) model to create a CSV file of the intents and utterances. You can then use the CSV file to edit, share, and import.").
{#nlu-test-set-creation-management__ul_upw_b1s_dwb}

*[\>]: and then