---
sourceDocument: Xanadu Enable AI
sourceDocumentLink: https://servicenow-prod.fluidtopics.net/r/xanadu/intelligent-experiences

 Release :

    - xanadu

ft:locale :

    - en-US

ft:publication_title :

    - Xanadu Enable AI

ft:clusterId :

    - platai

bundleId :

    - platai

workflow :

    - Platform


---

# Multi-model Batch Testing

# Multi-model Batch Testing {#ariaid-title1}

* Release version: Xanadu
* 
* Updated August 1, 2024
* 
* ![](https://www.servicenow.com/docs/portal-asset/ico-clock) 2 minutes to read

Test multiple Natural Language Understanding (NLU) models against a large set of
utterances to evaluate the performance of the models. Add test sets, test multiple models, and see
test results.

## Summary usage {#batch-testing-nlu__section_fds_blv_vnb}

Use Multi-model Batch Testing to create and upload test sets comprised of utterances and their
expected intents. You can then run tests against your NLU models.

Multi-model Batch Testing works with models for all supported NLU languages. See [NLU language support](https://servicenow-prod.fluidtopics.net/sN6WhVtvMDOpcwpT92aYeg "The NLU Workbench application provides support for creating NLU models in different languages for use in other applications, such as Virtual Agent.").

## Installation {#batch-testing-nlu__section_kxx_jjd_lpb}

Multi-model Batch Testing is part of the NLU Workbench - Advanced Features app
available on the ServiceNow® Store.

To use Multi-model Batch Testing, ensure that the NLU Workbench - Advanced
Features (com.snc.nlu.workbench.advanced) plugin is active on your instance. For more
information, see [Install NLU Workbench - Advanced Features](https://servicenow-prod.fluidtopics.net/00W_IkegTREyjMfI15wftA "You can install the NLU Workbench - Advanced Features application (com.snc.nlu.workbench.advanced) if you have the admin role.") and [Activate the NLU Workbench](https://servicenow-prod.fluidtopics.net/rCLbf7GXrT3OPe5NypV1Iw "Activate the following plugins to activate the NLU Workbench.").

## Test sets {#batch-testing-nlu__section_npp_clv_vnb}

Test sets are lists of utterances and matched intents. Create a test set by using
a table in a CSV or XLSX (Excel workbook) file. The table should contain two columns: one for
utterances, and one for the expected intent. Your test set can include up to 10,000 rows.

To get the most out of testing your NLU models, your test sets should include utterances that
the model is likely to encounter from your users. Test utterances should be in the same language
as the model to be tested. The test set should also include utterances with no expected intents.
Including utterances with no expected intent helps assess your model's ability to detect
utterances which are irrelevant and shouldn't have any intent predicted.  
By including these types of utterances, the test better assesses the model's ability to perceive intents and respond to your users. If your test set does not cover at least 60% of the intents of the models, you can still run the test but the recommended threshold may not be optimal.  
Note:  
Certain test utterances are skipped during the test if their expected intent does not match any intents in the models.

To create a test set, see [Create a test set](https://servicenow-prod.fluidtopics.net/7ITXAPw3IYS5GN58ahd4eQ "To create or add to an NLU test set, you can upload a file of test utterances matched with correct intents. Use the test set to assess the performance of your model.").

After you have a test set, you can test trained NLU models. To begin testing, see [Run a multi-model batch test](https://servicenow-prod.fluidtopics.net/ofy9ObT_b_VqGM~skznW2w "Test multiple Natural Language Understanding (NLU) models against a test set. Evaluate the quality of your models and refine them to improve intent prediction.").

After running a test, your results appear on the Test results page.

## Test results {#batch-testing-nlu__section_syg_xct_lpb}

The Test results page lists your completed and in-progress tests. At a
glance, the results page shows the models tested against, the number of utterances, and
prediction percentages.

To see the details of a test result, click the name of the test set.

The Overview page shows summary information about the results and
includes a graphic with a breakdown of predictions.

The Intents that need attention (Current model) shows the top 5 missed
and incorrect intents. Click the intent name to drill down into the test utterances that were
predicted incorrectly. Use this information to improve the model.

The Detailed results tab lists information about each utterance that
was tested. From here, you can see the prediction outcome and confidence per model for each
utterance. Filter the results by using the search bar or interacting with the filter tools and
column headers.

You can also export the test results to a CSV file by clicking Export.
The file includes the same columns as the detailed results page.

For more information on understanding your test results, see [Test and publish your model](https://servicenow-prod.fluidtopics.net/dhylMV9ceAvrFnKgnGYwIA "Assess the performance of your NLU model to identify areas for improvement. Then publish your model to make it available to other applications such as Virtual Agent.").
* **[Create a test set](https://servicenow-prod.fluidtopics.net/7ITXAPw3IYS5GN58ahd4eQ)**   
  To create or add to an NLU test set, you can upload a file of test utterances matched with correct intents. Use the test set to assess the performance of your model.
* **[Run a multi-model batch test](https://servicenow-prod.fluidtopics.net/ofy9ObT_b_VqGM~skznW2w)**   
  Test multiple Natural Language Understanding (NLU) models against a test set. Evaluate the quality of your models and refine them to improve intent prediction.