Issue Auto Resolution tuning options
Summarize
Summary of Issue Auto Resolution tuning options
The Issue Auto Resolution (IAR) tuning options in the NLU Workbench allow ServiceNow customers to optimize their IAR model based on specific business goals: precision, automation, or a balance of the two. This tuning impacts the model’s confidence threshold, affecting how many incidents are automatically resolved and the accuracy of those resolutions.
Show less
Key Features
- Tuning Goals: Customers can select from three tuning goals in the Analyze step of IAR tuning:
- Precision: The model predicts only when confidence is high, minimizing errors but resolving fewer incidents. This is the default and recommended setting for ITSM models.
- Automation: The model predicts at a lower confidence threshold, increasing the number of resolved incidents but potentially increasing error rates.
- Balance: The model balances between precision and automation to moderate both resolved incident volume and error rates.
- Metrics Provided:
- Match Rate: The ratio of correctly predicted intents to total predictions, averaged across all intents except NOINTENT.
- IAR Coverage: The percentage of incidents resolved by the model based on predictions above the confidence threshold, which may include some errors.
- Interactive Analysis: Customers can compare projected match rates and coverage for different tuning selections before committing.
- Intent Management: After tuning feedback, customers can manage intent-to-topic mappings by mapping unmapped intents to Virtual Agent topics via the IAR Admin Console.
Using the Tuning Options
To tune the model, users with the nluadmin role navigate to All > NLU Workbench > Models, select the Issue Auto Resolution tab and the specific model, then proceed from the Feedback step to the Analyze step. Here, they select tuning goals and review projected match rate and coverage outcomes. They can also drill down into intent-specific results and adjust intent mappings if needed.
Once the optimal tuning goal is chosen, users save their choice and proceed to tune and publish the model to apply the changes.
Why This Matters
This tuning capability empowers ServiceNow customers to align IAR model behavior with their operational priorities—whether minimizing errors, maximizing automation, or balancing both. It helps improve incident resolution efficiency while controlling risk, ensuring the model's predictions support business needs effectively.
When you are tuning your Issue Auto Resolution model in NLU Workbench, you can adjust the output for several goals: precision, automation, or a balance of the two. Compare how your choice of tuning options affect match rate and coverage, before committing.
Summary usage
By default, Issue Auto Resolution tuning in NLU Workbench optimizes for precision. Depending on your business requirements, you can also tune the model for other objectives. In the Analyze step of Issue Auto Resolution tuning, the tuning goals list enables you to adjust for Precision, Automation, or Balance. As you select one of these options, the projected Match rate and IAR coverage percentages change accordingly, so you can compare possible outcomes.
- Navigate to .
- Select the Issue Auto Resolution tab, then select the model name. The tuning experience opens to step 1 (Feedback) initially.
- Provide feedback, then select the Analyze button. Step 2 (Analyze) opens.
- In the section Here are your tuning options and projected results, using the list You can tune for precision, automation, or balance, select options to see projected scenarios. You can also select the link Learn about tuning goals to open the following window.
Precision
When tuned for precision, the IAR model makes predictions only when its confidence is relatively high. This results in lower error rates, but also in fewer incidents resolved.
Precision is the recommended tuning option for the IAR ITSM model, so this option is selected by default.
Automation
When tuned for automation, the IAR model makes predictions at a lower confidence threshold. This results in more predictions, so more incidents are resolved. However, higher error rates are possible.
Balance
When tuned for balance, the IAR model attempts to strike a balance between precision and automation.
Match rate
The match rate is defined as the number of Incidents where the intent was predicted correctly, divided by the number of predictions for that intent. This ratio is averaged across all intents except for NO_INTENT.
IAR Coverage
Coverage is defined as the percentage of Incidents that would be resolved because the model was able to make predictions above its confidence threshold. The predictions may contain some errors.
Using tuning options
Select several different tuning options to compare projected results. Depending on the option you select, the system presents scenarios for projected Match rates and IAR coverage rates. Also the system displays how much these rates change according to your selection.
Review further information in the Here's a detailed breakdown section of Analyze. Here you can drill down into results that are specific for each intent in the model.
Note that the intents are grouped into mapped and unmapped intents, depending on whether they have been mapped to Virtual Agent topics. After providing feedback in IAR Tuning, you may wish to activate some intent-to-topic mappings. To do so, expand See unmapped intents, then select the Map more intents button. This opens the IAR Admin Console.
When you have decided the optimum tuning option for your requirements, select the Save choice button in the Learn about tuning goals window. Then, select the Tune and publish model button to advance to the next step.