ITSM Predictive Intelligence Workbench dashboard
Summarize
Summary of ITSM Predictive Intelligence Workbench Dashboard
The ITSM Predictive Intelligence Workbench dashboard provides tools to measure the effectiveness of machine learning in automating IT business processes, specifically incident management. This dashboard allows users to monitor use case models, view key statistics, and demonstrate business value to stakeholders.
Show less
Important: With the Xanadu release, the ITSM Predictive Intelligence Workbench is being prepared for deprecation, with full deprecation expected in the Yokohama release. Customers should install the Task Intelligence for ITSM application (com.snc.itsmmltask) for continued functionality.
Key Features
- Model Workflow Tracking: Automated comments provide updates on model integration, changes, and retraining, allowing users to see the current status of workflows.
- Net Automation Thresholds: Users can view thresholds that indicate the difference between estimated net automation and underperformance, with daily calculations and notifications for breached thresholds.
- Performance Indicators: The dashboard includes various metrics such as Prediction Coverage and Prediction Precision, helping users assess the quality of predictions and the effectiveness of automation.
- Data Visualizations: Visual representations of various metrics, including predictions skipped and class distributions, allow for deeper insights into model performance.
Key Outcomes
By utilizing the ITSM Predictive Intelligence Workbench dashboard, users can:
- Effectively communicate the value of machine learning initiatives to stakeholders.
- Enhance automation capabilities through data-driven insights.
- Maximize prediction quality and overall process efficiency.
- Utilize new database views to analyze and filter data by use case, improving understanding of impact on metrics.
ITSM Predictive Intelligence Workbench provides the Predictive Intelligence for Incidents dashboard to enable you to measure the value of using machine learning to automate your IT business processes. Monitor use case models and view associated statistics. Effectively demonstrate business value to stakeholders with dashboard views.
Starting with the Xanadu release, ITSM Predictive Intelligence Workbench is being prepared for future deprecation. It will be completed deprecated and will no longer be supported from the Yokohama release. To get the latest experience for this functionality, you must install the Task Intelligence for ITSM application (com.snc.itsm_ml_task) plugin. For more information, see Task Intelligence for ITSM
For details, see the Deprecation Process [KB0867184] article in the Now Support Knowledge Base.
Know exactly what stage your model workflows are in with automated comments on specific ServiceNow® Performance Analytics indicators. Comments appear on the Actual Net Automation indicators. Model stages comments include, when the model is integrated, when an integration is removed, when an existing model is changed to a new model, and when retraining is complete.
You can view the Net Automation Threshold graph on the Monitoring Models tab. This threshold is a computation of the difference between estimated net automation and the underperformance property values. Thresholds are calculated daily and notifications are sent to stakeholders when thresholds are breached.
- Model is integrated
- Model integration is removed
- Existing model is changed to a new model
- Integrated model is retrained
End user and roles
| End user and goal | Required role | Benefits |
|---|---|---|
| Process architect, process owner, or machine learning advocate: Visualize operational and automation metric performance​. Communicate machine learning value to stakeholders to increase the automation capabilities of predictive intelligence. |
Predictive Intelligence Workbench manager or viewer [piwb_manager] [piwb_viewer] |
Ability to measure the value of using Predictive Intelligence to automate ITSM processes. |
Indicators
- Prediction Coverage for Incident
- The score for this indicator is calculated according to the formula:
if ([[Number of attempted predictions based on Created Today for Incident]]==0){0}else{[[Number of applied predictions based on Created Today for Incident]]/[[Number of attempted predictions based on Created Today for Incident]]*100}. - Number of predicted results based on final value date for incident
- The number of predicted results is based on the final value date for an incident. The score is measured daily as unit #.
- Net Automation for Incident
- The score for this indicator is calculated according to the formula:
[[Prediction Precision for Incident]]*[[Prediction Coverage for Incident]]/100. - Number of applied predictions based on Created Today for Incident
- Number of applied predictions based on Created Today for Incident is measured daily as unit #.
- Number of attempted predictions based on Created Today for Incident
-
Number of attempted predictions based on Created Today for Incident is measured daily as unit #.
The goal for this indicator is to maximize the quality of predictions.
- Prediction Precision for Incident
- The score for this indicator is calculated according to the formula:
if ([[Number of predicted results based on final value date for Incident]]==0){0}else{[[Number of successful predictions not skipped based on final value date for Incident]]/[[Number of predicted results based on final value date for Incident]]*100}. - Estimated Net Automation
- Estimated net automation measured daily as unit %.
- Number of successful predictions not skipped based on final value date for Incident
-
Number of successful predictions not skipped based on final value date is measured daily as unit #. It is the count on data source MLPredictorResults.FinalValueDate.Incident, which is using the table: ml_predictor_results.
- Number of Predicted for Incidents
-
Number of Predicted for Incidents is measured daily as unit #. It is the count on data source MLPredictorResults.CreatedToday.Incident, which is using the table: ml_predictor_results.
Breakdowns
Use Case.
Data visualizations
| Title | Type | Source table | Description |
|---|---|---|---|
| Predictions Skipped | Single Score
|
sn_piwb_itsm_conte_dbv_ml_predictor_results | % Estimated Precision. Estimated precision measured as unit %, based off the data the model was trained on. |
| Predictions Skipped | Line |
sn_piwb_itsm_conte_dbv_ml_predictor_results | Predictions Skipped (num). Number of ___, due to low confidence. |
| Class Distribution - Training Data | Bar |
sn_piwb_itsm_conte_dbv_ml_classes | Predictions Skipped (line). |
| % Estimated Precision | Single Score
|
sn_piwb_itsm_conte_dbv_ml_solutions | Predicted Correctly (num). Number of ___, comparing initial predicted value to final record value. |
| Classes Excluded due to Low Distribution | Single Score
|
sn_piwb_itsm_conte_dbv_ml_excluded_classes | Predicted Correctly (line). |
| Class Distribution - Actual | Bar |
sn_piwb_itsm_conte_dbv_ml_predictor_results | Predicted Incorrectly (num). Number of ___, comparing initial predicted value to final record value. |
| Predicted Correctly | Single Score
|
sn_piwb_itsm_conte_dbv_ml_predictor_results | Predicted Incorrectly (line). |
| Predicted Correctly | Line |
sn_piwb_itsm_conte_dbv_ml_predictor_results | Class Distribution, Training Data. Distribution of classes in data that the solution was trained on. |
| Predicted Incorrectly | Single Score
|
sn_piwb_itsm_conte_dbv_ml_predictor_results | Class Distribution, Actual. Current distribution of classes in live data. |
| Predicted Incorrectly | Line |
sn_piwb_itsm_conte_dbv_ml_predictor_results | Predicted Classes. Number of values the model can return as a prediction. |
| Prediction Classes | Single Score
|
sn_piwb_itsm_conte_dbv_ml_classes | Classes Excluded due. Number of values the model was not confident enough to return as a prediction, due to not enough data. |