Configuration tips for Predictive Intelligence

  • Release version: Yokohama
  • Updated January 30, 2025
  • 4 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Configuration tips for Predictive Intelligence

    This guidance helps ServiceNow customers optimize Predictive Intelligence by addressing common issues in solution training and prediction. It emphasizes the importance of quality input data, proper configuration after instance cloning, and troubleshooting training or prediction failures.

    Show full answer Show less

    Input Data Recommendations

    • Use at least 30,000 records to train models for better accuracy.
    • Ensure data cleanliness to reduce noise and improve model precision.
    • Maintain data quality with valid input and output values for accurate predictions.
    • Use representative data distribution to enable generalized model predictions.
    • Split data approximately 80% for training and 20% for evaluation to validate model accuracy.

    Solution Training Tips and Troubleshooting

    • Stalled Training Status: Verify that the glide.servlet.uri property points to the correct instance URL, especially after cloning or initial provisioning.
    • Training Failures: Use the "Show Training Progress" link to identify issues. Ensure the sharedservice.worker user is active to prevent authentication failures.
    • Insufficient Data or Poor Distribution: Confirm the output field distribution is balanced. Increase the dataset size or exclude records with null input fields to improve model creation success.
    • Multilingual Data Handling: For datasets with multiple languages, either set the processing language to the predominant non-English language or create separate solutions filtered by language/region with corresponding processing languages.

    Solution Prediction Troubleshooting

    • If predictions fail with Java exceptions, review Predictive Intelligence logs and submit an incident with detailed error information.
    • When predictions do not apply to records but appear in REST API tests, check the confidence threshold for prediction outcomes. Adjust the precision and coverage settings on the ML Solution Definition to align thresholds with confidence levels and enable predictions.

    Instance Cloning Considerations

    • Include the mlartifacts table in cloning operations because it contains critical predictive intelligence components stored as attachments; omission leads to prediction failures.
    • After cloning, ensure the sharedservice.worker user is active and not locked out to avoid training failures.

    If you encounter issues during your solution training and solution prediction, follow these suggested resolutions.

    Input Data

    It is recommended to have at least 30,000 records to train your models with, but the accuracy of the model is determined by the input data.

    There are three primary factors that determine the quality of the input data used to train solutions:

    • Cleanliness: Sanitized data reduces noise, making the model more accurate.
    • Quality: The input and output should be valid and correct to train the model to make accurate predictions.
    • Distribution: Data that represents the entire dataset as a whole will result in a model that can make more generalized predictions.

    Most raw data sets contain dirty and unusable data. Reviewing your input sets before training is essential to keeping accurate predictive models.

    It is recommended to use approximately 80% of your input data to train your model and about 20% of the data to evaluate whether the model is accurate. You can compare the model's predicted results against the real values for the 20% of remaining data.

    Solution training

    Issue Resolution or suggested action
    The solution training remains in Waiting for Training status for too long, as the scheduler job is using an incorrect Glide callback instance URL. Ensure the glide.servlet.uri property in the Glide instance is set to the correct instance URL. This issue can occur when:
    • An instance is cloned from production, yet it still refers to the production URL for the glide.servlet.uri property.
    • The Glide instance is provisioned and runs the training for the first time.
    New categories have been added and aren't yet having an impact on training. This is expected behavior, as the new categories may not yet have sufficient data until the solution is retrained.
    The solution training fails.

    When the training fails, click the Show Training Progress related link on the solution screen to determine where the potential problem resides.
    The solution training fails due to user authentication. Navigate to System Security> Users and ensure the sharedservice.worker user is set to Active.
    The model training returns saying the model cannot be created. The training fails and shows the “Error while training solution” message. The training progress window shows this message: “Solution training failed as either the data used isn't sufficient or the input field isn't predictive of the output field." This issue can occur when the data quantity or the distribution of field values isn't sufficient for a model to build successfully. Follow these steps to troubleshoot:
    1. Ensure the distribution of the output field isn't skewed.
    2. Retrain the model by changing the date filters to use a larger amount of data.
    3. If the input fields are not fully populated, add a filter to remove null records.
    The solution has data in multiple languages but the coverage and precision results are poor.

    Use the following options to help improve your metrics.

    Option 1: Update the processing language of the solution to the most prominent non-English language.
    Note:
    English is applied by default for all datasets.
    Option 2: If there's sufficient data for each language/region:
    1. Add a filter criteria for a specific language/region where the primary language can be identified (Dutch, English, French, German, Japanese, or Spanish).
    2. Generate a solution for each language/region and apply the proper processing language to each solution.

    Solution prediction

    Issue Resolution or suggested action
    The prediction fails and returns a Java exception where the cause is unknown.
    1. Search for the exception in the Predictive Intelligence Glide logs.
    2. Submit an Incident record for Predictive Intelligence including all relevant details, such as the exception, the impacted instance, the solution name, and the input string.
    There is no prediction applied to the incident/case record but the prediction returns a value when tested in the Rest API Explorer. This can occur when the confidence of the prediction is less than the threshold required to make a prediction. After your solution is trained, use the following steps to confirm if your solution settings need adjusting.
    1. Navigate to System Web Services > REST > REST API Explorer to find the confidence level for the prediction. See Test a classification solution prediction.
    2. On your ML Solution Definition record, check the threshold set for your outcome class that was returned in the prediction by clicking on the name of the class. The Class page appears.
    3. Check the Estimated Precision and Estimated Coverage values. If the corresponding threshold is more than the prediction confidence of the outcome, this is the root cause for why you did not see any prediction.
    4. Adjust the class precision and coverage values to increase coverage or precision. See Tune a trained classification solution.

    Instance cloning

    Issue Resolution or suggested action
    After an instance is cloned, predictions for your existing solutions fail. The ML solution artifacts in the [ml_artifacts] table are stored in the [sys_attachment table]. If the [ml_artifacts] table isn't included in the clone when you run it, the predictions fail. Ensure your clone includes the machine-learning artifacts, as these are critical components of your Predictive Intelligence solution.
    After an instance is cloned, the solution training fails. As the cloning run proceeds, it is possible that the sharedservice.worker user has either been inactivated, locked out, or the user ID isn't set. Resolve these problems so that the solution training succeeds.