Data collector Performance Analytics properties
Summarize
Summary of Data collector Performance Analytics properties
Data collector properties in Performance Analytics allow you to configure limits that control the data collection process, ensuring system performance and stability. These properties have default values suitable for most environments, and adjusting them should be done cautiously, ideally with expert guidance, as increasing limits can impact system performance.
Show less
Key Properties and Their Practical Use
- Script Timeout (com.snc.pa.dc.scripttimeout): Limits the maximum time (seconds) a script runs per record during data collection. Exceeding this causes the record to be skipped. Simplify scripts before increasing the default of 30 seconds.
- Query Time Limit (com.snc.pa.dc.querytimelimit): Sets the maximum duration (minutes) for a single query before logging a warning. Default is 60 minutes.
- Maximum Rows per Indicator Source:
- Non-Optimized (com.snc.pa.dc.maxrowcountindicatorsource): Caps records collected per indicator source at 50,000 by default. If exceeded, no indicators are collected from that source.
- Optimized (com.snc.pa.dc.hsql.maxrowcountindicatorsource): Applies when optimized data collection is enabled, with a default limit of 1,000,000 records per indicator source. Overrides the non-optimized property.
- Breakdown Elements Limits:
- Non-Optimized (com.snc.pa.dc.maxbreakdownelementslimit): Limits breakdown elements per source to 10,000. Exceeding disables collection for that source. Behavior varies if nested collection is enabled.
- Optimized (com.snc.pa.dc.hsql.maxbreakdownelementslimit): No default limit, but setting one is recommended if performance issues arise with large breakdown sources.
- Two-Level Breakdown Matrix Limits:
- Non-Optimized (com.snc.pa.dc.maxbreakdownelementslevel2limit): Default limit is 1,000,000 elements in a breakdown matrix; exceeding this stops collection.
- Optimized (com.snc.pa.dc.hsql.maxbreakdownelementslevel2limit): No default limit; set with caution if performance issues occur.
- Maximum Error Count (com.snc.pa.dc.maxerrorcount): Caps errors per data collection run at 500 before stopping the job. Review and fix scripts if this limit is reached.
- Maximum Records per Snapshot (com.snc.pa.dc.maxrecords): Limits the number of sysids stored in a single Snapshot record to 5,000 by default. Excess records are truncated, and increasing this may affect performance.
Key Outcomes for ServiceNow Customers
Configuring these properties allows you to tailor Performance Analytics data collection to your environment’s scale and complexity while safeguarding system performance. Understanding the distinction between optimized and non-optimized data collection properties is crucial, especially after upgrades activating optimized collection by default. Properly set limits prevent data loss from skipped collections and avoid performance degradation during data collection runs.
When adjusting properties, always consider the impact on system resources and consult with ServiceNow experts or partners to find alternatives that avoid compromising performance. Monitoring warnings and errors related to these properties helps maintain reliable and efficient data collection processes.
Data collector properties enable you to configure various limits for Performance Analytics data collection. The properties are configured to safeguard the data collection process. The default values are appropriate for most environments.
| Property | Description |
|---|---|
| com.snc.pa.dc.script_timeout | The maximum time in seconds that a script is allowed to run during a data collection cycle, such as an indicator source script or a breakdown script. This limit applies individually for each record processed by the data collection job. If a script exceeds this limit, the data collection job skips the current record. If your scripts frequently reach the default limit, simplify the scripts as much as possible before modifying this property.
|
| com.snc.pa.dc.query_time_limit | The maximum duration in minutes that a single query for a data collection job can run before a warning is logged.
|
| com.snc.pa.dc.max_row_count_indicator_source | The maximum number of records that a job can collect from a single indicator source. Important:
This property applies
only to jobs that do not use optimized data collection. If optimized data collection is enabled, which is the default condition, see the property
com.snc.pa.dc.hsql.max_row_count_indicator_source. This limit applies separately to each indicator source included in a data collection job. The number of indicators associated with each indicator source does not affect this limit. For example, if a data collection job collects scores for 12 indicators from three indicator sources, the job can collect a maximum of 150,000 records by default: 50,000 from each indicator source. Warning:
|
| com.snc.pa.dc.hsql.max_row_count_indicator_source | The maximum number of records that a job can collect from a single indicator source. Important:
This property applies only to jobs that use optimized data collection. Otherwise, com.snc.pa.dc.max_row_count_indicator_source applies. For more information, see Optimizing data collection. This limit applies separately to each indicator source included in a data collection job. The number of indicators associated with each indicator source does not affect this limit. For example, if a data collection job collects scores for 12 indicators from three indicator sources, the job can collect a maximum of 3 million records by default: 1 million from each indicator source. On upgrade to a later version than San Diego, the optimized data collector becomes enabled. This property then overrides com.snc.pa.dc.max_row_count_indicator_source. The value of the new property is set to the default 1 million, unless the value of com.snc.pa.dc.max_row_count_indicator_source was more than 1 million. In the latter case, the original property value is retained. Warning:
|
| com.snc.pa.dc.max_breakdown_elements_limit | Maximum number of breakdown elements retrieved by data collection for each breakdown source. If the number of elements on a breakdown source exceeds this limit, the data collector does not collect scores from that breakdown source. Important: This property applies only to jobs that do not use optimized data collection. For optimized data collection, see the property com.snc.pa.dc.hsql.max_breakdown_elements_limit. Also
see Optimizing data collection. This limit applies separately to each breakdown source included in a data collection job. The number of breakdowns associated with each breakdown source does not affect this limit. For example, if a data collection job collects scores for 12 breakdowns from three breakdown sources, the job can collect a maximum of 30,000 records by default: 10,000 from each breakdown source.
Warning: Increasing this limit increases the processing load on the node. This property is handled differently depending on whether nested collection is enabled: If nested collection is enabled, only elements with non-null scores count against this limit. The count is taken as the data collection job runs. When the number of elements with scores reaches this limit, the breakdown source is disabled and the scores cleared from memory. When nested collection is not enabled, all elements of a breakdown source are counted against the limit. The count is taken before any scores are collected. Consider a case where the default limit of 10,000 elements is applied to a breakdown source with 1 million elements of which 5,000 have scores. Under nested collection, the job collects the scores for those 5,000 elements. With nested collection turned off, the job does not collect any scores for this breakdown source.
|
| com.snc.pa.dc.hsql.max_breakdown_elements_limit | Maximum number of breakdown elements retrieved by data collection for each breakdown source. If the number of elements on a breakdown source exceeds this limit, the data collector does not collect scores from that breakdown source. Important: This property applies only to jobs that use optimized data collection. Otherwise, com.snc.pa.dc.max_breakdown_elements_limit applies. For more information, see Optimizing data collection. This limit applies separately to each breakdown source included in a data collection job. The number of breakdowns associated with each breakdown source does not affect this limit. By default, no limit is set on the number of breakdown elements. If you experience performance issues on jobs with breakdown sources with large numbers of elements, consider setting a limit in consultation with Customer Service and Support. On upgrade to a later version than San Diego, optimized data collection is activated. This property then overrides com.snc.pa.dc.max_breakdown_elements_limit.
|
| com.snc.pa.dc.max_breakdown_elements_level2_limit | Maximum number of breakdown elements allowed in a matrix of two breakdowns. If this limit is exceeded, the breakdown elements in the matrix are not collected. Important: This property applies only to jobs that do not use optimized data collection. For optimized data collection, see the property com.snc.pa.dc.hsql.max_breakdown_elements_level2_limit.
Also see Optimizing data collection. For example, with the default limit of 1 million elements, a first-level breakdown with 10,000 elements, and a second-level breakdown with 500, the resulting matrix of 5 million elements is not collected. If the
first-level breakdown had only 1,000 elements, the resulting 500,000 elements would be collected. Note:
This limit is also affected by the com.snc.pa.dc.max_breakdown_elements_limit property. No
breakdown matrix is collected where one of the breakdown sources exceeds that limit.
|
| com.snc.pa.dc.hsql.max_breakdown_elements_level2_limit | Maximum number of breakdown elements allowed in a matrix of two breakdowns. If this limit is exceeded, the breakdown elements are not collected. Important: This property applies only to jobs that use optimized data collection. See Optimizing data collection. By default, no limit is set on the number of breakdown elements in a matrix. If you experience performance issues on jobs with breakdown sources with large numbers of elements, consider setting a limit in consultation with Customer Service and Support. Note:
This limit could be affected by the com.snc.pa.dc.hsql.max_breakdown_elements_limit property. If that limit is set, no breakdown matrix is collected where one of the breakdown sources exceeds that
limit.
|
| com.snc.pa.dc.max_error_count | The maximum number errors that may occur for a single data collection job run before data collection is stopped. Errors during data collection usually occur due to an invalid script, or when encountering the script timeout limit. If you encounter this limit, review any scripts that run during data collection to ensure that they are valid and perform as expected.
|
| com.snc.pa.dc.max_records | Maximum number of sys IDs that are stored in a single Snapshot [pa_snapshots] record. A Snapshot record is created for each collected score, and a field in this record contains all the sys IDs of the records that
contribute to the score. If this limit is exceeded, a Snapshot record is created with the maximum number of records. This limit applies only when Collect records is selected for an indicator. For example, say you are using the default limit of 5,000 and you run a job where 5,010 records contribute to the score. The system creates a Snapshot record with a comma-separated list of the first 5,000 sys IDs in the sys_id field. Generally, the default limit provides enough detail into collected records. Increasing this limit may impact performance during data collection or when performing operations on the Snapshot table.
|