Log anomaly detection
Summarize
Summary of Log Anomaly Detection
Log Anomaly Detection in ServiceNow's Health Log Analytics automatically identifies unusual patterns in your log data to help detect emerging IT issues before they impact users. When anomalies are found, the system generates alerts that integrate with the ServiceNow Event Management application, enabling proactive incident management.
Show less
Key Features
- Lexical Keywords: The system scans logs for predefined keywords such as "crashed" or "failed" that indicate potential problems. Thresholds based on normal keyword frequency trigger alerts when exceeded. Customers can manage global keywords or customize keywords for specific source types to tailor detection.
- Alert Metrics: Multiple log metrics are monitored for anomalies. Operators can provide feedback on alerts to prioritize significant issues or mute irrelevant alerts, reducing noise. Muted metrics stop generating alerts but can be reactivated as needed.
- Log Correlators: Correlators identify relationships in log data by matching keys or values across multiple alerts, such as detecting simultaneous warnings from the same network device interface. This helps uncover broader system issues.
- Advanced Alert Filtering: Customers can create custom filters to refine alert conditions, dropping non-critical alerts and minimizing noise. Filters can be tested, updated, and activated dynamically.
- Custom Alerting Rules: Users can define specific alert rules for log metrics with customizable thresholds and properties to generate tailored alerts aligned with their operational needs.
Practical Benefits
ServiceNow customers using Log Anomaly Detection can expect improved situational awareness through early detection of log anomalies, reduced alert noise via filtering and feedback mechanisms, and enhanced correlation of related issues for faster troubleshooting. Customizable keywords, alert rules, and filters provide flexibility to adapt detection to unique environments, enabling more effective IT operations and incident response.
Health Log Analytics discovers patterns in your log data and learns their unique data behavior. When it finds an anomalous pattern, it sends an event to the ServiceNow® Event Management application. You can use these predictive alerts to handle emerging IT issues before they impact users.
Health Log Analytics uses various methods to detect anomalies.
Lexical keywords
Health Log Analytics scans your logs for words that can indicate important issues. Lexical keywords such as "crashed" or "failed" signal a condition that can merit attention.
The system sets a threshold for each lexical keyword that is based on what it considers the normal occurrence pattern and frequency of that keyword in your logs. When it scans your logs, it finds all occurrences of the keyword. If the number exceeds the threshold, it generates an alert.
For information about managing global keywords, see Add, edit, or delete Health Log Analytics lexical keywords. To create or delete keywords for a specific source type, see Configure source type capabilities.
Alert metrics
Health Log Analytics monitors multiple metrics as a means to detect anomalies. When it identifies an anomalous pattern for a metric, it generates an alert.
Operators can provide feedback about the generated alerts. Their feedback "teaches" Health Log Analytics that a specific alert is significant or irrelevant to them. The application then either raises the priority of the alert metric or mutes it to reduce noise.
When a metric is muted, Health Log Analytics removes the current alert and any other alerts based on that metric from the feed. It also stops generating new alerts from that metric. You can reactivate a muted alert metric. For more information, see Restore normal importance to an alert metric.
Correlations
Log correlators are keys or values in log data that detect correlations between alerts. For example, a log correlator could detect when the interface ID of a particular network device occurs simultaneously in multiple warnings across different service instances. For more information, see Using log correlators to detect relationships in log data.
Advanced alert filtering
Add advanced log alert filters to scan alerts for conditions that you specify. The filters reduce noise by dropping alerts that do not indicate a significant issue. While developing a filter, you can test, update, publish, or activate the filter at any time. For more information, see Create advanced log alert filters.
Custom alerting rules
Define a Log Analytics alert rule when you encounter log data that should generate an alert. The alert rule generates an alert for a specified metric with a threshold that you specify and sets the properties of the generated alert. For more information, see Add a Log Analytics alert rule.