Understanding ServiceNow Health Log Analytics (HLA)

Release version: Xanadu

Updated August 1, 2024

3 minutes to read

Summarize

Summarized using AI

Summary of Understanding ServiceNow Health Log Analytics (HLA)

ServiceNow Health Log Analytics (HLA) is an application designed to predict IT issues before they impact users by collecting, analyzing, and correlating machine-generated log data in real time. It detects deviations from normal behavior automatically and alerts IT operators of potential problems. HLA processes logs received via the MID Server and integrates seamlessly with the ServiceNow Event Management application for consolidated alert handling.

Show full answer Show less

Key Features

Log Data Processing: Supports any textual machine-generated logs including application, infrastructure, and network logs. Only UTF-8 encoded logs are supported; binary logs are not.
Flexible Data Ingestion: Logs can be streamed directly from servers, endpoints, or repositories like Splunk and Elasticsearch using connectors such as Rsyslog, Beats, and the MID Server.
Automated Data Structuring: Extracts key properties from logs (message, timestamp, host, severity, external IDs) and auto-maps logs into logical silos called Components to organize data efficiently.
Enrichment: Identifies variable elements, keywords, and contextual properties within log messages, enhancing the depth of analysis.
Machine Learning Analysis: Uses unsupervised machine learning to learn normal log behavior patterns, dynamically sets thresholds, detects anomalies instantly, and supports root cause identification.
Integration with Event Management: Automatically sends detected anomaly events and alerts to Event Management, consolidating all alerts in one interface for easier operator response.

What You Can Expect

With Health Log Analytics, ServiceNow customers can expect proactive IT issue detection by leveraging real-time log analysis powered by machine learning. The application enables faster problem resolution by:

Continuously monitoring diverse log sources without manual intervention.
Automatically structuring and enriching log data for meaningful insights.
Detecting anomalies and deviations from normal operations as they occur.
Providing actionable alerts directly within ServiceNow Event Management for streamlined incident handling.

This leads to improved operational uptime, reduced manual log analysis effort, and enhanced visibility into the IT environment’s health.

Health Log Analytics predicts IT issues before they affect your users. The application helps you solve problems faster by collecting, understanding, and correlating machine-generated log data in real time. It discovers any deviation from normal behavior as it happens and alerts you of possible issues.

Health Log Analytics receives and processes logs via the MID Server and sends events to the ServiceNow Event Management application.

Data that Health Log Analytics can process

Health Log Analytics can handle any kind of machine-generated textual log data. It can process application, infrastructure, and network logs, as well as other types of textual log data. Although a configuration management database (CMDB) can be helpful to generate high-quality events and alerts, it is not necessary.

Note:

Health Log Analytics supports only UTF-8 logs. The application does not support binary logs.
If you are sending logs in a language other than English, additional configuration may be required..

Architecture

Health Log Analytics collects logs streaming into your ServiceNow instance from endpoints or data lakes, such as Splunk and Elasticsearch. The instance receives the logs via the MID Server connector instance. Health Log Analytics identifies and triages anomalies in your log data using unsupervised machine-learning (ML) models. It then groups the anomalies together and applies further algorithms to help identify the root cause of the issue.

The following figure shows a setup using Rsyslog, Splunk, Filebeat, and Elasticsearch.

Health Log Analytics architecture. — Figure 1. Health Log Analytics architecture

Workflow

Health Log Analytics collects and processes log data automatically. It structures the data logically for operators to analyze, and generates meaningful alerts and suggestions that display in Event Management.

The diagram shows the Health Log Analytics workflow from collecting the data through sending an event or alert to Event Management.

Ingestion

This layer connects your environment to Health Log Analytics. You can stream your logs directly from servers and endpoints or from log repositories. The optional guided setup helps you create data input connectors for the following common data sources:

Rsyslog
Beats
Splunk
Elasticsearch
MID Server
TCP

Structuring

This layer deals with structuring log data and auto-mapping it to logical silos, called Components. Data structuring can be done automatically or manually.

The system auto-structures log data by extracting the following properties from incoming log messages: Message, Timestamp, Host, Severity, and External-IDs. It extracts explicit values, like "property-name" and "value is IP." and semantic ones such as length, number of English words, and variance.

Auto-mapping assigns log samples and metadata to the appropriate tags automatically. The system tries to map log lines by analyzing the source that streams the data. The mapping is based on agent hints and common transport header fields.

Enrichment

This layer handles identifying the variable parts of a log message.

It also identifies keywords and contextual properties. In the image, "WARN" and "Failed" are the keywords to track. "User," "source IP," and "port" are the contextual properties.

Analysis

In this layer, each log line is indexed. Health Log Analytics extracts properties from the inner log message that contribute to models of behavior that the system learns to expect. Anomalous behavior departs from this expected behavior. You can search for an event and its most significant properties for manual triaging.

Machine Learning (ML) and Artificial Intelligence (AI)

Health Log Analytics uses advanced unsupervised machine-learning algorithms to discover patterns within logs and learn their unique data behavior. It then sets dynamic thresholds based on the data signature in real time to detect issues when they first occur. When the system detects a deviation from the typical pattern, it sends an event to Event Management.

Alert in Event Management

Health Log Analytics sends events to Event Management. In Event Management, Health Log Analytics alerts appear in the All alerts list. This list enables operators to see alerts from the event and the Health Log Analytics alert type in a single location.