Health Log Analytics configuration preferences

  • Release version: Yokohama
  • Updated January 30, 2025
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Health Log Analytics configuration preferences

    This document outlines essential configuration preferences for Health Log Analytics in the Yokohama release, focusing on MID Server settings, log ingestion capabilities, throughput expectations, and log source retention policies. These guidelines help ServiceNow customers optimize their log ingestion performance and manage log retention effectively.

    Show full answer Show less

    Key Features

    • MID Server Log Ingestion: The log ingestion capability must be enabled on the MID Server. Enabling all capabilities on a MID Server includes this by default.
    • Dedicated MID Servers: It is recommended to use dedicated MID Servers for log ingestion to improve performance.
    • Recommended MID Server Specifications: For optimal Health Log Analytics performance, configure MID Servers with 8 CPUs, 32 GB RAM, up to 10 Gbps network bandwidth, up to 4,750 Mbps EBS bandwidth, and an 8,192 MB Java heap size.
    • Minimum MID Server Requirements: At least 4 CPUs, 16 GB RAM, and 8 GB Java heap size are required to stream logs effectively.
    • Log Ingestion Throughput: Throughput varies based on log message size, with estimates provided for 300 bytes, 1.1 KB, and 2 KB log messages on a Washington DC instance.
    • Ulimit Settings and Throughput: Adjusting the ulimit for open files on the MID Server affects network throughput. Different queue types (in-memory or disk-based) and log sizes have corresponding throughput rates.
    • Lightning gRPC Client: Starting August 2024, enabling the Lightning gRPC client can increase log streaming speeds by up to six times. This requires manual activation and configuration.
    • Data Input Limitations: By default, each MID Server can handle up to 10 data inputs, configurable per MID Server or globally.
    • Java Runtime Environment: MID Servers with Health Log Analytics capability must run on JRE 11 or higher, compatible with both FIPS and non-FIPS modes.
    • BC-FIPS Compliance: Health Log Analytics requires upgrade to version 34.0.37 (December 2024) to support BC-FIPS version 2.0.
    • Log Source Retention: Default retention is 3 days per log source and cannot be modified unless using Health Log Analytics version 22.0.12 (December 2021) or later from the ServiceNow Store, which allows retention policy adjustments per source or for multiple sources.

    Key Outcomes

    • By following these configuration preferences, customers can ensure efficient log ingestion performance tailored to their MID Server capabilities and network environment.
    • Applying the recommended hardware and software settings supports scalable log streaming throughput aligned with organizational needs.
    • Enabling the Lightning gRPC client offers a significant performance boost for log streaming.
    • Understanding and configuring log source retention policies allows customers to manage storage and compliance requirements effectively.

    Commonly used settings for Health Log Analytics properties and general configuration.

    MID Server settings

    • The MID Server log ingestion capability must be enabled.
      Note:
      Enabling All capabilities on the MID Server includes enabling the log ingestion capability.
    • Use dedicated MID Servers for log ingestion whenever possible.
    • To enable MID Servers to run multiple products, Health Log Analytics must have at least the Java Virtual Machine (JVM) memory setting for the standard product for each MID Server thread configuration.
    The preferred MID Server settings for Health Log Analytics are:
      • CPUs: 8
      • RAM: 32 GB
      • Network Bandwidth: Up to 10 Gbps
      • EBS Bandwidth: Up to 4,750 Mbps
      • Maximum Java heap size for MID Server: 8,192 MB
      With the above specifications, the expected log ingestion throughput on a Washington DC instance is as follows:
      • For a log message of 300 bytes: 20,000
      • For a log message of 1.1 KB: 12,300
      • For a log message of 2 KB: 7,970
      The minimum requirements for streaming logs to Health Log Analytics are:
      • CPUs: 4
      • RAM: 16 GB
      • Java heap size for MID Server: 8 GB

      For general information, see: MID Server system requirements.

    • To increase log ingestion throughput, you can either increase the ulimit or the network bandwidth, or decrease the size of the logs being streamed. The ulimit setting can be configured on an individual MID Server. However, the correlation between the ulimit and the throughput can’t be modified.

      The following table lists the ulimit settings for open files relating to network throughput on the MID Server. It shows the size of the logs being streamed from the MID Server to the agent, and the gRPC streaming rate equivalent to the throughput.

      Table 1. Ulimit settings in relation to throughput
      Queue Type Log line size gRPC rate
      In Memory Queue 300 bytes 18,000
      In Memory Queue 1.1 KB 13,000
      In Memory Queue 2 KB 10,000
      Disk-based Queue 300 bytes 11,000
      Disk-based Queue 1.1 KB 5,000
      Disk-based Queue 2 KB 3,000

      Starting from the August 2024 release, you can enhance MID Server communication with the ServiceNow instance by using the Lightning gRPC client, which can increase log streaming speeds to Health Log Analytics by up to six times. The Lightning gRPC client requires manual configuration to activate. For more information, see the Lightning gRPC client - Enabling the new MID gRPC streaming architecture [KB1648419] article in the Now Support Knowledge Base.

    • By default, the number of data inputs per MID Server is limited to 10. You can configure this limitation for an individual MID Server or for all MID Servers.
    • Both in FIPS and non-FIPS mode, MID Servers with Health Log Analytics capability must run on the Java Runtime Environment (JRE) 11 or above.
      Note:
      To support BC-FIPS version 2.0, Health Log Analytics requires an upgrade to version 34.0.37, December 2024.

    Log source retention settings

    By default, log retention per source is set to three days. This setting can't be modified.

    When using Health Log Analytics application, Version 22.0.12 - December 2021 and later, available from the ServiceNow Store , you can modify the log retention policy per source or for multiple sources together. For more information, see Modify the log source retention period.