Microsoft Azure Log Analytics data input configuration fields

  • Release version: Washingtondc
  • Updated February 1, 2024
  • 5 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Microsoft Azure Log Analytics Data Input Configuration Fields

    This document outlines the configuration fields for setting up Microsoft Azure Log Analytics data inputs within ServiceNow. Proper configuration allows for efficient log data ingestion from Microsoft Azure into your ServiceNow instance, enhancing IT operations management capabilities.

    Show full answer Show less

    Key Features

    • Name: Required field for the data input name.
    • Description: Optional description of the data input.
    • Execute on: Choose between a specific MID Server or a MID Server cluster for log data processing.
    • MID Server: Select a specific MID Server that supports basic authentication; mTLS is not supported.
    • MID Server Cluster: Use only failover clusters for log ingestion, ensuring at least one server has fewer than 10 data inputs.
    • Application Service: Required binding of log data to an application service; create one if none exists.
    • Log Query: Custom queries can be configured for fetching logs, with a fallback to default settings if left empty.
    • Transport Protocol: Specifies the streaming method for log data.

    Key Outcomes

    By effectively configuring these fields, ServiceNow customers can:

    • Streamline log data ingestion from Microsoft Azure, improving visibility into operational health.
    • Utilize custom queries for tailored log data retrieval, enhancing data relevance.
    • Manage log data processing through configurable MID Servers or clusters, ensuring reliability and failover capabilities.
    • Optimize event processing with adjustable parameters such as worker counts and polling intervals to suit specific operational needs.

    Description of the fields on the Microsoft Azure Log Analytics data input configuration form.

    Basic configuration

    Field Description
    Name Name of the new data input. This field is required.
    Description Description of the data input.
    Execute on Option to determine whether to use a specific MID Server or a MID Server cluster.

    This feature is supported in the Health Log Analytics application, Version 26.0.17 - February 2023 and later, available from the ServiceNow Store.
    MID

    (Only when the Execute on field is set to Specific MID Server)

    MID Server to which log data from Microsoft Azure Log Analytics is pulled.
    Note:
    • You can select only MID Servers that support basic authentication. MID Servers that support mTLS are not listed.
    • The default maximum number of data inputs streaming logs to a single MID Server is 10. You can modify this number in the MID Server properties.
    • If log ingestion is not enabled for the selected MID Server, Health Log Analytics enables it automatically.
    This field is required.
    MID Server Cluster

    (Only when the Execute on field is set to Specific MID Server Cluster)

    The MID Server cluster to which the log data is pulled.

    The data input runs on a single MID Server in the cluster until that MID Server fails. The system then moves all the data input tasks to the next available MID Server in the cluster according to the configured order.

    This feature is supported in the Health Log Analytics application, Version 26.0.17 - February 2023 and later, available from the ServiceNow Store.

    Note:
    • Health Log Analytics supports only failover MID Server clusters. In these clusters, multiple MID Servers are grouped together for failover protection. When selecting a cluster from the data input form, the MID Server Clusters list displays only failover clusters.
    • The MID Server cluster must include only MID Servers that support basic authentication. mTLS is not supported for log ingestion.
    • Log ingestion must be enabled for each MID Server in the cluster. If log ingestion is not enabled for the active MID Server, Health Log Analytics enables it automatically.
    • The default maximum number of data inputs streaming logs to a single MID Server is 10. A cluster passes capacity validation if it contains at least one MID Server with fewer than 10 data inputs running on it, even when that MID Server is down.
    For more information about MID Server clusters, see Configure a MID Server cluster.

    This field is required.
    Application service The application service to which to bind the log data.
    Note:
    If no relevant application service exists, Create an application service and add CIs to it. Set the status of the new application service to Operational.
    This field is required.
    The following fields show read-only information:
    Field Description
    Status Status of the data input.
    Transport Protocol used to stream the log data.

    This data input uses Microsoft Azure Log Analytics to stream log data to your instance.
    Disabled since The time when the data input stopped or failed.
    Sources count The number of log sources this data input has created.
    Last log time The time when the last log streamed in the data input.
    Error message The streaming error.

    This field is populated automatically. It displays only when a streaming error has occurred.
    Table 1. Query settings tab

    Health Log Analytics uses the values set in these fields to generate the query for receiving log data from Microsoft Azure Log Analytics. The Log query field enables you to configure a custom query
    Field Description Example
    From Starting date and time for reading the data. Data older than this date and time is not read.
    Note:
    Setting this value to a past date might require the system to read large amounts of data, causing congestion.

    This field is required.

    		 Now -1 week
    Data source name The name of the table in Microsoft Azure Log Analytics where the data input fetches the log data. For more information, see the View table information section in the Microsoft Azure documentation.

    This field is required.

    		 ContainerLog
    Event time property name The Microsoft Azure Log Analytics field in which to detect the event time.

    This field is required.

    		 TimeGenerated
    Max documents per query The highest number of rows retrieved in each query. 500
    Columns to select Comma-separated list of column names to return.
    Note:
    This field is ignored when you provide a custom query.
    		 LogEntry,LogEntrySource
    Log query A custom query for receiving log data from Microsoft Azure Log Analytics.

    The settings in this field override those in all other fields on the Query settings tab except the From field. If the Log query field is empty, Health Log Analytics generates the query using the values set in the other fields.

    For the custom query, use the following JSON format:

    {"query": "query | where TimeGenerated > %s | take 500"}

    		 
    {
"query":"ContainerLog
                    | where LogEntry contains 'cartservice'
                    | where TimeGenerated > %s
                    | take 500",
"workspaces": ["defaultworkspace-3ab145ff-f9cd-433f-8533-d1b1ee24aee6-eus"],
"project": ["TimeGenerated", "LogEntry", "LogEntrySource"]
}
    Table 2. Transport tab
    Field Description
    Redirect Url The redirect URL of the access log application.

    The URL refers to the Microsoft Azure redirect_uri authorization property. For more information, see the Authorization code URL (GET request) section in the Microsoft Azure documentation.

    This field is required.
    Azure service principal credentials The credentials used to access Microsoft Azure resources.

    Choose Client Secret from the drop-down list.

    Note:
    You do not need Azure Enterprise Agreement (EA) credentials.

    This field is required.
    Workspace Id The Customer ID used to call the Microsoft Azure Log Analytics REST API.

    Advanced configuration

    Table 3. Advanced configuration form
    Field Description Default value
    Event Processor workers The number of concurrent event processing workers, where each worker processes a batch of events independently. 4
    Workers Queue Size The queue size of the Event Processor workers. 5
    Sub sample drop ratio The number of events to batch together, out of which one will be discarded. This setting is used to reduce the number of fetched events. -1
    Sub sample receive ratio The number of events to batch together, out of which all but one will be discarded. This setting is used to decrease the number of received events. -1
    Character encoding The character encoding for this data input. UTF-8
    Sleep interval The interval, in seconds, to wait before querying again after a query has returned no events. 60
    Polling interval The interval, in seconds, to wait before polling for new events. 0
    Drop if queue is full Option for selecting to discard logs if there is a load on the MID Server. False