Microsoft Azure Event Hubs data input configuration fields
Summarize
Summary of Microsoft Azure Event Hubs Data Input Configuration Fields
This document outlines the configuration fields for setting up data inputs from Microsoft Azure Event Hubs. It provides essential details on basic, query, transport, and advanced configuration settings necessary for ServiceNow customers to effectively stream log data.
Show less
Key Features
- Name: Required field for identifying the data input.
- Description: Allows users to describe the data input configuration.
- Execute on: Choose between a specific MID Server or a MID Server cluster, with requirements for basic authentication.
- MID Server: Required when a specific MID Server is selected; supports only basic authentication.
- MID Server Cluster: Used for failover protection, ensuring tasks are distributed across available MID Servers.
- Service Instance: Required for binding log data; must be set to Operational status.
- Status: Displays the current state of the data input.
- Transport: Specifies that Microsoft Azure Event Hubs is utilized for log data streaming.
Key Outcomes
By configuring the data input, ServiceNow customers can seamlessly ingest log data from Microsoft Azure Event Hubs, ensuring robust log management and operational insights. Important configuration settings such as the query settings (starting date, max batch size) and transport settings (event hub namespace, event hub name, credentials) are crucial for establishing a reliable connection and efficient data processing. Advanced configurations further enhance performance and manage event processing according to specific needs.
Description of the fields on the Microsoft Azure Event Hubs data input configuration form.
Basic configuration
| Field | Description |
|---|---|
| Name | Name of the new data input. This field is required. |
| Description | Description of the data input. |
| Execute on | Option to select whether to use a specific MID Server or a MID Server cluster. This field is required. |
| MID |
(Only when the Execute on field is set to Specific MID Server) MID Server to which log data from Microsoft Azure Event Hubs is pulled.Note: This field is required.
|
| MID Server Cluster | (Only when Execute on is set to Specific MID Server cluster.) The MID Server cluster to which the log data is pulled. This field is required.The data input runs on a single MID Server in the cluster until that MID Server fails. The system then moves all the data input tasks to the next available MID Server in the cluster according to the configured order. Note: For more information about MID Server clusters, see Configure a MID Server cluster.
|
| Service instance | The service instance to which to bind the log data. Note: This field is required.If no relevant
service instance exists, Create an service instance and add CIs to it. Set the status of the new service instance to
Operational. |
| Field | Description |
|---|---|
| Status | Status of the data input. |
| Transport | Protocol used to stream the log data. This data input uses Microsoft Azure Event Hubs to stream log data to your instance. |
| Sources count | The number of log sources this data input has created. |
| Disabled since | The time when the data input stopped or failed. |
| Last log time | The time when the last log streamed in the data input. |
| Error message | The streaming error. This field is populated automatically. It displays only when a streaming error has occurred. |
| Field | Description | Example |
|---|---|---|
| From | Starting date and time for reading the data. Data older than this date and
time is not read. Note:
Setting this value to a past date might require the system
to read large amounts of data, causing congestion. This field is required. |
Now -1 week |
| Max batch size | The maximum number of events passed to a single process call. | 100 |
| Field | Description |
|---|---|
| Event Hubs namespace | The scoping container for the event hub or hubs. This field is required. |
| Event Hub name | The event hub from which to fetch log data. This field is required. |
| Event Hub credentials | Shared access authorization policy with which to authenticate to the event hub. This field is required. |
| Consumer Group name | The Consumer Group to use. This field is required. |
Advanced configuration
| Field | Description | Default value |
|---|---|---|
| Restore point rate | The number of events the data input can read before a restore point is saved. | 100 |
| Processor threads count | The total number of processor threads. | 2 |
| Operation timeout | The number of seconds to wait before timing out event hubs operations. | 120 |
| Receive idle timeout | The number of seconds to wait before timing out receive operations. | 60 |
| Prefetch count | The number of events received in advance of event hubs operations. | 500 |
| Default timezone | The default timezone if the log doesn't include timezone information. | GMT |
| Sub sample drop ratio | The number of events to batch together, out of which one will be discarded. This setting is used to reduce the number of fetched events. | -1 |
| Sub sample receive ratio | The number of events to batch together, out of which all but one will be discarded. This setting is used to decrease the number of received events. | -1 |
| Max length in bytes | The maximum length, in bytes, of events. | 32766 |
| Character encoding | The character encoding for this data input. | UTF-8 |
| Sleep interval | The interval, in seconds, to wait before querying again after a query has returned no events. | 60 |
| Polling interval | The interval, in seconds, to wait before polling for new events. | 0 |
| Drop if queue is full | Option to discard logs if there is a load on the MID Server. | False |