Configure a data input for streaming log messages to your ServiceNow instance using a Splunk heavy forwarder. Data input configuration is an essential step in setting up the Health Log Analytics (HLA) application.
Before you begin
Role required: evt_mgmt_admin
About this task
This setup procedure is for streaming logs to your instance using a Splunk heavy forwarder. If you can't use a heavy forwarder, you can
use a universal forwarder instead. For more information, see the Splunk Universal Forwarder as a Shipping method
[KB0961378] article in the Now Support Knowledge Base.
Note: A MID Server that is down can cause a blockage in your Splunk pipeline. A full processing queue does not affect the
pipeline.
Note: All Splunk configuration files are located in the
$SPLUNK_HOME/etc/system/local/ folder. If a
configuration file that you need to modify doesn't exist, create it and save it
to this folder.
Procedure
-
Navigate to .
-
On the Data Inputs page, select New.
-
Choose the Splunk data input.
-
On the Getting Started tab, fill in the form
fields.
-
On the Outputs.conf tab, add the following stanzas to
the outputs.conf file to make the shipper forward log data
over the selected transport protocol on the selected port, and then select
Next.
Note: If you have already configured outputs, merge these lines with your
existing configuration.
-
On the Props.conf tab, edit the
props.conf file, and then select
Next.
-
Modify existing stanzas or add stanzas to mark source types, service instances, and hosts for forwarding to Health Log Analytics.
Note: For best results, mark only source types for forwarding.
When adding stanzas, use the following name formats:
-
Add the following line to the end of each stanza that you want to
forward to Health Log Analytics over either TCP or
UDP.
-
Add the following stanza to apply all the relevant transforms needed
for Health Log Analytics processing.
Note: Splunk enables you to anonymize sensitive data on the
cloned source type for the selected protocol. For more information,
see the "Anonymize data" section in the
Splunk documentation.
-
On the Transforms.conf tab, add the following stanzas to
the transforms.conf file, and then select
Next.
The third stanza clones the logs for further manipulation without affecting
your existing indexing. The remaining stanzas add the information needed to
enable correct
Health Log Analytics processing.
Note: You can
obfuscate sensitive data by adding a transform here and then modifying the
stanza of the cloned source type in the props.conf
file.
[accepted_keys]
#Custom field for preserving sourcetype
hla_sourcetype_preservation=_hla_sourcetype
#Store sourcetype in a custom field, since CLONE_SOURCETYPE overwrites it
[clone_for_hla_store_sourcetype]
SOURCE_KEY = MetaData:Sourcetype
REGEX = ^sourcetype::(.+)$
FORMAT = hla_sourcetype::$1
DEST_KEY = _hla_sourcetype
[clone_for_hla]
REGEX=.
DEST_KEY = _TCP_ROUTING
FORMAT = health_log_analytics
CLONE_SOURCETYPE=send_to_hla_tcp
#Only used in case of UDP forwarding
[clone_for_hla_udp]
REGEX=.
DEST_KEY = _SYSLOG_ROUTING
FORMAT = health_log_analytics
CLONE_SOURCETYPE=send_to_hla_udp
#Add metadata to the log message, since metadata is lost when forwarding externally
[health_log_analytics_add_sourcetype]
SOURCE_KEY = _hla_sourcetype
REGEX = ^hla_sourcetype::(.+)$
FORMAT = sourcetype="$1"] $0
DEST_KEY = _raw
[health_log_analytics_add_host]
SOURCE_KEY = MetaData:Host
REGEX = ^host::(.+)$
FORMAT = host="$1" $0
DEST_KEY = _raw
[health_log_analytics_add_source]
SOURCE_KEY = MetaData:Source
REGEX = ^source::(.+)$
FORMAT = source="$1" $0
DEST_KEY = _raw
[health_log_analytics_add_syslog5424]
REGEX=.
FORMAT = - - [sdid@1234 $0
DEST_KEY = _raw
[health_log_analytics_add_index]
SOURCE_KEY = _MetaData:Index
REGEX = ^(.+)$
FORMAT = $1 $0
DEST_KEY = _raw
[health_log_analytics_add_syslogHost]
SOURCE_KEY = MetaData:Host
REGEX = ^host::(.+)$
FORMAT = $1 $0
DEST_KEY = _raw
[health_log_analytics_add_time]
SOURCE_KEY = _time
REGEX = ^(.+)$
FORMAT = <182>1 $1 $0
DEST_KEY = _raw
-
On the Finish.conf tab, restart Splunk by
running the
$SPLUNK_HOME/bin/splunk restart splunkd
command.
-
Select Save.
Health Log Analytics adds the data input record to the
Data Inputs table.
-
Ensure that the data input is configured correctly by selecting Test
connection.
Health Log Analytics tries to connect the MID Server to the data repository.
Note: You can revert to the last published configuration by selecting
Revert Changes. This option is available only
when you're modifying a configuration that has been published
previously.
-
Select Publish to publish the data input to the MID Server.