ServiceNow Kafka Consumer

  • Release version: Washingtondc
  • Updated February 1, 2024
  • 2 minutes to read

    • Integrates your ServiceNow instance with Kafka Consumer and stores data in the ServiceNow tables.

    Request apps on the Store

    Visit the ServiceNow Store website to view all the available apps and for information about submitting requests to the store. For cumulative release notes information for all released apps, see the ServiceNow Store version history release notes.

    Integration Hub subscription

    This spoke requires an Integration Hub subscription. For more information, see Legal schedules - IntegrationHub overview.

    Supported versions

    This spoke was built for Confluent REST Proxy API v2.

    Spoke dependencies

    If you’re having trouble installing the app, ensure that these dependent plugins are installed:
    • Confluent Kafka REST Proxy Spoke
    • ServiceNow IntegrationHub Runtime (com.glide.hub.integration.runtime)
    • ServiceNow IntegrationHub Action Step - RTE (com.glide.hub.action_step.rte)
    Note:
    Some of these plugins are licensable features and require appropriate licenses, if used outside the spoke implementation.

    Connection and credential alias requirements

    Integration Hub uses aliases to manage connection and credential information. Using an alias eliminates the need to configure multiple credentials and connection information profiles when using multiple environments. If the connection or credential information changes, you don't need to update any actions that use the connection. For more information, see Connections and Credentials.

    ServiceNow Kafka Consumer uses the connection and credential alias of the Confluent Kafka REST Proxy Spoke.

    Configure ServiceNow Kafka Consumer

    Retrieve events pertaining to the specified topics and store it in the required ServiceNow tables.

    Before you begin

    Procedure

    1. Navigate to All > Process Automation > Kafka Consumer.
    2. Click New.
    3. On the form, fill these values.
      Field Description
      Name Name to identify the record.
      Broker Connection Alias Connection alias of the Confluent Kafka REST Proxy Spoke.
      Topic Topic from which you want to retrieve the events.
      Robust Transformer Definition Robust Transformer Definition that is used for the event transformation and data storage in ServiceNow tables. For information, see Robust Import Set Transformers.
      Offset Starting Point Value of the Kafka Consumer auto.offset.reset. This specifies what the Kafka Consumer should do when there is no initial offset or if the current offset does not exist any more on the server. See the Kafka 2.7 documentation for more information.
      Partitions per group Number of partitions per consumer thread.
      Page size (bytes) Maximum size of a request.
      Page limit Maximum number of requests per iteration.
      Embedded Format Format of the Kafka message for the topic.
      Request timeout (ms) Value of the Kafka Consumer consumer.request.timeout.ms. This specified the maximum amount of time the client waits for the response of a request. See the Kafka 2.7 documentation for more information.
      Stop Upon Transform Errors Option to stop listening when transformation errors are encountered.
      Verbose transformation Option to log all the imported events to import set and produce additional debug logs for transformation.
      Note:
      You must create one Kafka Consumer record for each topic.
    4. Click the Start Sink Listener related link.
      The Kafka Partition Group Listener States table is populated with the topic partition data.
    5. Optional: If you want to stop listening, click the Start Sink Listener related link.
    6. Navigate to System Scheduler > Scheduled Jobs.
    7. Open the record, Kafka Consumer Trigger.
    8. From the Trigger Type list, select Interval.
    9. Click Update.