Pattern-based discovery in Service Mapping

  • Release version: Xanadu
  • Updated August 1, 2024
  • 3 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Pattern-based discovery in Service Mapping

    Pattern-based discovery is the primary method Service Mapping uses to collect detailed data about devices and applications that comprise application services. This data is then used to create accurate maps of application services and populate the Configuration Management Database (CMDB) with configuration items (CIs). Each pattern is a defined sequence of commands aimed at identifying CI attributes and their outbound connections, enabling comprehensive mapping of service components.

    Show full answer Show less

    How Pattern-based Discovery Works

    The discovery process begins at an entry point—a defined access point such as an IP address or hostname where clients connect to a service instance. Service Mapping first performs a horizontal discovery to identify the host device, then a top-down discovery to detect applications running on that host.

    Service Mapping relies on MID Servers within the organization network to execute discovery probes and communicate securely without firewall traversal. These MID Servers process discovery requests from the External Communication Channel (ECC) queue and run various probe types, including port discovery (Shazzam probes), classification, identification, and exploration.

    Discovery and Mapping Process

    • An administrator defines a service instance with an entry point, creating a record in the Mapped Application Service [cmdbciservicediscovered] table.
    • Service Mapping verifies if the host device exists in the CMDB; if not, it triggers Discovery to identify and add the host using probes.
    • Once the host is confirmed, Service Mapping creates an application discovery request targeting the entry point IP and assigns it to a MID Server.
    • The MID Server runs identification patterns to discover the application CI and then executes connectivity patterns to find outbound connections.
    • Discovered CI details, attributes, and connections are sent back through the ECC queue and stored in the CMDB, enriching the service map.
    • Service Mapping recursively creates discovery requests for connected applications until it encounters a CI with no outbound connections or a defined boundary, which stops further discovery.

    Important Considerations

    • Pattern operations currently only support English; non-English returned values cannot be parsed properly, causing discovery failures.
    • Service Mapping periodically runs service recomputation to refresh CI data, incorporate network and storage paths, and apply impact rules based on CI changes in the CMDB.

    Benefits for ServiceNow Customers

    Pattern-based discovery enables customers to automatically and accurately map the complex relationships between devices and applications in their services. This comprehensive mapping improves visibility into service dependencies, supports effective configuration management, and helps maintain an up-to-date CMDB. By leveraging MID Servers and structured discovery patterns, customers can reliably discover infrastructure and application components without manual intervention, reducing errors and operational effort.

    Pattern-based discovery is the main method of Service Mapping collecting data about devices and applications used in application services. After Service Mapping collects data, it then creates a map of application services and stores the collected data in the CMDB.

    ServiceNow applications refer to devices and applications that comprise a service instance as configuration items (CIs).

    Service Mapping uses patterns to discover and map CIs. A pattern is a sequence of commands whose purpose it is to detect attributes of a CI and its outbound connections. A typical Service Mapping pattern consists of two types of algorithms for identifying CIs and finding CI connections.

    The starting point of any discovery process is an entry point. An entry point is a point where clients access a service instance. For example, to map your electronic mailing application service, define an IP address or host name of the email server as an entry point. The discovery and mapping process begins from Discovery performing the horizontal discovery to identify the host. Once the host discovery is complete, Service Mapping starts the top-down discovery to find and map applications running on this host.

    Service Mapping uses MID Servers to communicate with CIs in your organization. MID Servers are located inside your organization network and Service Mapping can communicate with them without traversing firewalls.

    Note:
    Currently, pattern operations do not support multi-languages. If values returned from pattern operations are not in English, the returned data cannot be parsed properly and the pattern discovery will fail.
    The discovery and mapping process consists of the following interactions:
    1. An administrator defines an service instance with an entry point for an application CI.

      Service Mapping creates a record for the new service instance in the Mapped Application Service [cmdb_ci_service_discovered] table.

    2. The device hosting the application is identified.
      1. Service Mapping checks if the device hosting this application CI exists in the CMDB.
      2. If the device hosting this application CI does not exist, Service Mapping triggers Discovery to detect host.

        Service Mapping checks the CMDB and triggers the horizontal discovery.
      3. Discovery creates the first set of probes for port discovery, referred to as Shazzam probes, and places them as a discovery request in the External Communication Channel (ECC) queue.
      4. The MID Server checks the ECC queue and retrieves the discovery request assigned to it.
      5. The MID Server runs the probes against the host and discovers open ports.

        Discovery uses the ECC Queue to run probes by the MID Server.
      6. The MID Server passes information on the host ports to the ECC queue.
      7. Discovery checks the ECC queue and receives information on the host ports.

        The MID Server passes the information to the ECC queue for Discovery to collect.
      8. These steps are repeated for other types of probes: classification, identification, and exploration.
      9. Discovery adds the host to the CMDB.

        Discovery adds the host to the CMDB.
      10. During the host discovery using probes, Service Mapping checks the ECC queue if this process is complete. When the host discovery is complete, Service Mapping checks whether this host exists in the CMDB.
      Note:
      For the detailed description of the horizontal discovery flow, refer to Horizontal discovery process flow with probes and sensors.
    3. Once the host is found in the CMDB, Service Mapping discovers the application running on this host.
      1. Service Mapping creates an application discovery request for the IP address of the entry point. It then writes the request in the ECC queue and assigns a MID Server to the request.
      2. The MID Server checks the ECC queue and retrieves the discovery request assigned to it.
      3. The MID Server starts running identification sections of the patterns associated with the classifier to find the match for the entry point. When the identification section matches the entry point, the pattern discovers a CI.
        Service Mapping places patterns to run by the MID Server.
      4. The MID Server starts running connectivity sections of the pattern to find outgoing connections of the newly discovered CI.
      5. The MID Server passes information on the discovered CI, its attributes, and connections to the ECC queue.
      6. Service Mapping checks the ECC queue and receives information on the newly discovered CI.

        Whenever Service Mapping checks the ECC queue and receives information on a discovered CI, it checks these tables for any data on outbound connections related to the CI: the cmdb_tcp and sa_flow_connection tables. If these two tables contain unique data that patterns did not discover, Service Mapping enriches the information about the CI connections and adds them to the map.

      7. Service Mapping writes the information into the CMDB and adds this CI to the service instance map.

        Service Mapping pulls the information about a CI from the ECC queue and writes it into the CMDB.
      8. Service Mapping creates the discovery requests for all applications to which the newly discovered CI connects. Mapping is complete after Service Mapping maps a CI that either does not have any outbound connections or is marked as a boundary. A boundary makes Service Mapping stop discovery from this point and not follow outgoing connections.
    4. The system regularly runs the service recomputation to query the CMDB for the latest CI changes, add data for network and storage paths, and apply CI impact rules.