Cloud Request Retry Configuration

  • Release version: Yokohama
  • Updated January 30, 2025
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Cloud Request Retry Configuration

    The Cloud Request Retry Configuration in ServiceNow Yokohama release allows Discovery and Cloud admins to manage retries for cloud provider requests that are throttled during Discovery. This feature supports built-in retry configurations for AWS and Azure, which can be customized or new configurations created for other providers. The configurations govern how retry attempts are made before the system returns a final response.

    Show full answer Show less

    Admins can access and manage retry settings via All > Discovery > Cloud Request Retry Configuration. Each cloud provider can have a distinct retry configuration synced to MID Servers using the mid.cloud.discovery.retry.configuration property.

    Retry Strategies

    There are three main retry strategies available:

    • Exponential Backoff: Retries use increasing delays with random jitter added. The delay grows exponentially with each retry, capped by a maximum delay. For example, with settings like max retries 3, base delay 1000 ms, and max delay 10000 ms, the delay multiplier increases at each retry and a random jitter up to 1500 ms is added.
    • Response Header Backoff: The retry delay is based on the Retry-After response header sent by the cloud provider. The header value is converted to milliseconds, and a random jitter is added. This method respects server-suggested retry intervals.
    • Custom Backoff: Allows defining max retries and response codes while implementing a custom delay logic via a MID Server script include using the getDelay() function. This provides flexibility for specialized retry behaviors.

    Practical Benefits for ServiceNow Customers

    • Improves robustness of Discovery and Service Mapping by automatically handling throttled cloud API requests.
    • Allows customization to optimize retry behavior per cloud provider's throttling policies.
    • Minimizes failed requests and improves data accuracy by intelligently retrying instead of immediately failing.
    • Syncing retry configurations to MID Servers ensures consistent retry handling across Discovery infrastructure.

    This configuration empowers customers to fine-tune how Discovery interacts with cloud APIs, reducing errors caused by throttling and enhancing overall reliability.

    If a request is throttled by a cloud provider during Discovery, Cloud Request Retry Configuration provides a customizable method to retry requests. Discovery and Service Mapping Patterns includes a retry configuration for AWS and Azure. You can customize the included configuration or create your own.

    Discovery admins and Cloud admins can access the request retry configuration at All > Discovery > Cloud Request Retry Configuration. You can create one configuration for each provider.

    When a request is throttled, the retry framework uses retry configuration defined for the provider to handle retries before returning the final response to the ApiCommand classes:
    • AwsApiCommand
    • AzureApiCommand

    The retry configurations are synced with the MID Servers through the MID Server property, mid.cloud.discovery.retry.configuration.

    There are the following retry strategies:
    • Exponential backoff
    • Response header backoff
    • Custom backoff

    Exponential backoff

    For the following example configuration:
    Setting Value
    Max retries 3
    Response codes 429
    Base delay in ms 1000
    Max delay in ms 10000
    Additional delay window in ms 1500
    The Exponential backoff retry strategy works as follows:
    • 1st retry—the backoff multiplier is randomly selected between 0 and 1. The max delay value is 400 ms (400 * 1).
    • 2nd retry—the backoff multiplier is randomly selected between 0 and 3. The max delay value is 1200 ms (400 * 3).
    • 3rd retry—the backoff multiplier is randomly selected between 0 and 7. The max delay value is 2800 ms (400 * 7).

    On subsequent retries, if the delay exceeds 10000 (the max delay), 10000 will be used as initial delay.

    Once the initial delay is generated, jitter is added to the delay. The jitter window is defined by the Additional delay window in ms field. The system selects a random value between 0 and 1500 and adds it to the initial delay.

    If the initial delay is 500, the final delay (with jitter) can be a value between 500 and 2000 ms.

    Response header backoff

    For the following example configuration:
    Setting Value
    Max retries 3
    Response codes 429
    Response header Retry-After
    Response header delay unit Seconds
    Additional delay window in ms 1500
    The Response header backoff strategy works as follows:
    • Fetch the value of header Retry-After from the server response.
    • Convert the Retry-After to milliseconds by multiplying by 1000.

    Once the initial delay is generated, jitter is added to the delay. The jitter window is defined by the Additional delay window in ms field. The system selects a random value between 0 and 1500 and adds it to the initial delay.

    If the initial delay is 2000, the final delay (with jitter) can be a value between 2000 and 3500 ms.

    Custom backoff

    With a custom backoff retry strategy, you define the Max retries and Response codes and create your own Mid script include that defines how requests are retried using the getDelay() function. For more information, see Script includes.