Cloud Request Retry Configuration

  • Release version: Xanadu
  • Updated August 1, 2024
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Cloud Request Retry Configuration

    The Cloud Request Retry Configuration feature in ServiceNow enables Discovery and Service Mapping to intelligently retry requests when throttled by cloud providers such as AWS and Azure. This capability helps maintain efficient and resilient cloud discovery operations by handling throttling responses automatically before final failure.

    Show full answer Show less

    Administrators can access and customize retry configurations for each cloud provider under All > Discovery > Cloud Request Retry Configuration. These configurations are synchronized with MID Servers using the mid.cloud.discovery.retry.configuration property, ensuring consistent retry behavior across the environment.

    Retry Strategies

    The retry framework supports three main strategies to control how retries are handled:

    • Exponential backoff: Applies increasing delays between retries, with random multipliers and added jitter to avoid request spikes.
    • Response header backoff: Uses server-provided retry delay values from headers like Retry-After, converting them to milliseconds and adding jitter.
    • Custom backoff: Allows creation of a custom retry strategy via MID Server script includes, where admins define retry logic and delay calculations through the getDelay() function.

    How Exponential Backoff Works

    Using an example configuration (max 3 retries, response code 429, base delay 1000 ms, max delay 10000 ms, additional jitter window 1500 ms):

    • Retries use a random backoff multiplier increasing exponentially (e.g., between 0-1 for 1st retry, 0-3 for 2nd, 0-7 for 3rd).
    • The calculated delay is capped by the max delay limit.
    • Jitter is added by randomly selecting an additional delay within the defined jitter window to smooth retry timing.

    How Response Header Backoff Works

    With this strategy (e.g., max 3 retries, response code 429, header Retry-After, delay unit seconds, jitter window 1500 ms):

    • The system reads the Retry-After header from the throttled response and converts the value to milliseconds.
    • Jitter is added similarly as in exponential backoff to vary the retry interval.

    Practical Benefits for ServiceNow Customers

    • Improves reliability of cloud discovery by automatically handling provider throttling.
    • Reduces manual intervention and failed discovery attempts due to rate limits.
    • Offers flexibility to tailor retry behavior per provider or create custom retry logic if needed.
    • Ensures retry configurations are consistently enforced across MID Servers, maintaining uniform behavior.

    If a request is throttled by a cloud provider during Discovery, Cloud Request Retry Configuration provides a customizable method to retry requests. Discovery and Service Mapping Patterns includes a retry configuration for AWS and Azure. You can customize the included configuration or create your own.

    Discovery admins and Cloud admins can access the request retry configuration at All > Discovery > Cloud Request Retry Configuration. You can create one configuration for each provider.

    When a request is throttled, the retry framework uses retry configuration defined for the provider to handle retries before returning the final response to the ApiCommand classes:
    • AwsApiCommand
    • AzureApiCommand

    The retry configurations are synced with the MID Servers through the MID Server property, mid.cloud.discovery.retry.configuration.

    There are the following retry strategies:
    • Exponential backoff
    • Response header backoff
    • Custom backoff

    Exponential backoff

    For the following example configuration:
    Setting Value
    Max retries 3
    Response codes 429
    Base delay in ms 1000
    Max delay in ms 10000
    Additional delay window in ms 1500
    The Exponential backoff retry strategy works as follows:
    • 1st retry—the backoff multiplier is randomly selected between 0 and 1. The max delay value is 400 ms (400 * 1).
    • 2nd retry—the backoff multiplier is randomly selected between 0 and 3. The max delay value is 1200 ms (400 * 3).
    • 3rd retry—the backoff multiplier is randomly selected between 0 and 7. The max delay value is 2800 ms (400 * 7).

    On subsequent retries, if the delay exceeds 10000 (the max delay), 10000 will be used as initial delay.

    Once the initial delay is generated, jitter is added to the delay. The jitter window is defined by the Additional delay window in ms field. The system selects a random value between 0 and 1500 and adds it to the initial delay.

    If the initial delay is 500, the final delay (with jitter) can be a value between 500 and 2000 ms.

    Response header backoff

    For the following example configuration:
    Setting Value
    Max retries 3
    Response codes 429
    Response header Retry-After
    Response header delay unit Seconds
    Additional delay window in ms 1500
    The Response header backoff strategy works as follows:
    • Fetch the value of header Retry-After from the server response.
    • Convert the Retry-After to milliseconds by multiplying by 1000.

    Once the initial delay is generated, jitter is added to the delay. The jitter window is defined by the Additional delay window in ms field. The system selects a random value between 0 and 1500 and adds it to the initial delay.

    If the initial delay is 2000, the final delay (with jitter) can be a value between 2000 and 3500 ms.

    Custom backoff

    With a custom backoff retry strategy, you define the Max retries and Response codes and create your own Mid script include that defines how requests are retried using the getDelay() function. For more information, see Script includes.