System properties for Cloud Cost Management

  • Release version: Xanadu
  • Updated August 1, 2024
  • 7 minutes to read
  • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of System properties for Cloud Cost Management

    This document details the system properties available in ServiceNow Cloud Cost Management (Release Xanadu) to configure various operational aspects such as billing downloads, spend analysis, business hours reporting, Azure and AWS integrations, rightsizing, pricing sheets, and the Cloud Cost Management Infra Stack. These properties allow you to optimize performance, manage workload sizes, control API call behaviors, and set timeouts for billing and recommendation processes.

    Show full answer Show less

    System Properties Configuration

    Most frequently used properties can be set via the System Properties form, while additional less common properties are accessible directly in the sysproperties table. Properties are categorized by function to help you tailor configurations specific to your environment and use cases.

    Key Property Categories and Their Impact

    • Billing Download Jobs: Control threading and batch sizes for processing AWS billing data to optimize CI placement speed and resource usage.
    • Spend Analysis: Configure limits on billing record processing per job, chunk sizes for spend flow launcher, and forecasting months for AWS spend predictions.
    • Business Hours Reporting: Adjust batch sizes and chunk parameters to manage report generation workloads and rightsizing recommendation executions.
    • Microsoft Azure: Set batch sizes for bulk action API calls (e.g., Start, Stop, Resize) to stay within Azure Batch API limits and tune workload chunk sizes for rightsizing and unused resource recommendations.
    • Rightsizing and Unused Resources (Azure): Define polling intervals for orchestrator jobs and chunk sizes for various asynchronous API calls related to recommendation generation and processing, balancing parallel execution and processing time.
    • AWS Pricesheet: Configure timeout duration for AWS price sheet download flows to prevent indefinite waits.
    • Cloud Cost Management Infra Stack: Manage job timeouts for different billing and processing stages, including file generation, job state transitions, and post-processing phases to ensure timely job completion and error handling.

    Practical Benefits for ServiceNow Customers

    • Optimizing batch sizes and thread counts improves data processing speed and system responsiveness during billing and spend data ingestion.
    • Configuring chunk sizes and workload distribution enables scalable and efficient handling of large cloud cost datasets and recommendations.
    • Setting appropriate timeout values helps avoid stalled jobs and ensures robust error handling, especially in integrations with cloud providers like AWS and Azure.
    • Adjusting polling intervals and parallel workload limits enhances real-time processing of rightsizing and unused resource recommendations, supporting better cost optimization decisions.
    • Understanding these properties empowers administrators to fine-tune the Cloud Cost Management system to match their organization's cloud environment scale and performance requirements.

    Next Steps

    Review your current Cloud Cost Management deployment and identify properties relevant to your cloud providers and use cases. Adjust system properties using the System Properties form or directly in the sysproperties table as needed. For critical timeout and job management properties related to the Infra Stack, coordinate changes with your ServiceNow support team to avoid unintended impacts.

    Set some property values on the System Properties form, but other lesser-used properties are available only on the System Property [sys_properties] table.

    Table 1. System properties for Billing Download jobs
    System property Description
    sn_cld_intg_core.ci_placement_threads_number Number of threads that are used by CI placement flow to create CIs in the billing process.
    • Type: integer
    • Default value: 4
    • Location: System properties
    sn_cld_intg_aws.max_unprocessed_records_for_ci_placement Maximum number of records to read in one batch from the AWS Cost And Usage Bill Data table for CI Placement.
    • Type: integer
    • Default value: 300000
    • Location: System properties
    sn_cld_intg_aws.min_unprocessed_records_for_ci_placement Minimum number of records to read in one batch from the AWS Cost And Usage Bill Data table for CI Placement.
    • Type: integer
    • Default value: 100000
    • Location: System properties
    Table 2. System properties for Spend analysis
    System property Description
    sn_cld_spend_core.metricbase_transform_limit Maximum number of Billing records to use in one Metric Base transform.
    • Type: integer
    • Default value: 20000
    • Location: System properties
    sn_cld_spend_core.spend_report_flow_launcher_chunk_size Number of non-empty CI chunks to send to the spend flow launcher.
    • Type: integer
    • Default value: 40
    • Location: System properties
    sn_cld_spend_core.spend_report_per_chunk_workload_size Number of workloads (CI ranges and metadata) for each chunk in the spend flow launcher.
    • Type: integer
    • Default value: 500
    • Location: System properties
    sn_cld_spend_aws.num_months_forecast Number of future months after the current month for which the AWS Forecast spend is retrieved during every execution of the Spend job.
    • Type: integer
    • Default value: 2
    • Location: System properties
    Table 3. System properties for Business Hours
    Note:
    The collection of data for a single download of billing or price sheet data is called a workload. Data in a workload is held in sets of data called chunks. You can specify the size of each chunk and the number of chunks in a workload.
    System property Description
    sn_clin_core.bh_report.batch_size The number of resources to include in a Business Hours report batch.
    • Type: integer
    • Default value: 5000
    • Location: System properties
    sn_clin_core.bh_recom_max_chunk_size Number of records used to create workload chunks of rightsizing recommendations. If you set a value greater than glide.db.max_view_records , then the glide.db.max_view_records value is used instead. A smaller number means more chunks and a larger number means fewer chunks.

    Records considered in one execution of generating recommendations = (number of chunks) * (data records/chunk)

    • Type: integer
    • Default value: 10000
    • Location: System properties
    sn_clin_core.bh_recom_num_chunks_per_workload Number of chunks per workload. This value times the value of sn_clin_core.bh_recom_max_chunk_size is the total number of records per workload when generating rightsizing recommendations.

    Records considered in one execution of generating recommendations = (number of chunks) * (data records/chunk)

    • Type: integer
    • Default value: 10
    • Location: System properties
    Table 4. System properties for Microsoft Azure
    System property Description
    mid.azure_action.batch_size

    Use batch_size to set the batch size to fewer than 20 for action calls like Stop, Start, Modify (resize), Terminate, and Describe.

    Cloud Cost Management uses the Azure Batch API to make a bulk request for the actions. By default, Azure Batch supports a maximum of 20 for the synchronous call.

    • Type: integer
    • Default value: 20
    • Location: System properties
    Table 5. System properties for Rightsizing and Unused resources operations for Microsoft Azure
    Note:
    The collection of data for a single set of recommendations is called a workload. Data in a workload is held in sets of data called chunks. You can specify the size of each chunk and the number of chunks in a workload.
    System property Description
    sn_cld_intg_azure.az_rs_recommendation_queue_orchestrator_polling_time Polling time for the Microsoft Azure recommendation orchestrator to check the status of the generate, get generate status, list recommendation, or process recommendation call and move it to the next stage for processing.
    Note:
    A short polling time is better in the case of fewer subscriptions.
    • Type: integer
    • Default value: 5000 (ms)
    • Location: System properties
    sn_cld_intg_azure.chunk_size_generate_recommendation_subscriptions

    Number of subscriptions to be processed by each workload. Each workload creates a maximum of 5 threads to process 10 subscriptions in parallel based on the available MID Server worker threads.

    Changing the setting to lower number increases the number of created workloads. Generate Recommendation is a asynchronous API call that quickly returns the generate recommendation operation ID.
    Note:
    For fewer than 50 subscriptions, you can decrease the value to enable parallel processing. For example, for 40 subscriptions, changing the chunk size to 10 allows 4 workloads to process subscriptions. By default, a maximum of 3 parallel workload executions are allowed. The maximum is configured in the flow launcher job that is defined in the sn_cld_intg_core_flow_launcher_job_config table.
    • Type: integer
    • Default value: 5
    • Location: System properties
    sn_cld_intg_azure.advisor_get_status_min_chunk_size

    Minimum number of subscriptions to process in a workload for Get generate status API call.

    The call creates a new workload when the specified minimum chunk size is met and, after all subscriptions are processed, does not create another workload.

    Note:
    By default, a maximum of 4 parallel workload executions are allowed. The maximum is configured in the flow launcher job that is defined in the sn_cld_intg_core_flow_launcher_job_config table.
    • Type: integer
    • Default value: 5
    • Location: System properties
    sn_cld_intg_azure.advisor_list_recommendation_min_chunk_size Minimum number of subscriptions to process in a workload for the list recommendation API call.

    The call creates a new workload when the specified minimum chunk size is met and, after all subscriptions are processed, does not create another workload.

    Note:
    Increasing the chunk size might increase the overall processing time because subscriptions are processed sequentially. By default, a maximum of 4 parallel workload executions are allowed. The maximum is configured in the flow launcher job that is defined in the sn_cld_intg_core_flow_launcher_job_config table.
    • Type: integer
    • Default value: 1
    • Location: System properties
    sn_clin_azure.advisor_process_recommendation_min_chunk_size

    The number of recommendation responses that should be processed in a workload for the API call and persisted in the Rightsizing or Unused Machines recommendation table.

    The call creates a workload when the specified minimum chunk size is met and, after all subscriptions are processed, doesn’t create another workload.

    Note:
    Increasing the chunk size might increase the overall processing time because subscriptions are processed sequentially. By default, a maximum of 4 parallel workload executions are allowed. The maximum is configured in the flow launcher job that is defined in the sn_cld_intg_core_flow_launcher_job_config table.
    • Type: integer
    • Default value: 20
    • Location: System properties
    Table 6. System properties for AWS Pricesheet
    System property Description
    sn_cld_intg_aws.pricesheet_timeout_aws Controls AWS price sheet download flow time out duration.
    • Value format: DD HH:MM:SS
    • Default value: 2 days
    Table 7. System properties for Cloud Cost Management Infra Stack
    Important:
    These system properties are used only if the Cloud Cost Management Infra Stack application is installed in addition to the Cloud Cost Management version 8.1.
    System property Description
    ready.job.files.timeout Maximum time in minutes to generate billing files for AWS assume role setup or blobs for Azure on Glide.

    You can monitor AWS billing job files [sn_cld_intg_aws_billing_files] table for AWS and Azure billing Job chunk blob detail [sn_cld_intg_azure_job_chunk_blob_detail] table for Azure only when the download is in progress and the time-out condition is met.

    • Type: String
    • Default value: 180
    • Location: System properties
    ready.job.nofiles.timeout Maximum time in minutes to wait for the backend to pick up the job when there’s no other job being processed for the same provider.
    Important:
    To configure this system property, contact your support team. Don't configure it on your own.
    • Type: String
    • Default value: 10
    • Location: System properties
    reserved.job.timeout Maximum time in minutes to wait for the job to change from Reserved to In progress state.
    Important:
    To configure this system property, contact your support team. Don't configure it on your own.
    • Type: String
    • Default value: 5
    • Location: System properties
    inprogress.job.timeout Maximum time in minutes to wait for the job to change from In progress to Sink complete state.
    Important:
    To configure this system property, contact your support team. Don't configure it on your own.
    • Type: String
    • Default value: 10
    • Location: System properties
    sinkcomplete.job.timeout Maximum time in minutes to wait for the job to change from Sink begin to Sink complete state.
    Important:
    To configure this system property, contact your support team. Don't configure it on your own.
    • Type: String
    • Default value: 20
    • Location: System properties
    billing.data.post.processing.timeout Maximum waiting time in minutes for billing from Post processing to Complete phase.

    The final stage of billing job includes inserting spend data into Glide. During the Post processing phase, the newly inserted data is activated and the existing data is inactivated. Increasing the time out value gives more time for the activation flow to complete.

    • Type: String
    • Default value: 30
    • Location: System properties
    retry.cred.failed.billing.job.timeout Maximum time in minutes to wait before canceling a job that's in the Requested state due to credential failure.

    When you add a credential, it’s sent to the backend only in the next billing run, resulting in a temporary failure. However, retries are made to establish a connection. This property specifies how long to wait before canceling the reattempted job.

    • Type: String
    • Default value: 30
    • Location: System properties
    sn_cld_intg_azure.billing_chunk_duration Defines the chunk size for Azure billing blob in days.

    By default, each blob contains billing data for three days.

    • Type: Number
    • Default value: 3
    • Location: System properties