MID Server pre-upgrade check
Summarize
Summary of MID Server pre-upgrade check
The MID Server pre-upgrade check is an automated process that runs before upgrading the MID Server to ensure the upgrade proceeds smoothly without causing outages. The AutoUpgrade monitor compares the MID Server version to the instance version and triggers pre-upgrade validation tests if an upgrade is needed. These tests run for both instance-triggered and manual upgrades.
Show less
Pre-upgrade Tests
The pre-upgrade check includes two test categories:
- Mandatory tests: All must pass to proceed with the upgrade. Failures block the upgrade, log errors to the MID Server Issue [eccagentissue] table, and halt the upgrade until resolved.
- Alert tests: Failures log warnings but do not stop the upgrade process.
Mandatory tests verify:
- At least 2 GB of free disk space on the MID Server host
- Access to the download site (install.service-now.com)
- Digital signature verification of a sample downloaded file
- File permission checks for extracting, copying, reading, and deleting upgrade files
- Host system is not 32-bit
- glibC version on Linux MID Servers is at least 2.17
Alert tests (Windows MID Servers) check for running upgrade blocker services and the status of Application Experience on the host machine. Running the MID Server as a non-administrator user is recommended for security but not mandatory.
Common Upgrade-Blocking Errors
- Insufficient free disk space (<2 GB)
- Inability to download update packages due to permission or network issues
- Failure in digital signature verification
- File permission issues such as inability to extract zip files or create folders
- MID Server Windows Service running without appropriate permissions
- Host machine not meeting minimum system requirements (e.g., 32-bit system, unsupported glibC version)
Non-blocking Warnings
These warnings log issues that do not stop upgrades but may require attention:
- Improper formatting of Windows service "Log On As" user
- Failure to retrieve user group memberships correctly
- PowerShell-related warnings indicating it is not installed or usable (does not block upgrade)
- Presence of services like CiscoAMP that block upgrades (require remediation)
- Disabled Application Experience service, which may cause upgrade failures
Disabling the Pre-upgrade Check
The pre-upgrade check runs by default but can be disabled:
- For an individual MID Server by setting the mid.upgrade.runprecheck parameter to
falsein that MID Server’sconfig.xml - Globally for all MID Servers by adding the mid.upgrade.runprecheck property with value
falsein the MID Server Property [eccagentproperty] table
Practical Implications for ServiceNow Customers
Understanding and monitoring the MID Server pre-upgrade check helps ensure your MID Server upgrades proceed without interruption. Addressing mandatory test failures promptly prevents upgrade blocks and MID Server downtime. Regularly review the MID Server Issue table for logged errors or warnings and verify that your MID Server host meets system requirements, has proper permissions, and network access. Maintaining these conditions supports a smooth upgrade process and uninterrupted MID Server operation.
Prior to an upgrade, the MID Server runs tests to identify issues that could cause the upgrade process to fail or result in a MID Server outage.
Each MID Server contains an AutoUpgrade monitor that compares the MID Server version with that of the instance to determine if the MID Server needs to upgrade. If the AutoUpgrade monitor discovers that the MID Server version is out of date, the monitor runs pre-upgrade validation tests for that MID Server. If an issue is detected, a message is logged to the MID Server Issue [ecc_agent_issue] table, and the upgrade is blocked. The AutoUpgrade monitor continues to run every hour, until all the tests pass. If there are no blocking issues, the MID Server downloads the appropriate upgrade package and begins the upgrade process. The MID Server pre-upgrade check also runs for the upgrades triggered by the instance when the instance was upgraded, as well as for manual upgrades.
Pre-upgrade tests
The pre-upgrade runs two sets of tests: Mandatory testsand Alert tests. Every mandatory test is required to pass to start MID Server upgrade. If a mandatory test was not passed, the test is recorded in the agent log and a MID Server active issue is created in the MID Server issue [ecc_agent_issue] table. These errors are published before the actual MID Server upgrade occurs and must be resolved before the upgrade can continue. MID Server stops the Upgrade process and changes the status based on MID Server states. If any alert test was not passed, a message is added in the agent log and MID Server issue table, but the pre-upgrade check continues and MID Upgrade doesn’t stop.
- At least 2 GB of free disk space
- Access to the download site at install.service-now.com
- Verify the digital signature of a sample downloaded file
- Permission to execute, which involves the following:
- Extract a Zip archiver to the upgrade temp folder: agent/work/upgrade_temp
- Copy files from the upgrade temp folder to the agent folder
- Read a text file and validate the content
- Delete the pre-upgrade check contents
- MID Server host machine is not a 32 bit system
- On Linux MID Servers, the glibC version is at least 2.17
- Any MID Server Upgrade blocker services is running on the host machine
- Status of Application Experience on the host machine
- For security best practice, run the MID Server as a non-administrator user. If the current Log On As user account for the MID Server is LocalSystem or is a user that is part of the Administrators group, the upgrade does not fail, but this configuration is not recommended.
Errors that block the upgrade
- Not enough free disk space. The system reports <n> bytes free
- This message is displayed when less than 2 GB of free disk space is detected on the MID Server host. This error is also written to the MID Server agent log.
- Unable to download updates from the install server
- This message indicates that either the MID Server host does not have permission to download the installation package from install.service-now.com, or network problems prevent connection. This error is also written to the MID Server agent log.
- Signature verification failed:<message>
- This message indicates that a Verification Exception has occurred when pre-upgrade check tries to verify the digital signature of the downloaded check file.
- Unable to verify digital signature:<message>
- This message indicates that the verification signature process was interrupted by an exception.
- Unable to extract contents of pre upgrade check zip
- This message indicates that the service account on the MID Server host does not have permission to extract the pre-upgrade ZIP archive to the temporary folder.
- Unable to create folder <upgrade check file path>
- This message indicates that the MID Server service account does not have permission to create the upgradeCheck folder for the pre-upgrade checking files in the agent/package path.
- Unable to verify file permissions: <message>
- This message indicates an exception has occurred when checking file permissions, such as a file that does not exist or access failure.
- MID Server Windows Service is not running as LocalSystem or a local Administrator
- This message warns that the Windows service is not running with the desired permissions.
- Upgrade Failure: The host machine does not meet the minimum system requirements to upgrade the MID server.
- MID upgrade is not supported on 32-bit host. Please refer to KB0863694 for more information.
- Upgrade Failure: Host does not meet the minimum system requirements to upgrade the MID server. Please refer to MID Server product documentation on GlibC library for more information
- This error can only occur for Linux MID Servers. Refer to Java version support for more information.
Non-blocking warnings
- Unable to parse $logOnAsUser
- This message warns that the Log On As User value for the Windows service is not in
either of these expected formats:
- user@domain.company.com
- domain\user
- Unable to look up Log On As user's groups
When the instance attempts to look up the logged on user's group memberships, it executes the net user <username> command. The instance expects a certain output structure by the Windows service from this command and issues this warning if the expected output does not match the actual output.
These PowerShell warnings are written only to the MID Server agent log. Because PowerShell is not required to use a MID Server, these configuration issues do not prevent a Windows MID Server from upgrading. However, these warnings might indicate issues in your environment that require attention.
- Skipping PowerShell upgrade checks since PowerShell is not usable
- PowerShell 3.0 (at a minimum) is not installed or powershell.exe is not available to the MID Server service user.
- Skipping Powershell test: Target folder has not been initialized
- Skipping Powershell test: Unable to detect the service name
- Service CiscoAMP running on the MID Server host machine is identified as a MID Server upgrade blocker
- For more information, refer to KB0870329.
- The Application Experience is disabled on the MID Server host, which can cause the auto-upgrade process to fail and the MID Server to go down. Enable the service on the host to ensure successful auto-upgrade
- For more information, refer to KB0597552.
- Continuing with upgrade, but the following issue was encountered during upgradeCheck: <exception message>
- This message indicates that there was an issue running the PowerShell portion of the pre-upgrade tests.
Disabling the pre-upgrade check
A MID Server configuration parameter called mid.upgrade.run_precheck is set to true by default, which allows the automatic pre-upgrade test to run. To disable these tests for a single MID Server, add this parameter to that MID Server's config.xml file and set it to false. To disable these tests for all MID Servers, add a new record to the MID Server Property [ecc_agent_property] table called mid.upgrade.run_precheck. Set the value of this property to false and leave the MID Server field blank.