Browser recommendations for Automated Test Framework
Summarize
Summary of Browser recommendations for Automated Test Framework
This guidance helps ServiceNow customers optimize the use of browsers when running automated tests with the Automated Test Framework (ATF). It addresses browser performance, reliability, and test integrity considerations to ensure smooth and accurate test execution.
Show less
Key Recommendations
- Periodic browser restarts: Some browsers like Internet Explorer, Edge, and older Firefox versions have memory management limits. Periodically closing and restarting these browsers prevents performance degradation during client test runner executions.
- CPU throttling avoidance: To prevent browsers from reducing CPU usage on out-of-focus windows, run each client test runner in its own browser window and keep that window partially visible. Also, ensure the system screen is neither locked nor turned off.
- Browser zoom level: Set the browser zoom to 100% to ensure screenshots taken during tests are clear and accurate.
- OS X specific guidance: On OS X using Chrome or Safari, locked screens or hidden client test runner tabs cause slow or timed-out tests. Running tests in a virtual machine environment where the screen remains active is recommended.
- Rollback behavior: Session cookies rollback all changes made during tests within the same browser session. Avoid making manual changes to the instance or navigating in other tabs in the same session while tests run, as these changes may be lost or interfere with test outcomes.
- Parallel testing best practices: - Always run each client test runner in an incognito/private window to isolate sessions and prevent rollback conflicts between tests running in parallel.
- Close client test runner windows after tests complete to ensure rollbacks do not affect legitimate changes made elsewhere.
Practical Impact
By following these browser configuration and usage guidelines, ServiceNow customers can:
- Maintain consistent test performance and prevent slowdowns caused by memory and CPU issues.
- Ensure test results are reliable by avoiding unintended rollbacks and session conflicts.
- Effectively run multiple tests in parallel without interference.
- Obtain accurate screenshots and avoid test timeouts, especially on OS X platforms.
Configure client test runner browsers to run automated tests and avoid performance degradations.
Periodic browser restarts
- Internet Explorer
- Edge
- Older versions of Firefox
How often you should close the browser depends on the memory allocation in the browser application.
Browser CPU throttling
- Run each client test runner in its own browser window.
- Ensure that the client test runner browser window is always partially visible on the screen.
- Ensure that the system screen is not locked or shut off.
Browser zoom level
Client test runners take screen shots as they run tests. For best results with screen shots, leave the browser zoom level set to 100%.
OS X CPU throttling
On OS X with the client test runner on Chrome or Safari: If the screen is locked or the client test runner tab is not shown, when the system attempts to run the test suite, tests run significantly slower and may time out. For best performance, run client test runners for scheduled suites in a virtual machine (VM) environment in which the screen does not become locked or disabled.
Rollback in browser sessions
The session cookies roll back all the changes made during a test. When a test is running, everything performed in that session is recorded for rollback. Don't modify your instance when a test is running in the same browser session. For example, if you modify records while a test is running in the same session, the changes are rolled back after the test completes. If you navigate around in other tabs in the same session, your work may be rolled back and interfere with tests that rely on implicit navigation.Parallel testing
Follow these guidelines to avoid issues when running multiple tests in parallel.
- Run each client test runner in an incognito or private window
- Because parallel tests roll back all changes tied to the same browser session, it's possible for legitimate changes made in another browser tab to be rolled back during parallel testing. To prevent unwanted rollback of changes, always run client test runners in their own browser session. Opening client test runners in an incognito or private window ensures that they always have their own browser session.
- Close client test runner windows when testing is complete
- To prevent unwanted rollback of changes, always close client test runners after testing is complete. Closing the browser window ensures that test rollback doesn't revert any legitimate changes made in another browser tab.