Adaptive desktop actions for web-based tasks

  • Release version: Australia
  • Updated September 5, 2025
  • 3 minutes to read
  • Adaptive desktop actions enables AI agents to automate repetitive tasks across web applications through a browser extension. The agent interacts directly with the browser by clicking, typing, and scrolling, without preconfigured APIs, scripts, or back-end logic.

    Adaptive desktop actions overview

    Desktop actions are tools that AI agents use to interact with web applications through a browser extension.

    When you configure an AI agent and select Desktop action as a tool, you choose how it operates:
    Table 1. Desktop action operating modes
    Mode How it works Use when
    Defined path Follows fixed steps preconfigured in AI Desktop Actions. The workflow is stable and steps are known in advance.
    Adaptive path Works from a high-level goal. Dynamically plans and executes steps based on your instructions. The workflow varies or steps cannot be fully defined in advance.

    Specify a high-level goal — such as updating user roles or scheduling maintenance — and the agent plans and executes the steps to complete it. You can take manual control at any point.

    How adaptive desktop actions work

    Stages in which adaptive desktop actions work

    LLM provider

    Adaptive desktop actions use AWS Anthropic Sonnet model provider.

    Users

    Adaptive desktop actions are available to all users who perform tasks across enterprise applications and automate repetitive work.
    Table 2. Users and descriptions
    Users Description
    Administrators Manage permissions, roles, and agentic workflows
    Developers Build and configure AI agents with desktop action tools
    Fulfillers Automate routine fulfillment tasks across multiple systems
    Requestors Manage browser extensions and submit requests that trigger AI agents to automate web workflows

    Operating desktop actions

    You access desktop actions through the Now Assist panel that has enhanced chat enabled. The AI agent provides updates on its progress in the chat interface. As the agent works, you receive:

    • Real-time status updates in the chat
    • Periodic screenshots of the web pages the agent navigates
    • Notifications when external websites require login credentials

    When an external website requires login, you're prompted in the chat. Switch to the external website tab, provide your credentials, then switch back to the Now Assist panel. The agent continues after authentication is complete.

    Note:
    When you close the chat, you have the option to delete the chat log, including all screenshots containing sensitive information. For more information, see Delete an AI agent chat log.

    Limitation

    Desktop actions operate as browser extensions with the following limitations:

    • Can only access content within the browser
    • Cannot interact with desktop applications or local files (except for downloading files)
    • Cannot upload data from the local file system

    For tasks requiring local file access, consider using defined desktop actions. For more information, see Defined path desktop actions for desktop and web-based tasks.