Now Assist Guardian Overview

Generative AI is an emerging technology. Human interactions are unpredictable, and outputs generated by large language model (LLM) are probabilistic, which means that they're based on probabilities. Running the same input twice may generate two different outputs. Managing risk is an important element of deciding how you want to implement generative AI on your instances.

Now Assist Guardian monitors requests sent to LLMs and their responses to help protect you, your users, and your data. There are three types of content that are monitored for: offensive or harmful content, prompt injection attempts, and filtered subjects. For offensive content and prompt injection attempts, logs are generated if activated, but you can also choose to block the content. When a filter has been activated, detected content that the filter applies to will redirect the user to the Sensitivity Detection: Fallback topic in Virtual Agent.

Guardrails

Offensive content

Due to the probabilistic nature of generative AI, it's possible for an LLM to generate offensive content. If there's offensive content in the input of the request, offensive content can also occur in the response. Examples of offensive content include language that is toxic, defamatory, or fraudulent.

Prompt injection

Prompt injection is a type of security attack where bad actors override the normal instructions of an LLM to access restricted information or elicit unexpected behaviors. Prompt injection detection is based on the LLM which has been trained on various types of prompt injection techniques such as role playing, paraphrasing, repetition, instructions to ignore other instructions, persuasion, etc. However, due to the probabilistic nature of the model as well as evolving prompt injection techniques, prompt injection attempts may not be identified by Now Assist Guardian in some cases.

Filtered subjects

Certain subjects, such as workplace safety or employee compensation, might not be best suited for generative AI conversations. You can activate filters that detect if these kinds of subjects are included in the conversation so that you can redirect the user to the Sensitivity Detection: Fallback Virtual Agent topic.

Logging and blocking

Now Assist Guardian can monitor requests and can log when these kinds of material are detected. You can access logs from the Now Assist Admin console in the Now Assist Guardian page of the Settings tab. Data included in the logs include information about the request and the conversation that contains the offensive content, including user feedback.

Besides logging, you can also choose to block offensive content or prompt injection attempts. If the material is detected and blocking is turned on, you see a standard error message instead of the generated response. The message is a standard error message indicating that the request couldn’t be completed, and you don't see what the AI generated.

Before deciding to block content, you can monitor logs for some time to determine how prevalent these issues are for you and your use cases.

Redirection for sensitive filtered topics

Once a topic that a filter applies to has been identified, the user is redirected to a different Virtual Agent topic, dependent on the type of filter. Filters for subjects like employee personal issues will redirect to the Sensitivity Detection: Fallback topic. This topic can redirect a user to a live agent or help them create an HR case.

Now Assist Guardian at runtime

All skills that use Now Assist Guardian remove personally identifiable information (PII) before the request reaches the LLM. You can choose what kinds of data are caught. See Configuring Now Assist for Data Privacy for more information.

For conversational skills, semantic search processes requests to determine whether a filter has been detected. If so, the user is redirected to a Virtual Agent topic that asks if they want to make an HR case or speak to a live agent.

For catalog item generation and agent skills like summarization and resolution note generation, offensiveness and prompt injection guardrails are run on inputs and outputs of requests. If either is detected, Now Assist Guardian logs the request. If you’ve chosen to block this content, then a standard error message appears, and the user doesn’t see the result of the request.

Safety Categories

Skills that support Now Assist Guardian