Now Assist Guardian Overview

Generative AI is an emerging technology. Human interactions are unpredictable, and outputs generated by large language model (LLM) are probabilistic, which means that they're based on probabilities. Running the same input twice may generate two different outputs. Managing this risk is an important consideration when implementing generative AI on your instance. Now Assist Guardian evaluates requests sent to LLMs and their responses in real time to reduce that risk.

Guardrails

Offensive content

Due to the probabilistic nature of generative AI, it's possible for an LLM to generate offensive content. If there's offensive content in the input of the request, offensive content can also occur in the response. Examples of offensive content include language that is toxic, defamatory, or fraudulent.

When offensive content is detected, Now Assist Guardian logs the event. You can also configure it to block the content. This guardrail applies to specific Now Assist skills and workflows.

Prompt injection

Prompt injection is a type of security attack where someone tries to override the normal instructions of an LLM to access restricted information or cause unintended behaviors. Now Assist Guardian detects prompt injection attempts by using an LLM trained on various types of prompt injection techniques, such as role playing, paraphrasing, repetition, instructions to ignore other instructions, and persuasion.

Note:

Due to the probabilistic nature of the model and evolving attack techniques, Now Assist Guardian may not identify every prompt injection attempt in some cases.

Prompt injection protection applies to all generative AI applications and features on your instance. It is not limited to specific skills or workflows.

Filtered subjects

Certain subjects, such as workplace safety employee compensation, or personal well-being may not be best suited for generative AI responses. You can activate filters that detect these kinds of subjects in Virtual Agent conversations and redirect users to the Sensitivity Detection: Fallback Virtual Agent topic instead of generating an AI response.

Note:

Sensitive topic filters are available only with HR Service Delivery and apply only to Virtual Agent conversational skills.

Logging and blocking

Now Assist Guardian logs detected events for offensiveness and prompt injection. You can access logs from Now Assist Admin > Settings > Now Assist Guardian. Log dataincludes information about the request, the conversation that contains the offensive content, and any user feedback.

In addition to logging, you can configure Now Assist Guardian to block offensive content or prompt injection attempts. When blocking is enabled and content is detected, you see a standard error message instead of the generated response. The standard error message displays that the request couldn’t be completed, and you don't see what the AI generated. Before enabling blocking, review logs for a period of time to understand how frequently these issues occur in your environment.

Redirection for sensitive filtered topics

After a filter detects a sensitive topic, Now Assist Guardian redirects you to the Sensitivity Detection: Fallback topic in Virtual Agent. This topic can redirect you to a live agent or help you create an HR case.

Now Assist Guardian at runtime

All skills that use Now Assist Guardian remove personally identifiable information (PII) before the request reaches the LLM. You can configure what type of data is anonymized. For more information see, Configuring Now Assist for Data Privacy.

For conversational skills, semantic search processes requests to determine whether a filter has been detected. If so, the user is redirected to a Virtual Agent topic that asks if they want to make an HR case or speak to a live agent.

For catalog item generation and agent skills, such as summarization and resolution note generation, offensiveness and prompt injection guardrails run on inputs and outputs of requests. If either is detected, Now Assist Guardian logs the request. If you’ve chosen to block this content, then a standard error message appears, and the user doesn’t see the result of the request.

Safety categories captured in Now Assist Guardian logs

Skills that support offensive detection