Health tab in AI Control Tower
Summarize
Summarized using AI
This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.
Summary of Health tab in AI Control Tower
The Health tab in the AI Control Tower dashboard enables ServiceNow customers to monitor the performance and effectiveness of guardrails enabled through Now Assist Guardian. It provides insights into offensive content and prompt injection occurrences across AI assets, helping organizations ensure compliance and responsiveness.
Show less
Key Features
- Average Latency Monitoring: Tracks latency resulting from active guardrails, indicating guardrail activity levels.
- Offensive Content Tracking: Displays counts and percentages of flagged offensive content and prompt injection occurrences, segmented by skills.
- Content Guardrail Effectiveness: Visualizes the number of flagged content items and their percentage of total requests to the large language model (LLM) service.
- Offensive Content Breakdown: Provides categories of offensive content occurrences and visualizations of occurrences by skill over time.
- Prompt Injection Insights: Similar visualizations as offensive content for tracking prompt injection occurrences, including latency and totals by skill.
Key Outcomes
By utilizing the Health tab, ServiceNow customers can effectively monitor and evaluate the performance of their guardrails, ensuring minimal latency and high compliance in AI interactions. This allows organizations to maintain a safer and more reliable AI environment, enhancing user experience and trust in AI outputs.
Monitor the performance of guardrails enabled through Now Assist Guardian.
The Health tab in the AI Control Tower dashboard helps you monitor and evaluate the effectiveness of offensive content and prompt injection guardrails active on your ServiceNow AI assets.
The visualizations on the Health tab provide the following insights.
- Average latency as a result of active offensive content and prompt injection guardrails. High latency could mean increased guardrail activity in the period.
- Count and percentage of offensive content and prompt injection occurrences.
- Skills where offensive content and prompt injection occurrences were detected.
The dashboard does not consider historical data for Health metrics.
Apply the filters on the dashboard to view guardrail activity for skills in a date range.
Content guardrail effectiveness
- Number of content items flagged
- This area of the dashboard shows the number of offensive content and prompt injection occurrences in the selected date range.
Figure 2. Number of content items flagged - Percentage of content items flagged of total use
- This area of the dashboard shows the percentage of requests and responses to and from the large language model (LLM) service that are flagged for offensiveness and prompt injection.
Figure 3. Percentage of content items flagged of total use
Offensive content visualizations
- Guardrail-added latency
- This area of the dashboard shows the average latency as a result of the active offensive content guardrail for the selected skills and date range.
Figure 4. Guardrail-added latency for offensiveness - Percentage flagged as offensive
- This area of the dashboard shows the percentage of requests and responses to and from the large language model (LLM) service that are flagged for offensive content.
Figure 5. Percentage flagged as offensive - Total offensive content occurrences
- This area of the dashboard shows the total number of offensive content occurrences for the selected skills and date range.
Figure 6. Total offensive content occurrences - Categories of offensive content
- This area of the dashboard shows a breakdown of offensive content occurrences by the categories. If content is deemed to be offensive under more than one category, for example, toxic and defamatory, the occurrence is counted
individually toward both the categories. For more information on offensive content categories, see Now Assist Guardian.
Figure 7. Categories of offensive content - Offensive content occurrences by skill
- This area of the dashboard shows the number of offensive content occurrences over time by the skills in which the content is detected.
Figure 8. Offensive content occurrences by skill
Prompt injection visualizations
- Guardrail-added latency
- This area of the dashboard shows the average latency as a result of the active prompt injection guardrail for the selected skills and date range.
Figure 9. Guardrail-added latency for prompt injection - Percentage flagged as prompt injection
- This area of the dashboard shows the percentage of requests and responses to and from the LLM service that are flagged for offensive content.
Figure 10. Percentage flagged as prompt injection - Total prompt injection occurrences
- This area of the dashboard shows the total number of offensive content occurrences for the selected skills and date range.
Figure 11. Total prompt injection occurrences - Prompt injection occurrences by skill
- This area of the dashboard shows the number of prompt injection occurrences over time by the skills where prompt injection attempts were detected.
Figure 12. Prompt injection occurrences by skill