Rate and review search results in an AI Search Tuning evaluation

  • Freigeben Version: Australia
  • Aktualisiert 12. März 2026
  • 2 Minuten Lesedauer

    • Select, review, and rate results for search terms included in an AI Search Tuning evaluation.

    Vorbereitungen

    The AI Search Tuning ServiceNow® Store application must be installed on your instance. For details on installing this application, see Install AI Search Tuning.

    Role required: sn_ais_data.tuning_reviewer

    Warum und wann dieser Vorgang ausgeführt wird

    Reviewers rate the relevancy of search results for AI Search Tuning evaluations. Search result relevance is rated separately for each search term defined in the evaluation. The accumulated search result ratings determine the overall search relevancy quality for the search applications being evaluated.

    Each reviewer selects search terms to review from the list of terms defined in the evaluation. Once a reviewer selects a search term, that search term becomes unavailable for selection by other reviewers for the same evaluation.

    When reviewing results for a search term, the review page displays up to 10 de-duplicated Genius Results and search results from either or both of the two search applications being evaluated. Results appear in an arbitrary order, and aren't sorted by relevancy score. Reviewers specify a numeric rating of 2, 1, or 0 for each result displayed, indicating its relevance to the selected search term.

    Prozedur

    1. Navigate to All > AI Search Tuning > Reviewer Homepage.
    2. In the card for the AI Search Tuning evaluation you want to work on, select Get started or Resume reviewing.
    3. In the Select search terms dialog, select search terms to rate and review.
      1. Wahlweise: Use the search field and Search icon to refine the set of search terms displayed, or select the Clear input icon to view the full list of search terms.
      2. Select all search terms that you want to rate and review.
      3. Select Start reviewing.
    4. On the Review search results page, rate and review search terms.
      1. Select a search term to rate and review from the Search terms list.
        Hinweis:
        The Search terms list shows search terms in alphabetical order and highlights the selected search term. A Reviewed count shows the number of search results rated for the selected search term and the total number of search results it returned.
      2. For each search result displayed, select a rating value:
        Rating Description
        2 - Exact The result is an exact match for the search. It should appear with high relevancy in results for the search term.
        1 - Useful The result isn't directly relevant for the search, but might be useful to some search users. It should appear with low relevancy in results for the search term.
        0 - Irrelevant The result isn't relevant for the search. It shouldn't appear in results for the search term.
      3. Wahlweise: If search results for the selected search term are missing, add them by selecting Add results, selecting those results, then selecting Add.
        Hinweis:
        If you rate all results for a search term as irrelevant, a message appears prompting you to add missing search results.
      4. Wahlweise: To return to the search term selection page, select Go back to list in the Search terms list.
        Hinweis:
        If no more search terms are available for selection, selecting Go back to list returns you to your list of evaluations instead.
      5. When your review is complete, select Got it.