---
sourceDocument: Xanadu ServiceNow AI Platform Administration
sourceDocumentLink: https://servicenow-prod.fluidtopics.net/r/xanadu/platform-administration

 Release :

    - xanadu

ft:locale :

    - en-US

ft:publication_title :

    - Xanadu ServiceNow AI Platform Administration

ft:clusterId :

    - platadm

bundleId :

    - platadm

workflow :

    - Platform


---

# Configure tables to use the Japanese tokenizer

# Configure tables to use the Japanese tokenizer {#ariaid-title1}

* Release version: Xanadu
* 
* Updated August 1, 2024
* 
* ![](https://www.servicenow.com/docs/portal-asset/ico-clock) 1 minute to read

Improve search results in Japanese language searches by configuring individual tables to use the Japanese tokenizer.

## Before you begin

Role required: admin

## About this task

Japanese language searches use the Kuromoji Japanese tokenizer in Zing. The Japanese tokenizer attribute is active by default, but you must configure individual tables to use it and then regenerate the text index.

## Procedure

1. Navigate to AllSystem DefinitionDictionary.
2. Search for the table name.
3. In the search results, open the table, which is the list item that has an empty Column name and a Type of Collection.
4. Scroll down the page and select the Attributes tab, and then select New.
5. Fill in the fields on the form.  
   {#configure-tables-japanese-tokenizer__table_gfw_5fx_4bb__entry__2}

   | Field | Description |
   |-|-|
   | Attribute | Enter Text index tokenizer language. |
   | Dictionary entry | Defaults to the name of the table. |
   | Value | Enter ja for the Japanese tokenizer. |
   [Table 1. Attributes fields]

   {#configure-tables-japanese-tokenizer__table_gfw_5fx_4bb}
6. Select Submit.
7. [Regenerate a text index for a table](https://servicenow-prod.fluidtopics.net/EhFB579yIrCuwIp2hhyGHA "You can regenerate a table text index when you change table stop words or display values.").  
   Note:  
   Text indexing can be a resource-intensive task that may take a while to complete. You may notice performance degradation or incomplete search results during index generation. To estimate text indexing duration, you can view historical [statistics](https://servicenow-prod.fluidtopics.net/t0_oxn~6xib1FACmS31utg "To view text indexing statistics and status, navigate to All > System Definition > Text Indexes .").

## Result

When the text index is generated and you go to the table you configured, the table now uses the Japanese tokenizer.
**Related concepts**   

* [Zing indexes punctuation as part of some words](https://servicenow-prod.fluidtopics.net/w7uB09y~FMmb2pFvBd6hwg "Zing indexes some punctuation marks as part of some words to improve search results for common search terms.")
* [Zing indexes some HTML elements](https://servicenow-prod.fluidtopics.net/ka_LSPsauIUfAhCQ_t7bIA "Zing indexes the contents of certain HTML elements to improve document search results. This indexing is addition to the normal indexing by word.")  
**Related tasks**   

* [Configure a table for indexing and searching](https://servicenow-prod.fluidtopics.net/ikx52T2SvdSyrAWcJbB0lA#configure-single-table-for-indexing "Enable Zing text indexing for a table to allow users to search its records for string values. Configure the table's text indexing attributes, stop words, and synonym dictionaries to control search behavior.")
* [Configure a text index group to search across multiple tables](https://servicenow-prod.fluidtopics.net/_Tqphpv96UK1ogrveyz4mg "Update tables to the V4 indexing format and include them in a text index group to facilitate searching across multiple tables.")
* [Regenerate a text index for a table](https://servicenow-prod.fluidtopics.net/EhFB579yIrCuwIp2hhyGHA "You can regenerate a table text index when you change table stop words or display values.")
* [Remove an index](https://servicenow-prod.fluidtopics.net/xgsKH0hNHvRqe9szc3ir2A "You can remove the index for a table if you no longer want the search engine to return results for that table. This procedure also removes the index for all tables that extend the specified table.")
* [Remove an index for a specific field](https://servicenow-prod.fluidtopics.net/CSQJCQNy12hVM~sn8G3m3g "You can remove the index for a specific field in a table if you no longer want the search engine to return results for that field.")
* [Remove the text index for a child table](https://servicenow-prod.fluidtopics.net/CO6UXxVULBBlub6YaQDqtg "You can remove the text index for a child table if you no longer want the search engine to return results for that child table.")
* [Change the query mode of an indexed table](https://servicenow-prod.fluidtopics.net/PrqCDK3ZPZwBhHgu2C02BQ "Set the query mode to specify whether searches on an indexed table must match all or only some of the search terms.")
* [Enable indexing of text in multi-row variable sets](https://servicenow-prod.fluidtopics.net/h3Rx9FN5RISMSrF0Tjb9sA "Make text from multi-row Service Catalog variable sets searchable in global search.")  
**Related reference**   

* [Zing index and search dictionary attributes](https://servicenow-prod.fluidtopics.net/cov1YoVQP8FsNSEuqNnsAg "The following dictionary attributes either affect the way Zing indexes tables or ranks search results.")
* [Text indexing statistics and status](https://servicenow-prod.fluidtopics.net/t0_oxn~6xib1FACmS31utg "To view text indexing statistics and status, navigate to All > System Definition > Text Indexes .")

*[\>]: and then