Data normalization
Summarize
Summary of Data Normalization
Data normalization in ServiceNow standardizes data extracted from documents into a uniform format across all fields, enhancing data usability for analysis and integration with other applications. With the Zurich release, the Document Intelligence feature is being prepared for deprecation, shifting focus to the Now Assist in Document Intelligence application for data extraction tasks.
Show less
Key Features
- Field Types: Various field types are converted for normalization, including:
- Date: Standardized to YYYY-MM-DD format.
- Reference Field: Matches extracted data to a standard reference in another table.
- Integer: Represents whole numbers, e.g., 12.
- Decimal: Supports numbers with up to two decimal places, e.g., 12.5.
- Floating Point Number: Allows up to seven decimal places, e.g., 12.0000000.
- Display: Extracted data is shown alongside converted values, with options to edit if necessary.
- Ambiguous Data Handling: DocIntel interprets ambiguous values based on predefined configurations, requiring user confirmation in some cases.
Key Outcomes
By implementing data normalization, ServiceNow customers can enhance data accuracy and facilitate better analysis and integration across their platforms. Users should be aware that errors in data extraction may require manual corrections to maintain data integrity. The transition to Now Assist in Document Intelligence will ensure continued support while improving data extraction processes.
Certain types of data extracted from documents are converted into a standard format so that they appear the same across all fields.
This process increases the usefulness of the data by enabling it to be grouped and analyzed more easily. It also supports integration with other applications on the ServiceNow AI Platform.
Field types
The following field types are converted to support data normalization:
| Field type | Description |
|---|---|
| Date | Standard date format. For example, YYYY-MM-DD. |
| Reference field |
A field that uses a field in another table as a standard. DocIntel matches the extracted data to the standard. For example, a use case has a reference field called Vendor that points to the Name column in the Company table as the reference. When processing a document task, DocIntel extracts “Degas Dairy Products, Inc” from the document and fills the Vendor field with that value. DocIntel compares the value to the company names in the reference table and finds “Degas Dairy Products, Inc” as a match. In the document task, “Degas Dairy Products, Inc” is matched to “Degas Dairy Products, Inc” in the reference. |
| Integer | Whole number. For example, 12. |
| Decimal | Number with up to two decimal places. For example, 12.5 or 12.55. |
| Floating point number | Number with up to seven decimal places. For example, 12.0 to 12.0000000. |
To set the field type, see Create a field for data extraction.
Display
A completed data extraction field shows the converted value next to it.
You can adjust the converted date value by selecting Edit.
Ambiguous data
If there is data in a document that can be understood in more than one way, DocIntel interprets that value based on the default selected for it in the use case configuration. DocIntel must interpret an ambiguous value in order to accurately convert it to the normalized format.
For example, a use case has a Date field, and Month first is selected as the default order to interpret ambiguous dates. When a document containing the date 1/2/2024 is processed for the use case, DocIntel interprets that date as January 2, not February 1, when it extracts that value and converts it.
In such cases, the user completing a document task may need to confirm or correct the conversion of ambiguous values. Depending on the field’s configuration in the use case, automated document processing may be interrupted to ensure the conversion is accurate.