Managing data fabric tables in Workflow Data Fabric Hub
Summarize
Summary of Managing data fabric tables in Workflow Data Fabric Hub
Data fabric tables in Workflow Data Fabric Hub provide a virtual representation of data stored externally or locally, accessible within the ServiceNow AI Platform. This enables interaction with external data as if it were stored locally, without duplicating it. Data stewards are responsible for creating and managing these tables.
Show less
Viewing and Managing Data Fabric Tables
Users can view and search all data fabric tables on the Data fabric tables tab, filtering by label, data source, connection, or creator. Tables from active and inactive connections are organized under separate tabs. Data fabric tables can be created, updated, deleted, and connected to different data sources. Additionally, they can be deployed between instances using update sets.
Primary Keys
Primary keys uniquely identify records in data fabric tables and enable references from other tables. These keys are usually derived automatically from the source table. If absent, a unique column can be designated as the primary key in the target table. Important considerations include:
- Primary key selection does not enforce uniqueness constraints; uniqueness must be ensured manually.
- Once defined, a primary key cannot be removed or modified without deleting and recreating the table.
- Without a primary key, creating references or performing searches is not possible.
- Reference columns cannot be used as primary keys.
Reference Tables
Data fabric tables can reference other data fabric tables or local tables to establish relationships and maintain data integrity. To reference a local table, an instance administrator must create a specific ACL rule permitting data fabric operations. References require:
- The referenced table to have a primary key with unique values matching the source column's data type.
- The referenced column’s display attribute set to true to show values properly.
Table Queries and Data Lifecycle
Data fabric tables query the external data source each time they are accessed. Retrieved data is held temporarily in memory only while the list or form is open and purged afterward, ensuring data is always current but transient.
Limitations
- Data sources must not have case-sensitive table names, as this prevents schema loading.
- References can only be made to tables with a single primary key column; composite keys are not supported.
- Only one primary key column can be used as a reference key.
Practical Actions for ServiceNow Customers
- Create data fabric tables to integrate external or internal data sources virtually.
- Define appropriate primary keys to enable record uniqueness and referencing capabilities.
- Establish references between tables to maintain data relationships and integrity, ensuring proper ACL configuration.
- Manage data fabric table lifecycle including updating mappings, changing connections, viewing records, and deleting tables as needed.
- Deploy data fabric tables across instances using update sets for consistent environments.
Access data virtually from an external source or a local table on your instance using data fabric tables in Workflow Data Fabric Hub.
Overview of managing data fabric tables
A data fabric table is a virtual representation of data stored in another source, accessible within the ServiceNow AI Platform. It enables users to interact with data from another source as if it's stored locally, without duplicating the data. Data stewards are responsible for creating and managing data fabric tables using the process shown in this infographic.
Viewing data fabric tables
View a list of all the data fabric tables that data stewards have created on the Data fabric tables tab.
- Search for a data fabric table by label or name.
- Filter the list of tables by data source and connection.
- Filter the list of tables by creator.
- View a list of data fabric tables from active connections in the Active tab.
- View a list of data fabric tables from connections that are deactivated or not configured in the Others tab.
Defining primary keys
A primary key is used to indicate uniqueness and identify records in a data fabric table. Designating a column as a primary key enables you to create references to that column from other data fabric tables. When mapping data in a new data fabric table, primary keys from the source table are automatically defined in the target data fabric table. If the source table doesn’t have a primary key, you can create one in the target table by choosing a column with unique values.
Note the following details when defining primary keys:
- Selecting a primary key doesn't enforce primary key constraints on the data fabric table or on the remote system. You must verify that records identified by the primary key are unique.
- Once you define a primary key, it can't be removed. If you determine that you need to change the primary key later, you must delete and recreate the data fabric table first.
- If you decide not to define a primary key when creating a data fabric table, you can do so by editing the table later. Note that without a primary key, you can't create references to this table and searches won't work until a primary key is defined.
- For every primary key column identified in the source table, a corresponding primary key is automatically created in the data fabric table. Once the data fabric table is created, you can't modify the primary keys that were derived from the source table.
- You can't use a reference column as a primary key.
Reference tables
When mapping external source columns to a data fabric table, you can add references to other tables. Establishing references between tables creates a relationship between the two tables and provides data integrity. A data fabric table can reference another data fabric table or a local table on your instance. A local table can also reference a data fabric table.
Note the following requirements when creating references between tables:
- To reference a local table on your instance from a data fabric table, an instance administrator must create an ACL rule with the data_fabric record operation on the local table. For example, to create a reference to the
User [sys_user] table, the following ACL rule is required:
- Type: record
- Operation: data_fabric
- Name: User [sys_user]
- To see values in a reference column when viewing records in the new data fabric table, you must set the Display attribute to true for the referenced column. You can manage this attribute in the referenced table's definition. See Select a field as the table display value.
- Ensure the referenced table has at least one primary key column.
- Ensure the primary key column in the referenced table contains unique values.
- Ensure the primary key column matches the data type of the source column.
Table queries
Records retrieved from a data stored and stored in a data fabric table have a finite duration, or life cycle, within the ServiceNow AI Platform.
After a data fabric table is connected to a data source, each time the data fabric table is viewed or accessed, a new query is made to the data source. The retrieved data lives in memory for as long as the list or form appears. After you close the list or form, that retrieved data is purged from memory. The next time that you use or view the retrieved data in the data fabric table, records are repopulated from the data source.
Data fabric table limitations
- When mapping data, ensure that the data source does not use case-sensitive table names. For example, if both MY_TABLE and my_table exist in the database, the schema will not load because the system cannot differentiate between the tables.
- When creating a reference to another table, you can only select one primary key column as the reference key. You can't select a composite key as the reference key.
- When creating a reference to another table, you can only reference a table that has one primary key defined.