Creating Data Types with a List of Expected Values

Data types can be created with a user-provided list of values – we call these "list custom field data types". List CFDTs enable the system to more accurately transcribe fields where a defined list of values is expected – for example, a list of account types, or a list of states or provinces.

How to define a list CFDT

The image below is the Data Type Information page, and it is where you can manually add values to the list CFDT, import a CSV, and update values as needed.

DataTypeExpectedValuesv30.png

Creating a new data type from scratch

To create a new data type using a custom list of values:

  1. Navigate to Library > Data Types.

  2. Click the Create Data Type button in the top right of the screen.

  3. In the modal, enter the desired name for the new data type, set the status, and then click Next.

  4. From there, select Create from a List of Values > Next.

To upload several list CFDTs at once, follow the instructions to create a CSV file that you can import into the system. If you need to see an example, download a sample CSV file from the option in the lower left of the import modal. 

  1. Your CSV file must contain a list of accepted values (e.g. the values the system should expect to see for fields of this type) and normalized values, or the standardized values the system will use to normalize a given accepted value (e.g. "United States of America" is normalized as "US.")

  2. Choose the desired CSV file for upload, then click the Next button to begin processing the file. If any issues are encountered while processing the file, an error or warning message will appear.

  3. After the CSV file is processed, the data type information will be shown, including the name, status, and list of Accepted and Normalized Values. Change the list of values by clicking the Import .csv option above the table and submitting a new CSV file.

  4. If you are satisfied with the list, select the Create button in the top right to confirm.

Alternatively, you can create list CFDTs directly in the UI by selecting Next on the screen where you are asked to upload a CSV.

Editing an existing data type

You can edit an existing list CFDT by going to Library > Data Types and selecting the CFDT from the table. You will then be taken to the Data Type Details page, where you can select Edit to make changes to the list of accepted values and normalization for that data type.

All edits made to a list CFDT that is currently used in a live release will update automatically by the system. This means that you do not have to edit the CFDT, update the layouts that use that CFDT, lock that layout and create a new release to deploy.

Alternatively, you can download the CSV of an existing list CFDT, edit it, and import it into the system to replace the existing list CFDT. This is valuable because you can edit multiple list CFDTs such that the changes propagate across layouts, including the live layouts.

There are two ways to download the CSV:

  1. In Library > Data Types, click on the blue cloud icon in the "Actions" column of the respective data type.

  2. In Library > Data Types, click on the list CFDT name from the table and then select "Download CSV" right above the table of values.

Once you download the CSV, you can edit it to update/remove/add new values to the list CFDT and import it back into the system to replace the existing list CFDT. Note that the values in the new CSV will replace all of the existing values. Upon saving, there will be a modal that indicates the changes have been made.

The .CSV file must meet the following criteria in order to be properly processed:

  • The maximum number of values for import is 200,000

  • Headers must be present for all each column

  • A value already present in the Normalized Value column cannot be listed in the Accepted Value column with a new normalized value

Several notes regarding this functionality:

  • The filename will appear in the audit log whenever a user creates or updates a list CFDT by uploading a CSV file or via the UI.

  • Fields that are sent to Supervision can be transcribed with values that are not present in the list of accepted values. However, doing so will then output the field with a normalization error. For more information on Supervision, see What is Supervision?

  • When a field defined by an expected list of values is sent to Supervision for transcription, the system will notify the keyer when the entered value is not included in the list of accepted values for that data type. As with other data types, the formatting restrictions on fields using the custom list of values can be overridden to remove this feedback.