Ingestion Data Types

Getting Started Developers Downloads Science Corner
Search…
Data Model Loader support a number of field types during ingestion of raw files.  The following field types are supported with the outlined configurations. Note that during ingestion, all values in a field must conform to the type format supported and indicated.  This includes special values used to indicate "Sparse" values.
For multi-select fields, the field in the raw data csv would be entered as "<value 1>|<value 2>|<value n>" with a "|" delimiter for entries where multiple values selected.  Note that not all entries require multiple entries for a field to be treated as multi-select. The "value n",  part of the multi-select must be non-null / non-empty. 
Date
A date (i.e. timepoint with day granularity) without special values (other than NULL). An example is Date of Birth
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
date
​
​
​
​
The following formats are supported for Date
yyyy
yyyy-[m]m
yyyy-[m]m-[d]d
Date Sparse
A date  with a few special values (other than NULL). An example is Date of Diagnosis where a date of "1900-01-01" indicates "Forgot to ask question". 
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
date
<provided>
yes
​
​
Datetime
A datetime without special values (other than NULL). An example is datetime of hospital admission.   Datetimes are stored in the system as UTC.
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
datetime
​
​
​
​
The following formats are supported for Datetime
yyyy
yyyy-[m]m
yyyy-[m]m-[d]d
yyyy-[m]m-[d]d [h]h:[m]m:[s]s.[ms][ms][ms][us][us][us]
yyyy-[m]m-[d]d [h]h:[m]m:[s]s.[ms][ms][ms][us][us][us]Z
yyyy-[m]m-[d]d [h]h:[m]m:[s]s.[ms][ms][ms][us][us][us]-[h]h:[m]m
yyyy-[m]m-[d]d [h]h:[m]m:[s]s.[ms][ms][ms][us][us][us]+[h]h:[m]m
yyyy-[m]m-[d]dT[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us]
yyyy-[m]m-[d]dT[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us]Z
yyyy-[m]m-[d]dT[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us]-[h]h:[m]m
yyyy-[m]m-[d]dT[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us]+[h]h:[m]m
Datetime Sparse
A datetime (i.e. timepoint with microsecond granularity) with a few special values (other than NULL). An example is Datetime of Treatment where a date of "1900-01-01 01:00" indicates "NA". 
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
datetime
<provided>
yes
​
​
Float
A double-precision floating point measurement without special values (other than NULL). An example is Weight.
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
float
​
​
​
​
The following formats are supported for Float
#.#
#.#E#
-#.#E#
Float  Sparse
A double-precision floating point measurement with a few special values (other than NULL). An example is Blood Glucose Level where a value of "-1.0" indicates "Read Failure".
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
float
<provided>
yes
​
​
Integer
An integer measurement without special values (other than NULL). An example is Age.
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
integer
​
​
​
​
The following formats are supported for Integer
#
Integer Sparse
An integer measurement with a few special values (other than NULL). An example is Cups of Coffee Consumed where a value of "-1" indicates "Prefer not to answer".
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
integer
<provided>
yes
​
​
Integer Categorical
A single-choice value (integer) from a controlled vocabulary of possible options (or NULL), originally encoded as integer. An example is Have you traveled in the last 30 days? where a value of "1" means "Yes" and a value of 0 means "No"
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
integer
<provided>
​
​
​
Integer Categorical supports all of the same formats as the Integer field
Integer Categorical Hierarchical
A single-choice value (integer) from a controlled vocabulary of possible options (or NULL) that are organized in a hierarchy, originally encoded as integer. An example is Job Code At Visit where an integer  of "12" means "VP of Operations" which is a child of "Corporate Managers And Senior Officials". 
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
integer
<provided>
​
​
<provided for child values>
Integer Categorical Multi-select
A multiple-choice value (integer) from a controlled vocabulary of possible options (or NULL), originally encoded as integer. An example is Patient Ethnic Background where  0 means "African", 1 means "Caucasian", 2 means "Asian"... etc, and a patient may be part of one or many races.  
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
integer
<provided>
​
yes
​
Integer Categorical Multi-select Hierarchical
A multiple-choice value (integer) from a controlled vocabulary of possible options (or NULL)  that are organized in a hierarchy, originally encoded as integer. An example is Cancer Codes Self Reported where  0 means "Parotid Gland Cancer", 2 means "Other Salivary Gland Cancer" and both have a parent of 3 "Salivary Gland Cancer" .. etc, and a patient may have one or more cancer codes entered.
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
integer
<provided>
​
yes
<provided for child values>
String
A free-form string without special values (other than NULL). An example is UUID  or Special Instructions.
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
string
​
​
​
​
The following formats are supported for String
any string
for quote escaping use "" (e.g. use <"the ""quote"" phrase"> to get  <the "quote" phrase>)
String Categorical
A single-choice value (string) from a controlled vocabulary of possible options (or NULL), originally encoded as string. An example is T-Shirt Size where "S" means "Small"
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
string
<provided>
​
​
​
String Categorical supports all of the same formats as the String field
​
String Categorical Hierarchical
A single-choice value (String) from a controlled vocabulary of possible options (or NULL) that are organized in a hierarchy, originally encoded as string. An example is State of Residency where a value of "CA" means "California" and a value of "OR" means "Oregon" and both values have a parent of "US" that means "United States of America".
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
string
<provided>
​
​
<provided for child values>
String Categorical Multi-select
A multiple-choice value (string) from a controlled vocabulary of possible options (or NULL), originally encoded as string. An example is Primary Diagnosis where  values are encoded to match ICD10 diagnostic codes.  
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
string
<provided>
​
yes
​
String Categorical Multi-select Hierarchical
A multiple-choice value (string) from a controlled vocabulary of possible options (or NULL)  that are organized in a hierarchy, originally encoded as string. An example is Secondary Diagnosis where  an entry may have one or more values encoded to match .
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
string
<provided>
​
yes
<provided for child values>
String Categorical Sparse 
A single-choice value (string) from a controlled vocabulary of possible options (originally encoded as string) or a write-in value (or NULL). Most values are expected to be from the controlled vocabulary. An example is Prescribed Medication where each entry is a single item from a list of RXNORM medications where "RX5640" means "Ibuprofen", etc. but a value may be written in if the medication is not available as an option.
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
string
<provided>
yes
​
​
String Categorical Sparse Hierarchical
A single-choice value (string) from a controlled vocabulary of possible options that are organized in a hierarchy (originally encoded as string) or a write-in value (or NULL). Most values are expected to be from the controlled vocabulary. An example is Previous Diagnostic Codes ICD-10 where the code "D03" has meaning "Melanoma in situ", and the code "D03.0" has meaning "D03.0 Melanoma in situ of lip", and "D03.0" is hierarchically situated under "D03", or any other free-form string that is not part of ICD10 like "Other trauma".
Ingestion files are configured as follows:
type
coding_name
is_sparse_coding
is_multi_select
(coding.csv) parent_code
string
<provided>
yes
​
<provided for child values>