To enable Verodat to prepare and request data for your organisation, you must define the target state for the data that you need. To do this, you must create a dataset.
A dataset specifies the target schema for how your data should be output. This includes a list of fields and rules to define how your fields should be validated, transformed, and/or enriched.
Users or data suppliers can upload data into a dataset to validate and transform it to your desired state.
Table of Contents
Setting up a dataset
Datasets can be set up and edited in the design tab of your workspace.
Create a clear name and description for the data in your dataset
To start building your dataset, click on the " + New Dataset" button from your work-space home page.
Enter a name and description for your dataset. This will tell your data suppliers, partners and colleagues more information about what data your dataset contains and the purpose of collecting that data.
Click "Save and Continue".
Add target fields to the Data Store
In the 'Define target fields' window, click the " + Add Target Field" button to add a new target data field.
In the pop-up that opens, specify the method type for adding a target field.
Add a Single Field
Upload from file
Specify the name, description and data type of the new field.
This description will help your data suppliers and colleagues understand what information is required in each field and help them map their data to yours if required.
The data type can be string(text), number, integer or date.
By specifying this type, Verodat will automatically validate that the received data matches the expected type and will return a critical error if the submitted data is not in the expected format.
Specify whether the field is mandatory or not using the checkbox.
If a field is marked as mandatory, then blank or null data entries into the field will trigger a critical error.
Specify whether the field is part of the primary key using the checkbox.
Specify whether the field is compound using the checkbox:
select target fields to create a compound key
specify a delimiter for merge fields.
specify maximum characters
Click "Add Field" once completed and repeat this action for any additional fields that you require. You can do this in multiple stages if required.
Press "Save" to save all fields against the data store.
Once created, your dataset will be available on your workspace home page
Remember - Datasets Descriptions can be edited until you have performed the first upload of data into this dataset. Once your first upload has been completed, your dataset description is locked, and cannot be edited without contacting support.
You can add additional Target Fields to a dataset after it has been set up.
You can also build rules to be applied to your dataset.
Add validation rules to the Data Store
Enter the 'Rules' window.
Select your rule type by clicking the '+ Add Validation Rule' button.
Specify the name of the rule and the target field it applies to in the popup
See the article on Creating Validation Rules here
Repeat the above steps for all validation rules that you wish to add, and then move to add transformation rules.
Add transformation rules to the Data Store
Enter the 'Rules' window.
Select your rule type by clicking the '+ Add Transformation Rule' button.
Specify the name of the rule and the target field it applies to in the popup
See the article on Creating Transformation Rules here
Use the rule builder functions to apply validation rules to your data.
Click "Save" in the rule builder and the popup window to add the transformation rule.
Repeat the above steps for all transformation rules that you wish to add, and then click "Save" in the Rules window.
You have now completed the setup for a dataset and can begin uploading data.