The Data section, or Data Repo, is the storage area for clean datasets and reference data.
Datasets with a single row of headers can be uploaded directly to the Data Repo to create a reference dataset for use in pipelines – for example, a target header schema to map to. The upload process will ignore any blank rows above or below the data, or blank columns either side of the data. The first line of data will be interpreted as column headers.
Read more about uploading data to the data repo.
Cleansed outputs from pipelines can also be exported to the Data Repo for onward sharing via API, sharing within Quantemplate, or reporting on in Analyse.
Datasets are organised in your Data repo. To access your datasets, click the folder button next to the dataset name: the repo view opens.
Datasets are sorted alphabetically by default by default.
The repo view shows:
Tags can be applied to any document in Quantemplate, to help organise assets.
Tags are created by Org Admins within the repo views for Feeds, Pipelines, Data and Analyse. They can be applied to documents by document owners. Tags are set at an organisation level and are available to all document types.
Within the repo sidebar, users see all the tags for documents they have access to. Clicking on a tag in the sidebar filters the document list by that tag.
Datasets can sorted by dataset name, updated date and user last updated by. Click the arrow button to reverse the sort direction.
Click in the search field to search datasets by name.
Click the filter icon or hit ‘F’ on the the keyboard to filter datasets by date created, date updated, dataset name, owner, updated by user.
Click the three dots at the end of the row and select ‘Duplicate’ from the popup menu. The dataset is instantly duplicated and the document opened. The suffix ‘– copy’ is added to the duplicate dataset’s name.
Click the three dots at the end of the row and select ‘Rename’ from the popup menu. Alternatively, edit the dataset name by opening the document and clicking on the name. Name changes propagate throughout the system – any reports or pipelines using that dataset will now display the updated name.
Click the three dots at the end of the row and select ‘Archive’ from the popup menu. Archived documents are not deleted, but hidden from the repo and the pipeline input selector
Archived datasets are ignored by the pipeline and will prevent a pipeline from running if used in a Join stage.
Archived datasets display a warning in the top right of the dataset view.
Note that datasets in Quantemplate cannot be deleted permanently by users. Contact us if you have a specific deletion requirement.
The full change history of a dataset is logged in the Dataset History popup, on the top right.
The dataset history shows a dataset's revision number, date/time of revision, last editor, and the name of the dataset at the time of revision.
Dataset owners and editors can restore to a previous version of a dataset by clicking the green restore button to the right of the dataset name in the popup. The restored version will become the latest revision of the dataset.