Quantemplate imports raw data via Integrate pipelines, cleanses and harmonises, then outputs it to the data repo for downstream processes such as querying in Quantemplate Analyse or sending to external systems.
The Data Repo is a storage area for clean datasets and reference data.
Cleansed outputs from pipelines can be exported to the Data Repo for onward sharing via API, sharing within Quantemplate, or reporting on in Analyse.
Datasets with a single row of headers can be uploaded directly to the Data Repo to create a reference dataset for use in pipelines – for example, a header schema to map to. The upload process will ignore any blank rows above or below the data, or blank columns either side of the data. The first line of data will be interpreted as column headers. Read more about uploading data.
Bordereaux or submission data which requires cleansing should be uploaded to a pipeline, rather than the Data Repo.
The best way to upload data to a pipeline is via the Input Selector.
Uploaded files are made available to all stages in the pipeline.
Uploads can be permanently removed from the pipeline via the ‘remove’ button in the Inputs tab. Uploads can also be added to the pipeline in the Inputs tab.
Quantemplate supports XLS, XLSX, CSV and GZipped CSV files.
We are aware of an issue where CSV files which contain a Byte Order Mark and have GZip compression applied may fail to upload. In this case, use an uncompressed CSV format or open the file in a text editor and export as UTF-8 with no BOM.