Help Centre

Inputs and outputs
Pipeline inputs

Quantemplate imports raw data via Integrate pipelines, cleanses and harmonises, then outputs it to the data repo for downstream processes such as querying in Quantemplate Analyse or sending to external systems.

About the Data Repo

The Data Repo is a storage area for clean datasets and reference data.

Cleansed outputs from pipelines can be exported to the Data Repo for onward sharing via API, sharing within Quantemplate, or reporting on in Analyse.

Datasets with a single row of headers can be uploaded directly to the Data Repo to create a reference dataset for use in pipelines – for example, a header schema to map to. The upload process will ignore any blank rows above or below the data, or blank columns either side of the data. The first line of data will be interpreted as column headers. Read more about uploading data.

Uploading data to a pipeline

Bordereaux or submission data which requires cleansing should be uploaded to a pipeline, rather than the Data Repo.

The best way to upload data to a pipeline is via the Input Selector.

  1. Open a stage.
  2. Click add inputs.
  3. Go to the Uploads tab and add files via drag-and-drop or the ‘choose files’ button.
  4. Select the uploaded files using the checkbox and click ‘Apply Selection’.

Uploaded files are made available to all stages in the pipeline.

Removing uploads

Uploads can be permanently removed from the pipeline via the ‘remove’ button in the Inputs tab. Uploads can also be added to the pipeline in the Inputs tab.

Supported file formats

Quantemplate supports XLS, XLSX, CSV and GZipped CSV files.

GZipping a CSV file can significantly reduce its file size and upload time. 7-Zip is useful tool for applying GZip compression. Mac users can also use a Terminal command.

We are aware of an issue where CSV files which contain a Byte Order Mark and have GZip compression applied may fail to upload. In this case, use an uncompressed CSV format or open the file in a text editor and export as UTF-8 with no BOM.