Quantemplate Integrate takes raw input data and
processes it via a transformation pipeline to produce cleansed output
This video shows you how to get started creating and running pipelines.
For more detailed video walk-throughs see Tutorials.
Data table concepts
Quantemplate Integrate takes data in unstructured/semi-structured
formats such as spreadsheets, and transforms them into harmonised, structured data tables.
simple structure consisting of a fixed set of columns and an arbitrary number of rows.
are the named
set of values that each row of the data table will contain. Conventionally these are
represented as a set of headers at the top of a table.
individual data points. In Quantemplate, the data values can be of any type but other
data processing systems you may wish to connect to Quantemplate sometimes apply strict
rules about the type of data that may be held in each column.
of related data values, with one value for each column, usually represented as
horizontal rows in Quantemplate.
How Quantemplate Integrate is different from Excel
Unlike spreadsheet tools such as Excel, Quantemplate Integrate applies a rules-based approach
to configures batch processing actions across multiple datasets. This allows you to quickly cleanse and harmonise data at scale,
and for your processes to be repeatable on similar sets of source data.
Because it's built for defining data processing rules, Quantemplate Integrate does not allow:
Cell-level alterations of individual values.
To change a value, set up a transformation targeting the
type of value you wish to change.
Table presentation such as multiple layers of header,
headers along the side of the table, totals of rows or columns.
provides tools to configure table presentation, add totals, apply filters, etc.
Presentational number formatting such as millions, billions, %.
provides tools to apply presentational formatting to numbers numbers.
A pipeline is a data transformation process built for a
set of input datasets, transforming them to a desired set of output datasets. A
Raw data for cleansing is uploaded directly via the
inputs interface. Quantemplate supports data in XLS, XLSX, CSV and GZipped CSV formats. Cleansed
data such as other pipeline outputs or reference codes can be connected in from
your data repo.
Stages and operations
Data transformations are sequenced and configured via
stages and operations.
components with a defined number of input and output datasets, whilst
are individual transformation
process, grouped together in a transform stage. See
Stages and operations for more details.
Each stage creates output datasets which are the result
of the input data, plus the stage transformations. A stage’s output datasets can
be connected to the inputs of a subsequent stage for further transformations,
or can be exported to your data repo,
downloaded or shared
with another organisation.
Pressing the run button executes the pipeline
transformations and creates the output datasets. The output datasets
from each run of the pipeline are retained, and can be previewed, exported
or downloaded. This allows comparison of different versions of your pipeline
outputs, which may be useful if source data or transformation operations