Append metadata allows dataset and pipeline metadata to be appended to the output data as new columns. The fields that can be appended are:
Section |
Function |
Description |
Pipeline |
Pipeline Auto Run by |
If a pipeline run was initiated by an Auto Run trigger updating,
record the trigger datasets or feeds that caused the run. |
Pipeline ID |
The ID is the unique reference Quantemplate gives to every pipeline and is used in the pipeline’s URL.
This is useful for audit purposes where it is necessary to prove that a dataset originated from a specific pipeline. |
|
Pipeline input row count |
The row counts of inputs to a stage that come from feeds, uploads or reference datasets.
Inputs from Stage Outputs are not counted. Rows are counted at the point at which they enter the stage.
Use in combination with Stage Input Filename to identify the input whose rows are being counted. |
|
Pipeline name |
The name of the pipeline. This is useful to augment the Pipeline ID above. |
|
Pipeline revision number |
The revision number of the pipeline at run time.
This is useful for audit purposes where it is necessary to ascertain that which pipeline revision number produced a dataset. |
|
Pipeline run by user |
Name and/or email of user who ran the pipeline. |
|
Pipeline run date |
The date and/or time of the start of the pipeline run.
Date-time will be recorded in the timezone set in this operation, which may be different from the user’s local timezone.
This ensures consistent results when working with teams across multiple time zones, or across daylight-saving time periods.
The field can optionally include the date, time and timezone in the same column. |
|
Pipeline run number |
The pipeline run number which generated the output dataset.
This is useful to link a dataset to a specific run number, enabling review of validations and seeing time of run. |
|
Stage input filename |
The name of the input file entering the stage.
This is useful for tracing the input file or extracting a company name or date from the file name. |
|
Feeds |
Deadline status |
When using a schedule,
records whether a submission-subsection was submitted on time or was overdue.
If overdue, returns ‘Overdue’. If not overdue, returns a blank. |
Feed ID |
The ID is the unique reference Quantemplate gives to every feed and is used in the feed’s URL.
This is useful for audit purposes where it is necessary to prove that a dataset originated from a specific feed. |
|
Feed name |
The name of the feed. This is useful to augment the feed ID above. |
|
Period end date |
If the feed uses a schedule to create submissions,
this records the date-time of end of the submission’s schedule period. |
|
Period start date |
If the feed uses a schedule to create submissions,
this records the date-time of start of the submission’s schedule period. |
|
Resubmission status |
If data from a feed was a resubmission, returns the value ‘Resubmission’, or otherwise a blank. |
|
Schedule end date |
If the feed uses a schedule to create submissions,
this records the date-time the schedule ceases running. |
|
Schedule name |
If the feed uses a schedule to create submissions,
this records the name of the schedule. |
|
Schedule start date |
If the feed uses a schedule to create submissions,
this records the the date-time the schedule started running. |
|
Stage input is additive / only latest |
If auto update has been set in the stage inputs, this records
whether the ‘Additive’ or ‘Only latest’ has been set. Can be used to indicate whether data is Inception-to-Date or Transactional.
Returns the values ‘Additive’ or ‘OnlyLatest’.
|
|
Submission created date |
Date-time the submission was created. This could include when the submission was created as a draft. |
|
Submission data version |
Version number of submission data. |
|
Submission first requested date |
Date-time the data was first requested. |
|
Submission name |
Name of the submission. |
|
Submission requested by user |
Name and/or email of user who created the feed. |
|
Submission uploaded date |
Date-time the data was uploaded. |
|
Submitted by user |
Name and/or email of the user who submitted the data. |
|
Subsection name |
Name of the submission-subsection. |
|
Reference datasets |
Dataset information |
Select one of the reference datasets used in the pipeline then toggle on
options to write the ID, name, row count, updated date or revision number of the dataset as columns of metadata.
To get the metadata for additional datasets, add another ‘Dataset information’ function to the Append Metadata operation. |
If there are more metadata fields which would be useful to you, please contact support.
To add and configure the Append Metadata operation:
Append the filename to multiple stage inputs, then union the files.
Input name: Acme-bdx-2022-08 |
|
Policy No |
Premium |
H24788 |
100 |
H84729 |
120 |
H89722 |
130 |
Input name: NewCo-bdx-2022-08 |
|
Policy No |
Premium |
8723XYZ |
140 |
8738GHI |
150 |
8264GUI |
160 |
Field: Filename
Column name: Input file
Feed both outputs from Append Metadata into a Union stage.
Output from Union stage |
||
Policy No |
Premium |
Input file |
H24788 |
100 |
Acme-bdx-2022-08 |
H84729 |
110 |
Acme-bdx-2022-08 |
H89722 |
130 |
Acme-bdx-2022-08 |
8723XYZ |
140 |
NewCo-bdx-2022-08 |
8738GHI |
150 |
NewCo-bdx-2022-08 |
8264GUI |
160 |
NewCo-bdx-2022-08 |
Note: the Append Metadata operation replaces the Append Filename operation. Append Filename will continue to be supported in existing pipelines but should not be added to new pipelines.