Detect Headers
Detect Headers is a templated script operation
which converts the first row(s) of the input data into column headers.
This operation is also useful when you have a multi-row header which needs
to be collapsed into a single header, or multiple input files with different header rows.
Usage
useFirstNRowsAsHeader:
rowCount: 1
Where
rowCount
is the number of rows that make up the header.
Input the number of rows, starting from the top of the input files, that
are to be used as column headers.
The maximum row count is 10
Example
If you had an input file of:
and you select the first two rows to be the column names,
useFirstNRowsAsHeader: { rowCount: 2 }
then your output would be:
2015: Q1
2015: Q2
2015: Q3
2015: Q4
2016: Q1
2016: Q2
2016: Q3
2016: Q4
Multiple inputs with different header structures
The rowCount function works well when input files have an identical numbers of rows in the header. In scenarios where input files have different numbers of rows in the header, the function while: hasNoNumericValue can help.
After adding the first row to the header, this looks at each subsequent row in-turn adding them to the header if they have no numeric column value, up-to the number of rows defined by rowCount.
Usage
useFirstNRowsAsHeader:
rowCount: 3
while: hasNoNumericValue
Example
You have an input file with two tabs. The first tab has two rows in the header, the second tab has only one row in the header:
First tab
Second tab
Y2015: Q1
Y2015: Q2
Y2015: Q3
Y2015: Q4
Y2016: Q1
Y2016: Q2
Y2016: Q3
Y2016: Q4
Using while allows the tabs to be processed together.
useFirstNRowsAsHeader:
rowCount: 2
while: hasNoNumericValue
This will take the first row and any subsequent rows which do not have a numeric value as the header, up to a maximum of 2 rows.
