transformations:filesplitter
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision | ||
transformations:filesplitter [2021/03/10 12:51] – craigt | transformations:filesplitter [2021/07/19 02:35] – [Examples] craigt | ||
---|---|---|---|
Line 1: | Line 1: | ||
{{ transformations: | {{ transformations: | ||
- | =====SPLIT DELIMITED | + | ======SPLIT DELIMITED FILE ====== |
- | (Category: //Import | File//) | + | Category: |
+ | \\ | ||
=====Description===== | =====Description===== | ||
- | This action aims to help in processing very large text files. It splits a text file into smaller chunks (which are also text files) that can further be processed using iterations. The chunks can be either of fixed length (by row count), or split by unique values of a column (e.g. one chunk per unique Date). | + | This action aims to help in processing very large text files. It splits a text file into smaller chunks (which are also text files) that can further be processed using iterations. The chunks can be either of fixed length (by row count), or split by unique values of a column (e.g. one chunk per unique Date).\\ |
- | + | ||
- | Since File Splitter doesn' | + | |
- | In addition to splitting the input file, this action outputs a single-column dataset containing a list of the split files' filenames. | ||
- | \\ | ||
\\ | \\ | ||
- | |||
=====Action settings===== | =====Action settings===== | ||
^ Setting | ^ Setting | ||
- | |Input file|Fully-qualified file name of the text file to be split (includes relative or absolute path).| | + | |Input file< |
|Encoding|ASCII, | |Encoding|ASCII, | ||
- | |Skip first lines|This option specifies how many lines to skip from the beginning of a file. It can be helpful in cases where first rows contain\\ some metadata (file header) and actual tabular data starts only after that metadata. This setting can be specified\\ using a parameter as well.| | + | |Skip first lines< |
- | |Output folder|The folder location the split files will be saved to.| | + | |Output folder< |
- | |Output mode|The method by which the input file will be split.\\Options: | + | |Output mode|The method by which the input file will be split.\\Options: |
- | + | <sup>*</sup> Setting can be specified using a [[: | |
- | ^**//Output mode// settings**^^ | + | \\ |
+ | ====Output mode settings==== | ||
^Setting^Description^ | ^Setting^Description^ | ||
- | |Split by fixed row count|The input file will be split into a new file every // | + | |Split by fixed row count|The input file will be split into a new file every // |
|Split by column value|The input file will be split into a new file for every unique value found in the specified column. | |Split by column value|The input file will be split into a new file for every unique value found in the specified column. | ||
+ | < | ||
+ | |||
+ | \\ | ||
+ | =====Remarks===== | ||
+ | Since this action doesn' | ||
+ | |||
+ | In addition to splitting the input file, this action outputs a single-column dataset containing a list of the split files' filenames.\\ | ||
\\ | \\ | ||
=====Examples===== | =====Examples===== | ||
- | **Example 1: | + | **Example 1: |
- | > (Split by fixed row count) Chunk size (rows) | + | \\ |
+ | **Action parameters: | ||
+ | > (Split by fixed row count) Chunk size (rows) | ||
- | //Output:// 10 1000-row files with " | + | \\ |
+ | **Results:** | ||
+ | * 10 1000-row files with " | ||
+ | * A workflow dataset containing the fully qualified filenames of the split files. | ||
\\ \\ | \\ \\ | ||
- | **Example 2: | + | **Example 2: |
- | > (Split by column value) Separator | + | \\ |
- | > (Split by column value) Column | + | **Action parameters: |
+ | > (Split by column value) Separator | ||
+ | > (Split by column value) Column | ||
- | //Output:// One file for each unique Region value, with the Region values appended to the filenames. | + | \\ |
+ | **Results:** | ||
+ | *One file for each unique Region value, with the Region values appended to the filenames. | ||
+ | * A workflow dataset containing the fully qualified filenames of the split files. | ||
\\ | \\ | ||
- | =====See | + | =====See |
* [[transformations: | * [[transformations: | ||
transformations/filesplitter.txt · Last modified: 2022/12/04 16:34 by dmitry