transformations:filesplitter
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
transformations:filesplitter [2020/02/19 06:30] – dmitry | transformations:filesplitter [2022/12/04 16:34] (current) – [Action settings] dmitry | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ===== File splitter | + | {{ transformations: |
+ | ======SPLIT DELIMITED FILE ====== | ||
+ | Category: Import / File\\ | ||
- | This action aims to help processing very large text files. It splits a text file into smaller chunks (which are also text files) that can further be processed using iterations. The chunks can be either of fixed length (by row count), or split by unique values of a column (e.g. one chunk per unique Date). | + | \\ |
+ | =====Description===== | ||
+ | This action aims to help in processing very large text files. It splits a text file into smaller chunks (which are also text files) that can further be processed using iterations. The chunks can be either of fixed length (by row count), or split by unique values of a column (e.g. one chunk per unique Date).\\ | ||
- | Since File Splitter | + | \\ |
+ | =====Action settings===== | ||
+ | ^ Setting | ||
+ | |Input file< | ||
+ | |Encoding|ASCII, | ||
+ | |Skip first lines< | ||
+ | |Ignore quoting | When checked, double quotes are treated like a regular character. | | ||
+ | |Output folder< | ||
+ | |Output mode|The method by which the input file will be split.\\Options: | ||
+ | < | ||
+ | \\ | ||
+ | ====Output mode settings==== | ||
+ | ^Setting^Description^ | ||
+ | |Split by fixed row count|The input file will be split into a new file every // | ||
+ | |Split by column value|The input file will be split into a new file for every unique value found in the specified column. | ||
+ | < | ||
+ | |||
+ | \\ | ||
+ | =====Remarks===== | ||
+ | Since this action | ||
+ | |||
+ | In addition to splitting the input file, this action outputs a single-column dataset containing a list of the split files' filenames.\\ | ||
+ | |||
+ | \\ | ||
+ | =====Examples===== | ||
+ | **Example 1:** Splitting a comma-delimited text file with 10,000 rows, and splitting every 1,000 rows.\\ | ||
+ | |||
+ | \\ | ||
+ | **Action parameters: | ||
+ | > (Split by fixed row count) Chunk size (rows) is 1000\\ | ||
+ | |||
+ | \\ | ||
+ | **Results: | ||
+ | * 10 1000-row files with " | ||
+ | * A workflow dataset containing the fully qualified filenames of the split files. | ||
+ | |||
+ | \\ \\ | ||
+ | **Example 2:** Splitting a comma-delimited text file with 10,000 rows, and splitting by each unique Region value.\\ | ||
+ | |||
+ | \\ | ||
+ | **Action parameters: | ||
+ | > (Split by column value) Separator is Comma | ||
+ | > (Split by column value) Column is Region | ||
+ | |||
+ | \\ | ||
+ | **Results: | ||
+ | *One file for each unique Region value, with the Region values appended to the filenames. | ||
+ | * A workflow dataset containing the fully qualified filenames of the split files. | ||
+ | |||
+ | |||
+ | \\ | ||
+ | =====See also===== | ||
+ | |||
+ | * [[transformations: | ||
transformations/filesplitter.1582111853.txt.gz · Last modified: 2020/02/19 06:30 by dmitry