transformations:filesplitter
no way to compare when less than two revisions
Differences
This shows you the differences between two versions of the page.
Previous revisionNext revision | |||
— | transformations:filesplitter [2021/03/10 13:04] – craigt | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | {{ transformations: | ||
+ | ======SPLIT DELIMITED TEXT FILE ====== | ||
+ | (Category: //Import | File//) | ||
+ | |||
+ | =====Description===== | ||
+ | This action aims to help in processing very large text files. It splits a text file into smaller chunks (which are also text files) that can further be processed using iterations. The chunks can be either of fixed length (by row count), or split by unique values of a column (e.g. one chunk per unique Date). | ||
+ | |||
+ | Since File Splitter doesn' | ||
+ | |||
+ | In addition to splitting the input file, this action outputs a single-column dataset containing a list of the split files' filenames. | ||
+ | \\ | ||
+ | \\ | ||
+ | |||
+ | =====Action settings===== | ||
+ | ^ Setting | ||
+ | |Input file|Fully-qualified file name of the text file to be split (includes relative or absolute path).| | ||
+ | |Encoding|ASCII, | ||
+ | |Skip first lines|This option specifies how many lines to skip from the beginning of a file. It can be helpful in cases where first rows contain\\ some metadata (file header) and actual tabular data starts only after that metadata. This setting can be specified\\ using a parameter as well.| | ||
+ | |Output folder|The folder location the split files will be saved to.| | ||
+ | |Output mode|The method by which the input file will be split.\\Options: | ||
+ | |||
+ | ^**//Output mode// settings**^^ | ||
+ | ^Setting^Description^ | ||
+ | |Split by fixed row count|The input file will be split into a new file every // | ||
+ | |Split by column value|The input file will be split into a new file for every unique value found in the specified column. | ||
+ | |||
+ | \\ | ||
+ | =====Examples===== | ||
+ | **Example 1:** You're splitting a comma-delimited text file with 10,000 rows, __splitting every 1,000 rows__:\\ | ||
+ | |||
+ | > (Split by fixed row count) Chunk size (rows) = 1000\\ | ||
+ | |||
+ | //Output:// 10 1000-row files with " | ||
+ | |||
+ | \\ \\ | ||
+ | **Example 2:** You're splitting a comma-delimited text file with 10,000 rows, __splitting by each unique Region value__:\\ | ||
+ | |||
+ | > (Split by column value) Separator = " | ||
+ | > (Split by column value) Column = " | ||
+ | |||
+ | // | ||
+ | |||
+ | |||
+ | \\ | ||
+ | =====See Also===== | ||
+ | |||
+ | * [[transformations: | ||
transformations/filesplitter.txt · Last modified: 2022/12/04 16:34 by dmitry