User Tools

Site Tools


transformations:filterbytype
no way to compare when less than two revisions

Differences

This shows you the differences between two versions of the page.


Previous revision
Next revision
transformations:filterbytype [2021/04/26 00:21] craigt
Line 1: Line 1:
 +{{ transformations:FilterByTypeAction.png}}
 +====== FILTER BY TYPE ======
 +Category: Transform / Filters\\
  
 +\\ 
 +=====Description=====
 +This action filters values depending on their [[syntax:types|data type]]. Filtering can be applied to several columns at once, as well as multiple data types.\\
 +
 +\\
 +=====Use cases=====
 +Filtering by type helps detect data quality issues by ensuring that columns in the exported dataset are strongly typed (i.e. all values in a column are of the same type). This might be required before exporting into a target system with strongly typed fields, such as a relational database or a Tableau data extract.
 +
 +The //Filter by type// action is frequently followed by the [[transformations:halt|Halt on condition]] action that stops project execution if filtering returned a non-empty table with values of undesired types.\\
 +
 +\\ 
 +=====Action settings=====
 +^Setting ^Description ^
 +|Filter mode|Select whether to keep or remove values of the types specified.  Options:  //Keep only values of selected type(s)// or\\ //Remove values of the selected types//.|
 +|Data types|Select the data types to be filtered by.  Options:  //Text//, //Number//, //Boolean//, //Error//, and //Empty (null)//. |
 +|Columns to check|Select whether to check the entire table or specific columns.  Options:  //Entire table// or //Particular columns// (and\\ select the columns check).|
 +
 +\\
 +=====Remarks=====
 +In cases where no values match the search string:
 +  * If //Keep only values…// was selected, the result will be an empty dataset.
 +  * If //Remove values…// was selected, the result will be the full dataset (no rows removed).
 +
 +\\ 
 +=====Examples=====
 +**Objective:**  Filter out all non-numberic values in the "Length (km)" column.\\
 +
 +**Source table:** The longest rivers in the world
 +^River  ^Length (km)  ^Continent  ^
 +| Nile  |  6650| Africa  |
 +| Amazon  |  6400| South America  |
 +| Yangtze  |N/A  | Asia  |
 +| Mississippi  |  6275| North America  |
 +| Yenisei  |  5539| Asia  |
 +| Yellow River  |  5464| Asia  |
 +| Ob  |  5410| Asia  |
 +| Paraná  |  | South America  |
 +\\
 +**Action parameters:**
 +> Filter mode is "Keep only values of selected type(s)"
 +> Data types:  "Number" selected
 +> Columns to check; Particiular columns:  "Length (km)"
 +\\ 
 +**Result:**
 +^River  ^Length (km)  ^Continent  ^
 +| Nile  |  6650| Africa  |
 +| Amazon  |  6400| South America  |
 +| Mississippi  |  6275| North America  |
 +| Yenisei  |  5539| Asia  |
 +| Yellow River  |  5464| Asia  |
 +| Ob  |  5410| Asia  |
 +
 +\\ 
 +=====See also=====
 +  * [[transformations:selectionfilter|Filter]]
 +  * [[transformations:filter|Filter by condition]]
 +  * [[transformations:searchfilter|Filter by search]]
transformations/filterbytype.txt · Last modified: 2021/07/19 02:14 by craigt

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki