This is an old revision of the document!
Table of Contents
FILTER BY TYPE
Category: Transform / Filters
Description
This action filters values depending on their data type. Filtering can be applied to several columns at once, as well as multiple data types.
Use cases
Filtering by type helps detect data quality issues by ensuring that columns in the exported dataset are strongly typed (i.e. all values in a column are of the same type). This might be required before exporting into a target system with strongly typed fields, such as a relational database or a Tableau data extract.
The Filter by type action is frequently followed by the Halt on condition action that stops project execution if filtering returned a non-empty table with values of undesired types.
Action settings
Setting | Description |
---|---|
Filter mode | Select whether to keep or remove values of the types specified. Options: Keep only values of selected type(s) or Remove values of the selected types. |
Data types | Select the data types to be filtered by. Options: Text, Number, Boolean, Error, and Empty (null). |
Columns to check | Select whether to check the entire table or specific columns. Options: Entire table or Particular columns (and select the columns check). |
Remarks
In cases where no values match the search string:
- If Keep only values… was selected, the result will be an empty dataset.
- If Remove values… was selected, the result will be the full dataset (no rows removed).
Examples
Objective: Filter out all non-numberic values in the "Length (km)" column.
Source table: The longest rivers in the world
River | Length (km) | Continent |
---|---|---|
Nile | 6650 | Africa |
Amazon | 6400 | South America |
Yangtze | N/A | Asia |
Mississippi | 6275 | North America |
Yenisei | 5539 | Asia |
Yellow River | 5464 | Asia |
Ob | 5410 | Asia |
Paraná | South America |
Action parameters:
Filter mode is "Keep only values of selected type(s)"
Data types: "Number" selected
Columns to check; Particiular columns: "Length (km)"
Result:
River | Length (km) | Continent |
---|---|---|
Nile | 6650 | Africa |
Amazon | 6400 | South America |
Mississippi | 6275 | North America |
Yenisei | 5539 | Asia |
Yellow River | 5464 | Asia |
Ob | 5410 | Asia |