transformations:deduplicate
This is an old revision of the document!
Deduplicate
This transformation removes all duplicate rows in entire table. Deduplication can be performed on all columns, or only on specific columns. In the latter case uniqueness of values in non-selected columns will be ignored.
EXAMPLE
Source table: The longest rivers in the world
| River | Length (km) | Continent |
|---|---|---|
| Nile | 6650 | Africa |
| Amazon | 6400 | South America |
| Nile | 6650 | Africa |
Objective: Find and remove duplicate rows.
Output table:
| River | Length (km) | Continent |
|---|---|---|
| Nile | 6650 | Africa |
| Amazon | 6400 | South America |
transformations/deduplicate.1465744361.txt.gz · Last modified: 2016/06/12 11:12 by dmitry