Table of Contents
DEDUPLICATE ROWS
Category: Transform / Filters
Description
This action removes all duplicate rows in the entire table. Deduplication can be performed based on all columns, or only on specific columns.
Use cases
Use this action to remove records that may have been duplicated in the source dataset or during previous actions, and keep unique records only.
Action settings
| Setting | Description |
|---|---|
| Apply to | Select whether to base deduplication on all columns in the dataset, or only selected columns. Options: All columns or Selected columns (and select the columns to use from the list). |
Remarks
When deduplicating based on specific columns:
- The uniqueness of values in non-selected columns will be ignored.
- Duplicate rows are removed from the dataset from the bottom, up.
Examples
Example #1
Find and remove duplicate rows based on the values in all columns.
Before (source table)
| River | Length (km) | Continent |
|---|---|---|
| Nile | 6650 | Africa |
| Amazon | 6400 | South America |
| Nile | 6650 | Africa |
After (result table)
| River | Length (km) | Continent |
|---|---|---|
| Nile | 6650 | Africa |
| Amazon | 6400 | South America |
Action parameters
Apply to: All columns
Example #2
Remove duplicate rows based on First and Last Names.
Before (source table)
| FirstName | LastName | MiddleInit |
|---|---|---|
| Leah | Boswell | A |
| José | Silva | D |
| Gino | Basso | E |
| Leah | Boswell | F |
| Gino | Basso | R |
| Santa | Alegio | G |
After (result table)
| FirstName | LastName | MiddleInit |
|---|---|---|
| Leah | Boswell | A |
| José | Silva | D |
| Gino | Basso | E |
| Santa | Alegio | G |
Action parameters
Apply to: Selected columns (FirstName, LastName)
Community examples
- Example: use of Web Request and Iterate Web Request with the Community forum API (Project; Module: Main; Group: Tab 1;
Table: Recently active users; Action position: 9) - Create a new table with columns based on row values (Project; Module: Main; Group: Tab 1; Table: Table 1; Action position: 3)
- Dynamical genereting of path based on previous step (Project; Module: Main; Group: Tab 1; Table: Calculate folder paths;
Action position: 4) - How to process parent-child data structures (Project; Module: Module 1; Group: Tab 1; Table: Longest folder path; Action position: 2)