Both sides previous revisionPrevious revisionNext revision | Previous revision |
transformations:deduplicate [2016/06/12 11:12] – dmitry | transformations:deduplicate [2021/07/19 02:15] (current) – [Examples] craigt |
---|
===== Deduplicate ===== | {{ transformations:DeduplicateAction.png}} |
| ====== DEDUPLICATE ROWS ====== |
| Category: Transform / Filters\\ |
| |
This transformation removes all duplicate rows in entire table. Deduplication can be performed on all columns, or only on specific columns. In the latter case uniqueness of values in non-selected columns will be ignored. | \\ |
| =====Description===== |
| This action removes all duplicate rows in the entire table. Deduplication can be performed based on all columns, or only on specific columns.\\ |
| |
EXAMPLE | \\ |
| =====Use cases===== |
| Use //Deduplicate rows// to clean datasets of records that may have been duplicated in the source dataset, or during previous actions. |
| |
**Source table:** The longest rivers in the world | \\ |
| =====Action settings===== |
| ^Setting ^Description ^ |
| |Apply to|Select whether to base deduplication on all columns in the dataset, or only selected columns.\\ Options: //All columns// or //Selected columns// (and select the columns to use from the list).| |
| |
^ River ^ Length (km) ^ Continent ^ | \\ |
| Nile | 6650 | Africa | | =====Remarks===== |
| Amazon | 6400 | South America | | When deduplicating based on specific columns: |
| Nile | 6650 | Africa | | * The uniqueness of values in non-selected columns will be ignored. |
| * Duplicate rows are removed from the dataset from the bottom, up. |
| |
**Objective:** Find and remove duplicate rows. | |
| |
**Output table:** | \\ |
| =====Examples===== |
| **Example:** Find and remove duplicate rows.\\ |
| |
| **Source table:** The longest rivers in the world |
| ^ River ^ Length (km) ^ Continent ^ |
| | Nile | 6650| Africa | |
| | Amazon | 6400| South America | |
| | Nile | 6650| Africa | |
| \\ |
| **Action parameters:** |
| > Apply to "All columns" |
| |
^ River ^ Length (km) ^ Continent ^ | \\ |
| Nile | 6650 | Africa | | **Result table:** |
| Amazon | 6400 | South America | | ^River ^Length (km) ^Continent ^ |
| | Nile | 6650| Africa | |
| | Amazon | 6400| South America | |
| |
| \\ |
| ====Community examples==== |
| * [[https://community.easymorph.com/t//1269/1|Example: use of Web Request and Iterate Web Request with the Community forum API]] ([[https://community.easymorph.com/uploads/short-url/bGsC1xZtCgMa3rBWiPELwBTZLgX.morph|Project]]; Module: //Main//; Group: //Tab 1//;\\ Table: //Recently active users//; Action position: //9//) |
| * [[https://community.easymorph.com/t//1625/4|Create a new table with columns based on row values]] ([[https://community.easymorph.com/uploads/short-url/4oGLUNDHrUfB971H80ciuOAtOTG.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Table 1//; Action position: //3//) |
| * [[https://community.easymorph.com/t//1865/4|Dynamical genereting of path based on previous step]] ([[https://community.easymorph.com/uploads/short-url/70xVAnShgK4EiXuSa4HgMJtFAM2.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Calculate folder paths//;\\ Action position: //4//) |
| * [[https://community.easymorph.com/t//2156/1|How to process parent-child data structures]] ([[https://community.easymorph.com/uploads/short-url/zmylfn4Qr0ovzu3e5rybl0LVMNG.morph|Project]]; Module: //Module 1//; Group: //Tab 1//; Table: //Longest folder path//; Action position: //2//) |
| |
| \\ |
| =====See also===== |
| * [[transformations:keepduplicates|Keep duplicates]] |