User Tools

Site Tools


transformations:deduplicate

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
transformations:deduplicate [2016/06/12 11:12] dmitrytransformations:deduplicate [2021/07/19 02:15] (current) – [Examples] craigt
Line 1: Line 1:
-===== Deduplicate =====+{{ transformations:DeduplicateAction.png}} 
 +====== DEDUPLICATE ROWS ====== 
 +Category: Transform / Filters\\
  
-This transformation removes all duplicate rows in entire table. Deduplication can be performed on all columns, or only on specific columns. In the latter case uniqueness of values in non-selected columns will be ignored.+\\  
 +=====Description===== 
 +This action removes all duplicate rows in the entire table. Deduplication can be performed based on all columns, or only on specific columns.\\
  
-EXAMPLE+\\ 
 +=====Use cases===== 
 +Use //Deduplicate rows// to clean datasets of records that may have been duplicated in the source dataset, or during previous actions. 
  
-**Source table:** The longest rivers in the world+\\  
 +=====Action settings===== 
 +^Setting ^Description ^ 
 +|Apply to|Select whether to base deduplication on all columns in the dataset, or only selected columns.\\  Options:  //All columns// or //Selected columns// (and select the columns to use from the list).|
  
-^  River  ^  Length (km)  ^  Continent  ^ +\\ 
-| Nile  |  6650  |  Africa  | +=====Remarks===== 
-| Amazon  |  6400  |  South America  | +When deduplicating based on specific columns: 
-| Nile  |  6650  |  Africa  |+  * The uniqueness of values in non-selected columns will be ignored. 
 +  * Duplicate rows are removed from the dataset from the bottom, up.
  
-**Objective:** Find and remove duplicate rows. 
  
-**Output table:**+\\  
 +=====Examples===== 
 +**Example:** Find and remove duplicate rows.\\ 
 + 
 +**Source table:** The longest rivers in the world 
 +^ River  ^ Length (km)  ^ Continent 
 +| Nile  |  6650| Africa 
 +| Amazon  |  6400| South America 
 +| Nile  |  6650| Africa 
 +\\  
 +**Action parameters:** 
 +> Apply to "All columns"
  
- River  ^  Length (km)  ^  Continent +\\  
-| Nile  |  6650   Africa +**Result table:** 
-| Amazon  |  6400   South America  |+^River  ^Length (km)  ^Continent 
 +| Nile  |  6650| Africa 
 +| Amazon  |  6400| South America  |
  
 +\\ 
 +====Community examples====
 +  * [[https://community.easymorph.com/t//1269/1|Example: use of Web Request and Iterate Web Request with the Community forum API]] ([[https://community.easymorph.com/uploads/short-url/bGsC1xZtCgMa3rBWiPELwBTZLgX.morph|Project]]; Module: //Main//; Group: //Tab 1//;\\ Table: //Recently active users//; Action position: //9//)
 +  * [[https://community.easymorph.com/t//1625/4|Create a new table with columns based on row values]] ([[https://community.easymorph.com/uploads/short-url/4oGLUNDHrUfB971H80ciuOAtOTG.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Table 1//; Action position: //3//)
 +  * [[https://community.easymorph.com/t//1865/4|Dynamical genereting of path based on previous step]] ([[https://community.easymorph.com/uploads/short-url/70xVAnShgK4EiXuSa4HgMJtFAM2.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Calculate folder paths//;\\ Action position: //4//)
 +  * [[https://community.easymorph.com/t//2156/1|How to process parent-child data structures]] ([[https://community.easymorph.com/uploads/short-url/zmylfn4Qr0ovzu3e5rybl0LVMNG.morph|Project]]; Module: //Module 1//; Group: //Tab 1//; Table: //Longest folder path//; Action position: //2//)
  
 +\\ 
 +=====See also=====
 +  * [[transformations:keepduplicates|Keep duplicates]]
transformations/deduplicate.1465744361.txt.gz · Last modified: 2016/06/12 11:12 by dmitry

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki