Differences

This shows you the differences between two versions of the page.

--- transformations:deduplicate [2014/10/25 17:14] – created dmitry
+++ transformations:deduplicate [2021/07/18 01:22] – craigt
@@ Line 1: / Line 1: @@
-===== Deduplicate =====
+{{ transformations:DeduplicateAction.png}}
+====== DEDUPLICATE ROWS ======
+Category: Transform / Filters\\
-This transformation removes all duplicating rows in entire table. It doesn't have parameters.
+\\
+=====Description=====
+This action removes all duplicate rows in the entire table. Deduplication can be performed based on all columns, or only on specific columns.\\
+\\
+=====Use cases=====
+Use //Deduplicate rows// to clean datasets of records that may have been duplicated in the source dataset, or during previous actions.
+\\
+=====Action settings=====
+^Setting ^Description ^
+|Apply to|Select whether to base deduplication on all columns in the dataset, or only selected columns.\\  Options:  //All columns// or //Selected columns// (and select the columns to use from the list).|
+\\
+=====Remarks=====
+When deduplicating based on specific columns:
+  * The uniqueness of values in non-selected columns will be ignored.
+  * Duplicate rows are removed from the dataset from the bottom, up.
+\\
+=====Examples=====
+**Objective:** Find and remove duplicate rows.\\
+**Source table:** The longest rivers in the world
+^ River  ^ Length (km)  ^ Continent  ^
+| Nile  |  6650| Africa  |
+| Amazon  |  6400| South America  |
+| Nile  |  6650| Africa  |
+\\
+**Action parameters:**
+> Apply to "All columns"
+\\
+**Result table:**
+^River  ^Length (km)  ^Continent  ^
+| Nile  |  6650| Africa  |
+| Amazon  |  6400| South America  |
+\\
+====Community examples====
+  * [[https://community.easymorph.com/t//1269/1|Example: use of Web Request and Iterate Web Request with the Community forum API]] ([[https://community.easymorph.com/uploads/short-url/bGsC1xZtCgMa3rBWiPELwBTZLgX.morph|Project]]; Module: //Main//; Group: //Tab 1//;\\ Table: //Recently active users//; Action position: //9//)
+  * [[https://community.easymorph.com/t//1625/4|Create a new table with columns based on row values]] ([[https://community.easymorph.com/uploads/short-url/4oGLUNDHrUfB971H80ciuOAtOTG.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Table 1//; Action position: //3//)
+  * [[https://community.easymorph.com/t//1865/4|Dynamical genereting of path based on previous step]] ([[https://community.easymorph.com/uploads/short-url/70xVAnShgK4EiXuSa4HgMJtFAM2.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Calculate folder paths//;\\ Action position: //4//)
+  * [[https://community.easymorph.com/t//2156/1|How to process parent-child data structures]] ([[https://community.easymorph.com/uploads/short-url/zmylfn4Qr0ovzu3e5rybl0LVMNG.morph|Project]]; Module: //Module 1//; Group: //Tab 1//; Table: //Longest folder path//; Action position: //2//)
+\\
+=====See also=====
+  * [[transformations:keepduplicates|Keep duplicates]]