{{ transformations:DeduplicateAction.png}}
====== DEDUPLICATE ROWS ======
Category: Transform / Filters\\

\\ 
=====Description=====
This action removes all duplicate rows in the entire table. Deduplication can be performed based on all columns, or only on specific columns.\\

\\
=====Use cases=====
Use //Deduplicate rows// to clean datasets of records that may have been duplicated in the source dataset, or during previous actions. 

\\ 
=====Action settings=====
^Setting ^Description ^
|Apply to|Select whether to base deduplication on all columns in the dataset, or only selected columns.\\  Options:  //All columns// or //Selected columns// (and select the columns to use from the list).|

\\
=====Remarks=====
When deduplicating based on specific columns:
  * The uniqueness of values in non-selected columns will be ignored.
  * Duplicate rows are removed from the dataset from the bottom, up.


\\ 
=====Examples=====
**Example:** Find and remove duplicate rows.\\

**Source table:** The longest rivers in the world
^ River  ^ Length (km)  ^ Continent  ^
| Nile  |  6650| Africa  |
| Amazon  |  6400| South America  |
| Nile  |  6650| Africa  |
\\ 
**Action parameters:**
> Apply to "All columns"

\\ 
**Result table:**
^River  ^Length (km)  ^Continent  ^
| Nile  |  6650| Africa  |
| Amazon  |  6400| South America  |

\\ 
====Community examples====
  * [[https://community.easymorph.com/t//1269/1|Example: use of Web Request and Iterate Web Request with the Community forum API]] ([[https://community.easymorph.com/uploads/short-url/bGsC1xZtCgMa3rBWiPELwBTZLgX.morph|Project]]; Module: //Main//; Group: //Tab 1//;\\ Table: //Recently active users//; Action position: //9//)
  * [[https://community.easymorph.com/t//1625/4|Create a new table with columns based on row values]] ([[https://community.easymorph.com/uploads/short-url/4oGLUNDHrUfB971H80ciuOAtOTG.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Table 1//; Action position: //3//)
  * [[https://community.easymorph.com/t//1865/4|Dynamical genereting of path based on previous step]] ([[https://community.easymorph.com/uploads/short-url/70xVAnShgK4EiXuSa4HgMJtFAM2.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Calculate folder paths//;\\ Action position: //4//)
  * [[https://community.easymorph.com/t//2156/1|How to process parent-child data structures]] ([[https://community.easymorph.com/uploads/short-url/zmylfn4Qr0ovzu3e5rybl0LVMNG.morph|Project]]; Module: //Module 1//; Group: //Tab 1//; Table: //Longest folder path//; Action position: //2//)

\\ 
=====See also=====
  * [[transformations:keepduplicates|Keep duplicates]]