This shows you the differences between two versions of the page.
transformations:selectdbrowsbykeys [2017/05/03 03:10] (current)
|Line 1:||Line 1:|
|+||===== Select matching database rows =====|
|+||Selects rows from a database table where a combination of key fields matches a combination of key fields in EasyMorph table. |
|+||Under the hood the transformation creates a temporary table in the target database, uploads key field combinations into it and performs SELECT..WHERE ..EXISTS statement. After the statement is executed the temporary table is deleted.|
|+||This transformation can be used for //pre-filtering//. Pre-filtering means that instead of loading entire dataset and then filtering it, only a few fields are loaded (using [[transformations:importsql|Import from database]]) that are necessary for filtering, as well as primary/foreign keys (i.e. IDs). When imported these fields can be filtered by means of various EasyMorph transformations. The resulting subset of IDs is used to fetch full records from the source dataset using "Select matching database rows" transformation. Pre-filtering can be helpful when source table is large and doesn't fit RAM entirely.|
|+||Another use case is //cross-source filtering// which means importing database records which IDs are obtained from another data source (e.g. text file or spreadsheet).|