User Tools

Site Tools


overview

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
overview [2019/10/31 12:59]
dmitry [EasyMorph Server]
overview [2021/01/15 00:16] (current)
dmitry
Line 3: Line 3:
  
 ==== Data import ==== ==== Data import ====
-In order to start transforming data in EasyMorph, all data subject to manipulations has to be imported into EasyMorph. As of version 4.EasyMorph can import the following data sources/formats: +In order to start transforming data in EasyMorph, all data subject to manipulations has to be imported into EasyMorph. As of version 4.EasyMorph can import the following data sources/formats: 
-  * Native database drivers (Oracle, MS SQL Server, MySQL, PostgreSQL, SQLite) +  * Native and ODBC database drivers (see the [[connectors#database_connectors|full list of database drivers]])
-  * ODBC drivers (Oracle, MS SQL Server, MySQL, PostgreSQL, SQLite, Amazon Redshift, Vertica, SAP HANA, Apache Hive, Generic ODBC)+
   * REST APIs (JSON/XML)   * REST APIs (JSON/XML)
   * Text files with delimiters (e.g. CSV)   * Text files with delimiters (e.g. CSV)
   * Text files with fixed width columns   * Text files with fixed width columns
   * Excel spreadsheets (.xlsx, .xlsm, .xls)   * Excel spreadsheets (.xlsx, .xlsm, .xls)
-  * XML files+  * XML and JSON files
   * Google Sheets   * Google Sheets
   * SharePoint lists   * SharePoint lists
Line 18: Line 17:
   * EasyMorph datasets (.dset)   * EasyMorph datasets (.dset)
  
-Once tables are loaded into EasyMorph they can further be modified using actions (transformations) that are applied sequentially one after another.+Once tables are loaded into EasyMorphthey can be modified using actions (transformations) that are applied sequentially one after another.
  
-Databases can be queried using the visual Query Editor which is intended for users that don't know SQL.+Databases can be queried using the visual Query Editor intended for users who don't know SQL. 
 + 
 +Files can be loaded by simply dragging them into EasyMorph. It's possible to load and automatically append multiple files at once. 
 + 
 +[[https://easymorph.com/learn/load-file.html|Tutorial: Loading data from files]]\\ 
 +[[https://easymorph.com/learn/load-database.html|Tutorial: Loading data from databases]]
  
-Files can be loaded by simply dragging them into EasyMorph. In many cases, it's possible to load and automatically append multiple files at once. 
  
 ==== Actions ==== ==== Actions ====
-All manipulations with tables are done with the help of various [[[[transformations:start|actions]]. E.g. renaming a column is an action. Filtering, joining, aggregations -- all are actions. As of version 4.EasyMorph offers more than 110 various actions that can be divided into 5 categories:+All manipulations with tables are done with the help of various [[[[transformations:start|actions]]. E.g. renaming a column is an action. Filtering, joining, aggregations -- all are actions. As of version 4.5, EasyMorph offers more than 120 various actions that can be divided into 5 categories:
  
   * [[transformations:start#create|Create]] -- actions that create or generate a table   * [[transformations:start#create|Create]] -- actions that create or generate a table
Line 35: Line 38:
 {{::datamorphing.png?nolink|}} {{::datamorphing.png?nolink|}}
  
-Actions are executed consequently, step-by-step. One action is one step. Every action is applied to the result (output) of previous action. Therefore sequence of actions is important. The same actions in different order would usually produce a different result (although not always).+Actions are executed consequently, step-by-step. One action is one step. An action is applied to the result (output) of its previous action. Therefore the order of actions is important. The same actions ordered differently would usually produce a different result (although not always).
  
-The final state (or result) of a table is the result of all actions of this table applied consequently, one after another.+The final state (result) of a table is the result of all actions of this table applied consequently, one after another.
  
-Some actions (e.g. [[transformations:append|Append Table]]) require another table. These actions have two inputs -- one input is the result of previous action in the same table and the other input is the final state (i.e. the result of last action) of the other table.+Some actions (e.g. [[transformations:append|Append Table]]) require another table. These actions have two inputs -- one input is the result of the previous action in the same tableand the other input is the final state (i.e. the result of the last action) of the other table.
  
 ==== Expressions ==== ==== Expressions ====
-Some actions can use [[syntax:start|expressions]] in order to calculate new columns or filter rows. These expressions can be composed using [[syntax:operators|arithmetical operators]], column names, [[parameters|parameters]] and [[syntax:functions|functions]].+Some actions can use [[syntax:start|expressions]] in order to calculate new columns or filter rows. These expressions can be composed using [[syntax:operators|arithmetical operators]], column names, [[parameters|parameters]] as well as [[syntax:functions|functions]]. 
 + 
 +As of version 4.5 EasyMorph more than 130 functions can be used in expressions: 
 + 
 +  * [[syntax:functions#Number_functions|Number functions]] 
 +  * [[syntax:functions#Text_functions|Text functions]] 
 +  * [[syntax:functions#Logical_functions|Logical functions]] 
 +  * [[syntax:functions#Date_time_functions|Date/time functions]] 
 +  * [[syntax:functions#File_functions|File functions]] 
 +  * [[syntax:functions#Web_functions|Web functions]] 
 +  * [[syntax:functions#Special_functions|Special functions]] 
 + 
 +[[https://easymorph.com/learn/expressions.html|Tutorial: Type system and expressions]]
  
 ==== Derived tables ==== ==== Derived tables ====
-Derived tables is a powerful feature that allows transforming data in a non-linear way. When one table is derived from another (source) table it replicates the final result of the latter. So if the source table changes, the derived table changes automatically. In everything else derived tables is just a regular table and can further be modified using various actions.+Derived tables are a powerful feature that allows transforming data in a non-linear way. When one table is derived from another (source) tableit replicates the final result of the latter. So if the source table changes, the derived table changes automatically. In everything else derived tables is just a regular table and can further be modified using various actions
 + 
 +Since you can derive any number of tables from a table and then keep transforming every derived table in its own way, you can create non-linear action chains. For instance, you can produce two derived tables from a table, transform each of them in its own way and then append or merge them back into one table.
  
-Since you can derive any number of tables from a tableand then keep transforming every derived table in its own way, you can create non-linear action chains. For instance, you can produce two derived tables from tabletransform each of them in its own way and then append or merge them back into one table.+Derived tables can be calculated //conditionally//which allows designing conditionally branched workflows. For instance, if certain data quality check failedsend an email with bad records attached.
  
-Derived tables can be calculated //conditionally// which allows designing conditionally branched workflows. For instance, if a certain data quality check failed, send and email with bad records attached.+[[https://easymorph.com/learn/derived-tables.html|Tutorial: Derived tables]]
  
 ==== Data export ==== ==== Data export ====
-Exporting data is performed using export actions. They can be inserted at any point of workflow. Export actions do not modify tables, they just export them into specified file format or external system. Therefore the output of an export action is always exactly the same as the input.+Exporting data is performed using export actions. They can be inserted at any point of workflow. Export actions do not modify tables. They export them into specified file format or external system. Therefore the output of an export action is always the same as the input.
  
-As of version 4.EasyMorh can export into the following data formats and systems:+As of version 4.EasyMorh can export into the following data formats and systems:
  
-  * Native database drivers (Oracle, MS SQL Server, MySQL, PostgreSQL, SQLite) +  * Native and ODBC database drivers (see the [[connectors#database_connectors|full list of database drivers]])
-  * ODBC drivers (Oracle, MS SQL Server, MySQL, PostgreSQL, SQLite, Amazon Redshift, Vertica, SAP HANA, Apache Hive, Generic ODBC)+
   * REST APIs (JSON/XML)   * REST APIs (JSON/XML)
   * Text files with delimiters (e.g. CSV)   * Text files with delimiters (e.g. CSV)
Line 63: Line 79:
   * XML files (.xml)   * XML files (.xml)
   * Google Sheets   * Google Sheets
-  * Tableau extracts (.tde)+  * Tableau extracts (.tde, .hyper) 
 +  * Power BI datasets
   * Qlik data files (.qvd)   * Qlik data files (.qvd)
   * EasyMorph datasets (.dset)   * EasyMorph datasets (.dset)
  
-==== Automation ==== +[[https://easymorph.com/learn/export-file.html|Tutorial: Exporting data into a file]]\\ 
-EasyMorph workflows can have following automation:+[[https://easymorph.com/learn/export-database.html|Tutorial: Exporting data into a database]] 
 + 
 +==== Workflow automation ==== 
 +EasyMorph workflows can perform external actions, for instance:
   * File and folder operations (copy/rename/delete/unzip/zip/download)   * File and folder operations (copy/rename/delete/unzip/zip/download)
   * Running external applications, scripts and batch files   * Running external applications, scripts and batch files
   * Execution of Windows shell commands   * Execution of Windows shell commands
   * Embedded PowerShell scripting   * Embedded PowerShell scripting
 +
 +A workflow can be composed using advanced patterns:
 +  * FOR..EACH type of loops (iterations)
 +  * DO..WHILE/UNTIL type of loops (iterations)
 +  * IF..THEN..ELSE type of conditional branching
 +  * Subroutines
 +  * Error recovery
 +
 +Workflows can have a complex hierarchical structure when a project can consist of multiple modules, and each module can consist of multiple table groups. Such a structure allows designing and navigating workflows with hundreds of transformation steps (actions).
  
 ==== Integration with external systems ==== ==== Integration with external systems ====
Line 85: Line 114:
 | Google Drive  | Download / upload files  | | Google Drive  | Download / upload files  |
 | Amazon S3  | Download / upload files  | | Amazon S3  | Download / upload files  |
-| Tableau Server / Online  | Publish extracts, trigger extract refreshes |+| Power BI   | Publish push-datasets, trigger flows  | 
 +| Tableau Server / Online  | Publish extracts, trigger extract refreshes, trigger flows |
 | Qlik Sense  | Trigger task reloads | | Qlik Sense  | Trigger task reloads |
 | SSH  | Execute commands  | | SSH  | Execute commands  |
 +
 ==== Auto-documentation ==== ==== Auto-documentation ====
-EasyMorph can automatically generate human-readable documentation in plain English for current project. Read more about [[documentation|Auto-documentation]].+EasyMorph can automatically generate human-readable documentation in plain English for the current project. Read more about [[documentation|Auto-documentation]].
  
 ==== Scheduling ==== ==== Scheduling ====
-[[https://easymorph.com/scheduler.html|EasyMorph Launcher]] is a complementary scheduling/triggering utility that comes with EasyMorph Desktop. Launcher allows creating tasks with EasyMorph projects that triggered manually when neededor scheduled on various schedules (Once, Continuous, Daily, Weekly). Launcher resides in the system tray, starts on Windows start and is tightly integrated with Desktop.+[[https://easymorph.com/scheduler.html|EasyMorph Launcher]] is a free complementary scheduling/triggering utility that comes with EasyMorph Desktop. The Launcher allows creating tasks with EasyMorph projects that are triggered manually when needed or scheduled on various schedules (Once, Continuous, Daily, Weekly). The Launcher resides in the system tray, starts on Windows start and is tightly integrated with Desktop. 
 + 
 +[[https://easymorph.com/learn/scheduling.html|Tutorial: Scheduling]]
  
 ==== Command-line execution ==== ==== Command-line execution ====
-EasyMorph projects can be executed from command line thus allowing integration with third-party applications. Read more about [[command_line|command line parameters]].+EasyMorph projects can be executed from the command linethus allowing integration with third-party applications. Read more about [[command_line|command line parameters]]. 
 + 
 +A more robust integration option is the "ems-cmd" command-line utility that allows triggering EasyMorph Server tasks from external applications running on Windows or Linux. 
 + 
 +[[https://easymorph.com/learn/command-line.html|Tutorial: Running projects from the command line]]
  
 ==== EasyMorph Server ==== ==== EasyMorph Server ====
-[[https://easymorph.com/server.html|EasyMorph Server]] is a Windows service with a web interface that employs the same in-memory engine as EasyMorph Desktop under the hood. Server allows scheduled execution of projects, sharing files and datasets between users, and hosting connector repositories. It also provides a REST API for programmatic integration with external systems.+[[https://easymorph.com/server.html|EasyMorph Server]] is a Windows service with a web interface that employs under the hood the same in-memory engine as EasyMorph Desktop. The Server allows scheduled execution of projects, sharing files and datasets between users, and hosting connector repositories. It also provides a REST API for programmatic integration with external systems
 + 
 +==== EasyMorph Community ==== 
 + 
 +The [[https://community.easymorph.com|EasyMorph Community]] is an online forum for EasyMorph users. It contains hundreds of practical questions answered, frequently with downloadable, ready-to-use examples.
  
 **See also:** **See also:**
   * [[https://www.youtube.com/watch?v=1MtXLkJzKuI|10-minute video overview of EasyMorph 3.6]]   * [[https://www.youtube.com/watch?v=1MtXLkJzKuI|10-minute video overview of EasyMorph 3.6]]
   * [[https://easymorph.com/learn.html|Illustrated tutorial]]   * [[https://easymorph.com/learn.html|Illustrated tutorial]]
-  * [[https://easymorph.com/examples.html|Examples]]+  * [[https://easymorph.com/download.html|Download EasyMorph (free edition)]] 
 +  * [[https://easymorph.com|easymorph.com]]
overview.1572526768.txt.gz · Last modified: 2019/10/31 12:59 by dmitry