Differences

This shows you the differences between two versions of the page.

--- connectors [2018/05/09 20:42] – dmitry
+++ connectors [2024/04/19 00:37] (current) – [Database connectors] dmitry
@@ Line 1: / Line 1: @@
-===== Database connectors =====
+====== Connectors ======
+A connector in EasyMorph holds information about connection settings required to connect to an external system, such as a database, cloud service, email, enterprise application, or REST API.
-A database connector in EasyMorph holds information about connection settings required to connect to a particular database. Currently EasyMorph supports these database types:
+Connectors can be embedded into a project or can be stored in a shared repository. Connectors can be copied/pasted from any location to any location, e.g. from one project to another project, from project to a repository, from one repository to another repository, etc.
-  * MS SQL Server
-  * MySQL
-  * Oracle Database
-  * PostgreSQL
-  * SQLite
-  * Amazon Redshift
-  * DB2
-  * Vertica
-  * SAP HANA
-  * Apache Hive
-Connectors can be embedded into a project, or can be stored in a shared repository which is a file. Connectors can be copied from any location to any location, e.g. from one project to another project, from project to repository, from one repository to another repository, etc.
 === Embedded connectors ===
-If connectors are embedded into a project then the project can work on different computers using the same connection settings (as long as there is network connectivity for accessing the databases).
+Embedded connectors are stored right in the project. It allows using projects on different computers using the same connection settings without relying on a shared repository. However, if connection settings change (e.g. credentials changed), all embedded connectors should be changed manually.
 === Shared connector repository ===
-Shared connector repositories can be used in cases when many users need to share the same connection settings. Having shared repositories is also convenient for migrating projects from dev to test/prod environments.
+Shared connector repositories can be used in cases when many users need to use the same connection settings in a controllable, centralized fashion.
+An empty shared repository is created automatically by EasyMorph upon the first run. It can also be created manually from Connector Manager invoked by menu Project -> Manage connectors, or by pressing F7. It is possible to have several repositories, but only one repository can be active in EasyMorph. Switching to another repository can be done through the Connector Manager. Shared connectors are referenced in projects by name. Therefore, if you switch between two repositories (e.g. one for development and one for production), make sure that the connector name is the same in both repositories.
+To reduce the chance of human error, a shared repository can be protected with a password requested for adding new or modifying existing connectors. Using a connector for importing data doesn't require a password. Passwords can be set up and changed in Connector Manager.
+Having shared repositories is also convenient for migrating projects from dev to test/prod environments as shared connectors are referenced by name. I.e. a project can query a database that is specified by connector "Customers". In a test environment, a connector named "Customers" can point to a test database. At the same time, in a production environment, a connector also named "Customers" points to a production database. When moved from one environment to another, the project will use the necessary connector automatically.
+=== Server-hosted connector repository ===
+A shared repository can be served to Desktop users from EasyMorph Server. This ensures stronger security:
+  * The repository files are not exposed to users
+  * Connectors can be protected from copying and thus embedding
+  * If a repository is provided from a Server space with the Active Directory authentication mode, then only the users that are whitelisted in the space settings can access and use the repository
+  * EasyMorph Server allows locking and encrypting repositories
+  * User tasks (projects) on EasyMorph Server can be forced to run under Windows accounts that don't have access to the system folder that contains repositories.
+To use a Server-hosted repository, the Server Link must be configured in the Desktop. Please read the "EasyMorph Server Administrator Guide" which comes with the Server installer, for more details.
+==== Database connectors ====
+Currently, EasyMorph supports these database types and SQL dialects:
+^ Database type   ^ Driver type ^ Notes ^
+|SQL Server / Azure SQL Database	   |ODBC / Native  |Bulk export supported (only for the native driver) |
+|Oracle	           |ODBC / Native  |Bulk export supported. |
+|MySQL	           |ODBC / Native  |Bulk export supported. |
+|PostgreSQL	   |ODBC / Native  |Bulk export supported (only for the native driver). Tested with versions 10,12, and 15 |
+|MariaDB           |ODBC / Native  | |
+|SQLite	           |ODBC / Native  | |
+|Amazon Redshift   |ODBC / Native  | |
+|Snowflake         |ODBC / Native  |Bulk export supported (only for the ODBC driver) |
+|Teradata          |ODBC / Native  | |
+|Google BigQuery   |ODBC only  |Bulk export is supported. |
+|MS Access         |ODBC only  | |
+|Amazon Athena     |ODBC only  | |
+|DB2               |ODBC only  | |
+|DB2 for iSeries (AS/400) |ODBC only  | |
+|Exasol            |ODBC only  | |
+|SAP HANA          |ODBC only  | |
+|Sybase IQ         |ODBC only  | |
+|Vertica           |ODBC only  | |
+|Apache Hive       |ODBC only  |Import via visual queries only, custom SQL statements.|
+|Denodo            |ODBC only  |Import via visual queries only, custom SQL statements.|
+|Databricks        |ODBC only  | |
+|Generic SQL:2003  |ODBC only  |Custom SQL only |
+|Other SQL         |ODBC only  |A generic connector for any ODBC driver. Custom SQL only. |
+Database ODBC connections are configured in EasyMorph by creating an ODBC connector, selecting a 64-bit ODBC DSN configured in Windows, and specifying the required SQL dialect (if it wasn't auto-detected) in the connector settings.
+==== Email connectors ====
+  * [[connectors:email|Generic email server (IMAP)]]
+  * [[connectors:exchange|Microsoft Exchange]]
+  * [[connectors:awsses|Amazon SES]]
+  * [[connectors:outlook|Outlook.com (Hotmail, Live)]]
+  * [[connectors:gmail|Gmail]]
+==== System connectors ====
+  * [[connectors:ssh|SSH]] (used also for SFTP/SCP transfers)
+  * [[connectors:weblocation|Web location]] (used also for REST API)
+==== Enterprise applications ====
+  * [[connectors:sharepoint|Microsoft SharePoint]]
+  * [[connectors:powerbi|Microsoft PowerBI]]
+  * [[connectors:salesforce|Salesforce]]
+  * [[connectors:tableau|Tableau Server]]
+  * [[connectors:tableauonline|Tableau Online]]
+  * [[connectors:qlik|Qlik Sense]]
+==== Cloud services/applications ====
+  * [[connectors:airtable|Airtable]]
+  * [[connectors:awss3|Amazon S3]]
+  * [[connectors:azureblobstorage|Azure Blob Storage]]
+  * [[connectors:chatgpt|ChatGPT]]
+  * [[connectors:dropbox|Dropbox]]
+  * [[connectors:googleanalytics|Google Analytics]]
+  * [[connectors:googlesearchconsole|Google Search Console]]
+  * [[connectors:googledrive|Google Drive]]
+  * [[connectors:jira|Jira]]
+  * [[connectors:ntfy|Ntfy]]
+  * [[connectors:onedrive|OneDrive]]
+  * [[connectors:pipedrive|Pipedrive]]
+  * [[connectors:pushover|Pushover]]
+  * [[connectors:slackbot|Slack Bot]]
+  * [[connectors:teamsbot|Teams Bot]]
-An empty shared repository is created automatically by EasyMorph upon first run. It can also be created manually from Connector Manager invoked by menu Project -> Manage connectors, or by pressing F7. It is possible to have several repositories, but only one repository can be active in EasyMorph. Switching to another repository can be done through Connector Manager. Shared connectors are referenced in projects by name. Therefore, if you switch between two repositories (e.g. one for development and one for production), make sure that connector name is the same in both repositories.
-To reduce a chance of human error, a shared repository can be protected with a password requested for adding new or modifying existing connectors. Using a connector for importing data doesn't require a password. Passwords can be set up and changed in Connector Manager.
+==== Security considerations ====
+While EasyMorph encrypts connection passwords stored in projects and repositories, this can't be considered a strong security measure. Therefore for additional security, consider the following:
-=== Security considerations ===
+  * Use Server-hosted connector repositories when possible; otherwise limit access to shared repositories using standard Windows file system permissions
-While EasyMorph encrypts connection passwords stored in projects and repositories, this can't be considered a strong security measure. Therefore for addition security consider the following:
+  * For Server repositories, consider using the option for locking and encrypting repository; the option is described in the "EasyMorph Server Administrator Guide" that comes with the Server installer
+  * In connectors, use read-only accounts and don't expose unnecessary objects (e.g. schemas, tables, etc.)
+  * Don't use embedded connectors for accessing sensitive information because they can be easily copied unless dynamic authentication is used (such as "Windows integrated")
+  * Keep in mind that users may perform various operations (e.g. write and execute custom SQL queries/statements) using connectors without knowing the credentials stored in the connectors
+  * SQL-related actions (such as "Import from database") and commands (such as "Custom SQL statement") insert [[parameters]] as plain text into the generated SQL statement which can make them vulnerable to SQL injection. Don't use parameters in such actions when parameter values can't be trusted or sanitized.
-  * Make sure that the database account used in connector is read-only and doesn't expose unnecessary objects (e.g. schemas, tables, etc.)
-  * Limit access to shared repositories using standard Windows file system permissions
-  * When using embedded connectors make sure there is full control over distribution of a project with embedded connectors
-  * Don't use embedded connectors for very sensitive information
-  * Keep in mind that users can write custom SQL queries using connectors
-=== Using connectors ===
-To import data from a database using a connector use the [[transformations:importsql|Import from database]] action.