User Tools

Site Tools


transformations:sharedmemory

SHARED MEMORY

Category: Workflow / Internal

Description

This action can perform the following operations with the shared memory (details below):

  • Remember a value by a key
  • Forget a value and its associated key
  • Recall a list of keys
  • Recall a value by key
  • List keys and values


Key-value storage

From a technical perspective, the shared memory is key-value storage located in the current repository. It's the same repository that stores connectors.

The idea of shared memory is simple: it stores keys and for each key, it stores a value. Each key has one value associated with it. For instance:

Key Value
Path D:\Input files
CustomerDB\Last reload date 2021-09-03

To obtain a value, one should know the key. Keys and values can be added and removed on demand with the help of the Shared memory action. The shared memory is stored in a centralized repository. Therefore any EasyMorph workflow can read (recall) and write (remember) keys and values in the shared memory at any moment as long as it has access to the repository.


Use cases

The shared memory can be convenient in many cases:

  • Store temporary values: Store the last loaded date when doing incremental loads.
  • Pass values between workflows and users: Similar to shared connectors, the shared memory can be accessed from different Desktops. Once a value is remembered, it can be accessed by all Desktops and Server spaces that use the same repository.
  • Environment configuration: The shared memory can be used to store configuration settings that are different in PROD and TEST environments.
  • Workflow variable: While project parameters are constant and don’t change during a workflow run, the shared memory values can be set, removed, and changed in a single workflow.
  • Lightweight data store: Sometimes, using a relational database to store and share just a few values seems like overkill. The shared memory can be more appropriate for such cases.


Action settings

SettingDescription
CommandSelect the operation to perform on the shared memory repository. Options: Remember, Recall, Recall a list of keys, Forget, and List.


Remember settings

Description: Store a value in an associated key.

SettingDescription
Key*Enter the key name associated with the value. Nested keys are permitted using backlashes to define subkeys.
e.g. mykeys\key1, mykeys\key2\subkey2a.
PickThis button opens a window used to navigate through the current key structure.
Value*Enter the value associated with the key provided above.
If key already existsSelect how EasyMorph handles situations when the provided key already exists. Options: Update value, Fail
(display an error), or Do nothing (no action is taken).

* Setting can be specified using a parameter.


Recall settings

Description: Retrieve the value stored in a specified key.

SettingDescription
Key*Enter the key name or key\subkey "path" to the value to be retrieved.
PickThis button opens a window used to navigate through the current key structure.
Column nameChoose how the column in the action's output dataset will be named. Options: Last subkey (the name of the final key),
Full key name (the entire key/subkey path), or Custom* (enter a column name).
Append columnWhen unchecked, the output dataset will be a single cell containing the recalled value in a column named as selected
above. When checked, a new column will be appended to the incoming dataset with the recalled value appearing in each row.

* Setting can be specified using a parameter.


Recall a list of keys settings

Description: Retrieve the values of the shared memory keys listed within a column.

SettingDescription
Column nameSelect the column that contains a list of fully qualified shared memory keys.
If a key doesn't existSelect how EasyMorph will handle cases where the listed key does not exist in shared memory.
Options: Fail (the workflow stops with an error message) or Return error (the workflow continues, but an error message is returned in the "Value" column).


Forget settings

Description: Clear the value and associated key from the shared memory.

SettingDescription
Key*Enter the key name or key/subkey "path" to be removed. This must be a lowest-level key or an error is displayed.
PickThis button opens a window used to navigate through the current key structure.
If the key doesn't existSelect how EasyMorph handles situations when the provided key does not exist. Options: Fail (display an error), or
Ignore (no action is taken).

* Setting can be specified using a parameter.


List settings

Description: Generate a list of key-value pairs along a certain key/subkey path.

SettingDescription
Keys start with*Enter a whole or portion of a key name or key/subkey "path" from which a list of key-value pairs will be generated.
Entering the lowest-level key name will provide a single record with that key-value pair. Entering a higher-level key name
will produce a multi-record dataset of all subkeys and values under the entered key.
PickThis button opens a window used to navigate through the current key structure.

* Setting can be specified using a parameter.

For example:
Keys start with "mykeys\key1" produces a dataset with a single key-value pair:

KeyValue
mykeys\key1 12

Keys start with "mykeys" produces a dataset listing all subkeys with their associated values:

KeyValue
mykeys\key1 12
mykeys\key2 79
mykeys\key3 247


Remarks

Using shared memory

  • Use a backslash (\) to create nested keys.
  • Values are remembered in the shared memory and recalled in native EasyMorph data types. Errors cannot be remembered (stored) by design.
  • You can store large text objects, such as JSON, in the shared memory.
  • All stored data is encrypted and cannot be accessed from outside of EasyMorph.

Accessing shared memory

  • The same access restrictions as for data connectors apply.
  • The shared memory supports simultaneous operations from different computers as long as they can access the same repository.
  • Depending on repository location, network performance, and size of the stored keys and values, the shared memory can handle up to 100 operations per second.


Examples

Below is a sample process for manipulating the shared memory.

STORE VALUES:

Command: REMEMBER
Key: mykeys\key1
Value: 100
Command: REMEMBER
Key: mykeys\key2
Value: 200



LIST KEYS-VALUES:

Command: LIST
Keys start with: mykeys
Show last changed time is checked.
KeyValueLast changedChanged by
mykeys\key110017-Oct-2021UserName
mykeys\key220017-Oct-2021UserName



RETRIEVE A VALUE:

Command: RECALL
Key: mykeys\key1
Column name: Last subkey
key1
100



CLEAR A KEY-VALUE PAIR:

Command: FORGET
Key: mykeys\key1


KeyValueLast changedChanged by
mykeys\key110017-Oct-2021UserName
mykeys\key220017-Oct-2021UserName



RECALL A LIST OF KEYS:

Current dataset:

Key_List
mykeys\key1
mykeys\key2
mykeys\key3
otherkeys\keyAA
otherkeys\keyBB


Command: RECALL A LIST OF KEYS
Column name: Key_List


Key_List Value
mykeys\key1 100
mykeys\key2 250
mykeys\key3 700
otherkeys\keyAA Mark
otherkeys\keyBB Mary


transformations/sharedmemory.txt · Last modified: 2023/10/13 23:08 by craigt

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki