Correlation 

Some data objects are designed to allow new object instances (inserts) to be created in the client application (for example sales orders created by sales representatives in the field). These new instances are added to the records in the back end (sent through Data Orchestration Engine (DOE)).

The client application does not always fill the back-end key fields or only uses temporary key values as the back end is the final validation instance for the back-end keys. In this case, the back end fills or changes the back-end key (for example sales order numbers) according to its own logic.

The back-end key is the unique key for identifying a node instance in the client and the back end. The SyncKey is a DOE-internal key for identifying node instances in the consolidated data store (CDS) and is not sent to the back end).

When the DOE receives new object instances from the clients, it does not save them in the CDS but sends the individual node instances as separate messages to the back-end system and back to the client. If the back-end system changes a back-end key of such an instance, the client cannot know which node instances coming from the back end belong to which data object instance it has created. To enable the client to correlate the node instances coming back from the back end with the instances originally created in the client application, the Data Orchestration Workbench provides the functionality of defining correlation for such data objects.

Correlation can be defined as field-based and sort-based:

Correlation Types

Type	Meaning
Field-based	For field-based correlation, you define one or more correlation fields for the client to uniquely identify a node instance even if the back-end key is changed in the back end. You can choose existing fields in the data object node as correlation fields; you do not have to define additional fields for correlation. The back end keeps a record of the correlation fields and the corresponding back-end keys. These records are not required permanently and must be cleaned up using a correlation cleanup function implemented in the back end. You specify the correlation cleanup function to be used when you define the back-end adapter.
Sort-based	When using sort-based correlation, you can but do not have to select correlation fields. If you do not select any correlation fields, DOE sends the node instances to the back end in a random order and expects them to be sent back from the back end in the same order. If you select correlation fields, the client sends the node instances in an order based on the selected correlation fields, and expects them to be sent back from the back end in the same order. The sort order allows identification of the instances as long as the back end does not delete or add any fields. Sort-based correlation is faster than field-based correlation, but may result in errors if the back end adds or deletes fields.

An obvious solution for unique identification is to use the node’s synchronization key (SyncKey) as a correlation key. However, the SyncKey is a DOE key and not sent to the back end. To allow this, you must adjust the back-end adapter such that the BAPI wrappers also include the SyncKey.

Example

Sales orders created by a sales representative visiting a customer.

In this example case, the client application creates two orders with the order IDs SO1 and SO2 with two order items each for the same customer. The order ID is the back-end key of the root node (order header) of the Sales Order data object. Due to data object-internal key mapping, the sales order ID created by the client is saved in all the node instances (order items, example: ITEM_11, ITEM_12) of the sales order.

The logic and management of sales order IDs, however, is handled by the back end. Therefore the back end may change the order ID in the order header node because, for example, the ID created by the client application is not unique in the back end. The changed object instance is then sent by the back-end system to the DOE and from there to the client. Since the nodes are sent by the DOE as separate messages, the client cannot identify the parent node instance to which a node instance belongs because the back-end key of the root node has changed.

For example, let us assume that the back-end system receives instance 1 of the above data object. In the back-end system, SO1 is already assigned to a service order that has been created in the back-end system, and so the back-end system changes ORDER_ID SO1 to SO2 and SO2 to SO3. The client cannot identify which order item instances are related to which order header instance as it does not receive information about which back-end key is changed to which.

In this case, you can define correlation fields or use sort-based correlation without correlation fields to allow the instances to be identified by the client.