Delta Query Support

Delta query means “give me all services that were created/changed/deleted since I last asked”. In other words, the delta query protocol defines a pull-model protocol for clients to obtain changes from a store. Clients can poll the store for changes periodically or as required, or capable stores can provide change notifications to alert the client when changes are available to be pulled. The underlying design focuses on the ability of the store not to have to keep any per-client state. This delta query function can be used for the Catalog Service, for example.

The delta query is especially useful for projects where performance is critical and loading each time the full payload is not acceptable, for example for projects where the client often polls to get the latest data and any updates. Thus, for client and server performance it can be very useful to always show the current state in a poll-interaction but with the lowest overhead on both client and server.

The delta query protocol has the following components:

• Delta token

  The store generates a token that can be used to retrieve deltas from the current state of the store. This may be returned as part of a request for data or through some other store-defined means.

• Issuing a request with a delta token

  When the delta token is used to request data, changed or inserted results are returned ordered according to when the changes were made (older changes first).

For changed records, the result must include all changed fields and may contain additional unchanged fields from the record.

Deleted entries are a part of delta request (GET FEED) where the service returns not only the feed of entities but also the list of entities (to be more exact, entity IDs, called tombstones) that have been deleted since the time point specified in the delta token. The delta token URI parameter is processed and passed in exactly the same way as any other URI parameter, that is, as part of the request context. To support this feature, the backend response XML has been extended with the <DELETED_ENTITIES> tag that contains the list of deleted entities. The structure is the same as <ENTITY_SET> which is already present to contain the main feed, or a subset of it at least containing all key fields. Only the IDs need to be filled by the backend application.

A delta token needs to be issued in a normal feed to signal to the client that it supports delta tokens.

The way the delta token is issued and constructed is done by the issuer and is of his choice. Typically it will be a date-time-token (for default cases). In case of a date-time-token you only need to take into account that the delta token should be related to one specific time zone, such as UTC.

A delta token call from a client needs to be handled appropriately. The delta token holds specific information which is known by the issuer. With this information it should be easy to extract the changed data.

You can implement delta token in several ways and these options can be grouped into two main approaches.

With both approaches, the payload of the response is reduced, but only the first approach is also able to optimize the performance in the backend system.

• The first approach calculates deltas at modification time: The ABAP system tracks relevant changes when they occur. At request time, the deltas are already prepared and thus available. On the one hand, this approach requires more development. On the other hand it is more scalable and has an optimized overall performance.

• The second approach is based on delta determination at request time where the system compares old and new state to find out which records have been changed/deleted. The implementation effort is rather small but it does not optimize the performance of the backend. That means, the more records you have in the full collection, the longer the response time of the request.

  For this second approach a basic implementation is described in an SAP Community Network document under http://scn.sap.com/docs/DOC-47043 with a delta request log component. The implementation effort described is rather low, but the performace on the backend is not optimized.

Example of issuing a date-time based delta token:

*   determine delta token for catalog service.
    GET TIME STAMP FIELD lv_timestamp.
    convert time STAMP lv_timestamp TIME ZONE 'UTC   '
        INTO DATE lv_date
             TIME lv_time.
    CONCATENATE lv_date lv_time into es_response_context-deltatoken. 
 

Example of handling a delta token:

data lo_delta_context type /IWBEP/IF_MGW_REQ_ENTITYSTDLTA 
* first get the delta token
lo_delta_context =? io_tech_request_context.
lv_delta_token = lo_delta_context->get_deltatoken( ). "converts it automatically into a datetime    
loop at ET_SERVICE_MODEL_INFOS REFERENCE INTO lr_svc_mdl_info.
      TRY .
        
        if lr_svc_mdl_info->*-last_modified le lv_delta_token.
*         then add it to the return table
          delete ET_SERVICE_MODEL_INFOS index sy-tabix.
        endif.
*       catch as much as possible
      CATCH cx_root INTO lx_exception.
*       service does not exist anymore or is not compileable --> put it to the deleted entries
        lr_svc_mdl_info->*-deleted = abap_true.
      ENDTRY.
    endloop.