Data Package Job for Real-Time Data Acquisition

A background process that is triggered from another ODP background process in order to load new data records with another data transfer process.

The data package job controls and monitors a transfer process for real-time data acquisition. Depending on the scenario used, the daemon performs the following tasks:

• This controls the status of the data transfer.
• With a data transfer process, it starts the extraction from the ODP, transforms the data and posts it to the InfoProvider.
• It closes and opens requests when threshold values are reached.
• It triggers the subsequent process chains.

The data package job receives information from the data transfer process about the data to be transferred or further updated, InfoProviders to be filled, and when a request should be closed and a new one opened.

How it Works

In the ODP source system, a daemon (job name starts with ODQ_DAEMON) runs, checking every 15 seconds for new data records or data changes (deltas) for a real-time subscriber. If it finds any, a notification is sent to the subscriber. In the case of SAP Business Warehouse, a notification is sent in a Remote Function Call (RFC) containing a list of data transfer processes and the last data package retrieved.

If required, the BW system starts a data package job for every one of these data transfer processes in order to transfer and further process the new data records and/or data changes (deltas). If the data target of the data transfer process is an independent DataStore object or the DataStore object of a HybridProvider, the system activates the data automatically and writes it to the change log.

Operating Mode

Unlike a daemon for real-time data acquisition, which runs idly most of the time, a data package job processes exactly one data package from a data transfer process and then terminates. If a notification about further data records is received from the ODP source system during processing, the data package job starts a new data package job at the end of the process. Otherwise, the next data package job does not start until new notifications are received. This happens without having to close the requests for real-time data acquisition.

Error Behavior

The data package job writes each successfully executed step to a control table. If the extraction or update is terminated, the daemon restarts itself and continues the process from the point where it was terminated. It repeats the entire substep with the granularity of a data package. You can define the maximum number of failed attempts that the data package job can make when closing requests before it terminates completely in the data transfer process.