Select language:

Background documentation Performance Tips for Data Transfer Processes 

Request processing, the process of loading a data transfer process (DTP), can take place in various parallelization upgrades in the extraction and processing (transformation and update) steps. The system selects the most appropriate and efficient processing for the DTP in accordance with the settings in the DTP maintenance transaction, and creates a DTP processing mode.

To further optimize the performance of request processing, there are a number of further measures that you can take:

  By taking the appropriate measures, you can obtain a processing mode with a higher degree of parallelization.

  A variety of measures can help to improve performance, in particular the settings in the DTP maintenance transaction. Some of these measures are source-specific and some are data type-specific.

The following sections describe the various measures that can be taken.

Higher Parallelization in the Request Processing Steps

With a (standard) DTP, you can modify an existing system-defined processing mode by changing the settings for error handling and semantic grouping. The following table shows how you can optimize the performance of an existing DTP processing mode:

Original State of DTP Processing Mode

Processing Mode with Optimized Performance

Measures to Obtain Performance-Optimized Processing Mode

Serial extraction and processing of the source packages (processing mode 3)

Serial extraction, immediate parallel processing (processing mode 2)

Select the grouping fields

Serial extraction and processing of the source packages (processing mode 3)

Parallel extraction and processing (processing mode 1)

Only possible with persistent staging area (PSA) as the source:

Deactivate error handling

Serial extraction, immediate parallel processing (processing mode 2)

Parallel extraction and processing (processing mode 1)

Only possible with PSA as the source:

Deactivate error handling

Remove grouping fields selection

Further Performance-Optimizing Measures

Set the number of parallel processes for a DTP during request processing

To optimize the performance of data transfer processes with parallel processing, you can set the number of permitted background processes for process type Set Data Transfer Process globally in BI Background Management.

To further optimize performance for a given data transfer process, you can override the global setting:

In the DTP maintenance transaction, choose Goto Batch Manager Setting. Under Number of Processes, specify how many background processes should be used to process the DTP, and save your entries.

Note

Make sure that there are enough background processes available for processing. Note that when multiple data transfer processes are executed at the same time, each individual data transfer process uses the number of background processes set.

More information: Setting Parallel Processing of BI Processes

Set the size of data packages

In the standard setting in the data transfer process, the size of a data package is set to 50,000 data records, on the assumption that a data record has a width of 1,000 bytes. To improve performance, you can set the value for the data package size, depending on the size of the main memory.

Enter this value under Package Size on the Extraction tab in the DTP maintenance transaction.

Avoid too large DTP requests with a large number of source requests: Retrieve the data one request at a time

A DTP request can be very large, since it bundles together all transfer-relevant requests from the source. To improve performance, you can stipulate that a DTP request always reads just one request at a time from the source.

To make this setting, select Get All New Data in Source by Request on the Extraction tab in the DTP maintenance transaction. Once processing is completed, the DTP request checks for further new requests in the source. If it finds any, it automatically creates an additional DTP request.

With DataSources as the source: Avoid too small data packages when using the DTP filter

If you extract from a DataSource without error handling and a large amount of data is excluded by the filter, this can cause the data packages loaded by the process to be very small. To improve performance, you can modify this behavior by activating error handling and defining a grouping key.

Select an error handling option on the Updating tab in the DTP maintenance function. Then define a suitable grouping key on the Extraction tab under Semantic Groups. This ensures that all data records belonging to a grouping key within one package are extracted and processed.

With DataStore objects as the source: Extract data from the table of active data for the first delta or during full extraction

The change log grows faster than the table of active data, since it stores before and after images. To optimize performance during extraction in the Full mode or with the first delta from the DataStore object, you can read the data from the table of active data instead of from the change log.

To make this setting, select Active Table (with Archive) or Active Table (without Archive) on the Extraction tab in Extraction from… or Delta Extraction from… in the DTP maintenance function.

With InfoCubes as the source: Use extraction from aggregates

With InfoCube extraction, in the standard setting the data is read from the fact table (F table) and the table of compressed data (E table). To improve performance here, you can use aggregates for the extraction.

Select Use Aggregates on the Extraction tab in the DTP maintenance transaction. The system then compares the outgoing quantity from the transformation with the aggregates. If all InfoObjects from the outgoing quantity are used in aggregates, the data is read from the aggregates during extraction instead of from the InfoCube tables.

Note for using InfoProviders as the source

If not all key fields for the source InfoProvider in the transformation have target fields assigned to them, the key figures for the source will be aggregated by the unselected key fields in the source during extraction. You can prevent this automatic aggregation by implementing a start routine or an intermediate InfoSource. Note, however, that this affects the performance of the data transfer process. For more information, see SAP Note 1147419.

For master data (attributes and texts): Deactivate check for duplicate data records

When you copy master data attributes or texts with a DTP, you can use a flag in the DTP maintenance to check for duplicate data records for the characteristic key of the master data characteristic. This prevents a runtime error from occurring if there are multiple data records for a given characteristic key in a data package. If the flag is set, the updating process is slower. To improve the performance, you can deactivate the check if the following requirements are fulfilled:

  The key of the DataSource provides unique data records.

Note

For BI Content DataSources from SAP source systems, the field Delivery of Duplicate Data Records on the General tab page in the BI DataSource maintenance displays the behavior of the DataSource. Only deactivate the check in the DTP if the field has the value None. Since the field is for information only and is not checked during the extraction, we recommend that you always check the behavior of the DataSource before deactivating the check in the DTP.

  Only a single request is copied from the PSA.

To make this setting, select Get All New Data in Source by Request on the Extraction tab in the DTP maintenance transaction. More information is available further up in this section.

  The key of the DataSource corresponds to the characteristic key of the target InfoObject or represents a subset of the characteristic key.

  The number of records for each characteristic key is not changed in the start or end routines of the transformation.

More Information

Creating Data Transfer Processes

Processing Modes in the Data Transfer Process

 

 

Was this page helpful to you?