Creating Data Transfer Processes 

Use

You use the data transfer process (DTP) to transfer data from source objects to target objects in BI. You can also use the data transfer process to access InfoProvider data directly. 

Prerequisites

You have used transformations to define the data flow between the source and target object.

Procedure

Creating Data Transfer Processes Using Process Chains

You are in the plan view of the process chain that you want to use for the data transfer process.

Process type Data Transfer Process is available in the Loading Process and Post processing process category.

...

       1.      Use drag and drop or double-click to insert the process into the process chain.

       2.      To create a data transfer process as a new process variant, enter a technical name and choose Create.

The dialog box for creating a data transfer process appears.

       3.      Select Standard (Can Be Scheduled) as the type of data transfer process.

You can only use the type DTP for Direct Access as the target of the data transfer process for a VirtualProvider. More information: Creating Data Transfer Processes for Direct Access.

If you use the data transfer process in a process chain, you can only use the standard data transfer as the target of the data transfer process for a DataStore object. More information about data transfer processes for real-time data acquisition: Creating Data Transfer Processes for Real-Time Data Acquisition.

       4.      Select the target and source object.

First select the object type.

Two input helps are available when you select the source and target objects:

 Using the quick info Input Help: Existing Paths
This input help provides a selection of the objects for the starting object that were already defined in the data flow. If there is only one object in the data flow, this is selected by default.

 List with the quick info Input Help: List of All Objects

This input help enables you to select the object from the complete list of BI objects.

       5.      Choose Continue.

       6.      The data transfer process maintenance screen appears.

The header data for the data transfer process shows the description, ID, version and status of the data transfer process, along with the delta status.

       7.      On the Extraction tab page, specify the parameters:

                            a.      Choose Extraction Mode.

You can choose Delta or Full mode.

Unlike delta transfer using an InfoPackage, an explicit initialization of the delta process is not necessary for delta transfer with a DTP. When the data transfer process is executed in delta mode for the first time, all existing requests are retrieved from the source, and the delta status is initialized.

Only the extraction mode Full is available for the following sources:

■       InfoObjects

■       InfoSets

■       DataStore Objects for Direct Update

If you have selected transfer mode Delta, you can define further parameters:

                                                  i.       With Only Get Delta Once, define if the source requests should be transferred only once.

Setting this flag ensures that the content of the InfoProvider is an exact representation of the source data.

A scenario of this type may be required if you always want an InfoProvider to contain the most recent data for a query, but technical reasons prevent the DataSource on which it is based from delivering a delta (new, changed or deleted data records). For this type of DataSource, the current data set for the required selection can only be transferred using a full update.

In this case, a DataStore object cannot normally be used to determine the missing delta information (overwrite and create delta). If this is not logically possible because data is deleted in the source without delivering reverse records for example, you can set this indicator and perform a snapshot scenario. Only the most recent request for this DataSource is retained in the InfoProvider. Earlier requests for the DataSource are deleted from the (target) InfoProvider before a new one is requested (this is done by a process in a process chain, for example). They are not transferred again by the DTP delta process. When the system determines the delta when a new DTP request is generated, these earlier (source) requests are considered to have been retrieved.

                                                ii.       Define if you want to Get All New Data in Source Request by Request.

Since a DTP bundles all transfer-relevant requests from the source, it sometimes generates large requests. If you do not want to use a single DTP request to transfer the dataset from the source because the dataset is too large, you can set the Get All New Data in Source Request by Request flag. This specifies that you want the DTP to read only one request from the source at a time. Once processing is completed, the DTP request checks for further new requests in the source. If it finds any, it automatically creates an additional DTP request.

You can change this flag at any time, even if data has already been transferred. If you set this flag, you can transfer data by request as a one-off activity. If you deselect the flag, the DTP goes back to transferring all new source requests at once at periodic scheduled intervals.

 

If you set the indicator for a DTP that was created prior to NetWeaver 7.0 Support Package Stack 13, the DTP request only retrieves the first source request. This restricts the way in which the DTPs can be used because requests accumulate in the source, and the target might not contain the current data. To avoid this, you need to execute the DTP manually until all the source requests have been retrieved. The system therefore also displays the following indicator for such DTPs: Retrieve Until No More New Data. If you also set this indicator, the DTP behaves as described above and creates DTP requests until all the new data has been retrieved from the source.

                            b.      If necessary, determine filter criteria for the delta transfer. To do this, choose  Filter.

This means that you can use multiple data transfer processes with disjunctive selection conditions to efficiently transfer small sets of data from a source into one or more targets, instead of transferring large volumes of data. The filter thus restricts the amount of data to be copied and works like the selections in the InfoPackage. You can specify single values, multiple selections, intervals, selections based on variables, or routines. Choose Change Selection to change the list of InfoObjects that can be selected.

The  icon next to pushbutton  Filter indicates that predefined selections exist for the data transfer process. The quick info text for this icon displays the selections as a character string.

                            c.      Choose  Semantic Groups to specify how you want to build the data packages that are read from the source (DataSource or InfoProvider). To do this, define key fields. Data records that have the same key are combined in a single data package.

This setting is only relevant for DataStore objects with data fields that are overwritten. This setting also defines the key fields for the error stack. By defining the key for the error stack, you ensure that the data can be updated in the target in the correct order once the incorrect data records have been corrected.

More information: Handling Data Records with Errors and Error Stack.

During parallel processing of time-dependent master data, the semantic key of the DTP may not contain the field of the data source.

                            d.      Define any further settings that depend on the source object and data type.

       8.      On the Update tab page, specify the parameters:

                            a.      Make the settings for error handling. Define the following:

■       How you want to update valid records when errors occur.

■       How many errors can occur before the load process terminates.

More information: Handling Data Records with Errors.

                            b.      Apply any further settings that are relevant for the target object.

       9.      On the Execute tab page, define the parameters:

On this tab page, the process flow of the program for the data transfer process is displayed in a tree structure.

                            a.      Specify the status that you want the system to adopt for the request if warnings are displayed.

                            b.      Specify how you want the system to define the overall status of the request.

                            c.      Normally the system automatically defines the processing mode for the background processing of the respective data transfer process.

If you want to execute a delta without transferring data, like when simulating the delta initialization with the InfoPackage, select No data transfer; delta status in source: fetched as processing mode. This processing mode is available when the data transfer process extracts in delta mode. In this case you execute the DTP directly in the dialog. A request started like this marks the data that is found in the source as fetched, without actually transferring it to the target.
If delta requests have already been transferred for this data transfer process, you can still choose this mode.

If you want to execute the data transfer process in debugging mode, choose processing mode Serially in the Dialog Process (for Debugging). In this case, you can define breakpoints in the tree structure for the process flow of the program. The request is processed synchronously in a dialog process and the update of the data is simulated. If you select expert mode, you can also define selections for the simulation and activate or deactivate intermediate storage in addition to setting breakpoints. More information: Simulating and Debugging DTP Requests.

More information: Processing Types in the Data Transfer Process

   10.      Check the data transfer process, then save and activate it.

   11.      Start process chain maintenance.

The data transfer process is displayed in the plan view and can be linked into your process chain. When you activate and schedule the chain, the system executes the data transfer process as soon as it is triggered by an event in the predecessor process in the chain.

Creating Data Transfer Processes from the Object Tree in the Data Warehousing Workbench

The starting point when creating a data transfer process is the target where you want to transfer data to. In the Data Warehousing Workbench, an object tree is displayed and you have highlighted the target object.

...

       1.      In the context menu, choose Create Data Transfer Process.

The dialog box for creating a data transfer process appears.

       2.      Proceed as described in steps 3 to 10 in the procedure for creating a data transfer process using a process chain. In step 4, you specify the source object only.

You can now execute the data transfer process directly.

Additional Functions

Choose Goto → Overview of DTP to display information about the source and target objects, the transformations, and the last changes to the data transfer process.

Choose Goto → Batch Manager Settings to make settings for parallel processing with the data transfer process. More information: Setting Parallel Processing of BI Processes

By choosing Goto  → Settings for DTP Temporary Storage, you can define the settings for the temporary storage. More information: Handling Data Records with Errors

You can define the DB storage parameters with Extras  →  Settings for Error Stack. More information: DB Memory Parameters