For some SAP reports, the nights are getting too short. Especially at customers with large volumes of data, some SAP reports that customarily run in the background processing system (such as material planning runs) may have run times of many hours. It can be difficult to finish such jobs runs in the “night-time” that is available, especially if dialog users are spread across several time zones.
With Release 3.1G, SAP offers a solution to the “short nights” problem: parallel-processed background jobs. Long-running SAP reports can now implement parallel processing, which lets them parcel out the work to be done to available work processes in the SAP System and then collect and synchronize the results.
Parallel processing is implemented in ABAP reports and programs, not in the background processing system itself. That means that jobs are only processed in parallel if the report that runs in a job step is programmed for parallel processing. Such reports can also process in parallel if they are started interactively.
Parallel processing is implemented with a special variant of asynchonous RFC. It is important that you use only the correct variant for your own parallel processing applications: the CALL FUNCTION STARTING NEW TASK DESTINATION IN GROUP keyword. Using other variants of asynchronous RFC circumvents the built-in safeguards in the correct keyword, and can bring your system to its knees.
Parallel processing is implemented in the application reports that are to run in the background. You can implement parallel processing in your own background applications by using the following function modules and ABAP keywords:
● Function module SPBT_INITIALIZE: Optional. Use to determine the availability of resources for parallel processing.
You can do the following:
– check that the parallel processing group that you have specified is correct.
– find out how many work processes are available so that you can more efficiently size the packets of data that are to be processed.
● ABAP keyword CALL FUNCTION <function> STARTING NEW TASK <taskname> with the DESTINATION IN GROUP argument. Use this keyword to have the SAP System execute the function module call in parallel. Typically, you place this keyword in a loop in which you divide up the data that is to be processed into work packets. You can pass the data that is to be processed in the form of an internal table (EXPORT, TABLE arguments). The keyword implements parallel processing by dispatching asynchronous RFC calls to the servers that are available in the RFC server group specified for the processing.
● Function module SPBT_GET_PP_DESTINATION: Optional. Call immediately after the CALL FUNCTION keyword to get the name of the server on which the parallel processing task will be run.
● Function module SPBT_DO_NOT_USE_SERVER: Optional. Excludes a particular server from further use for processing parallel processing tasks. Use in conjunction with SPBT_GET_PP_DESTINATION if you determine that a particular server is not available for parallel processing (for example, COMMUNICATION FAILURE exception if a server becomes unavailable).
● ABAP keyword WAIT: Required if you wish to wait for all of the asynchronous parallel tasks created with CALL FUNCTION to return. This is normally a requirement for orderly background processing. May be used only if the CALL FUNCTION includes the PERFORMING ON RETURN addition.
● ABAP keyword RECEIVE: Required if you wish to receive the results of the processing of an asynchronous RFC. RECEIVE retrieves IMPORT and TABLE parameters as well as messages and return codes.
● Before you implement parallel processing, make sure that your background processing application and your SAP System meet these requirements:
● Logically independent units of work. The data processing task that is to be carried out in parallel must be logically independent of other instances of the task. That is, the task can be carried out without reference to other records from the same data set that are also being processed in parallel, and the task is not dependent upon the results of others of the parallel operations. For example, parallel processing is not suitable for data that must be sequentially processed or in which the processing of one data item is dependent upon the processing of another item of the data.
● By definition, there is no guarantee that data will be processed in a particular order in parallel processing or that a particular result will be available at a given point in processing.
● ABAP requirements (see also the online documentation for the CALL FUNCTION STARTING NEW TASK DESTINATION IN GROUP keyword):
– The function module that you call must be marked as externally callable. This attribute is specified in the Remote function call supported field in the function module definition (transaction SE37).
– The called function module may not include a function call to the destination “BACK.”
– The calling program should not change to a new internal session after making an asynchronous RFC call. That is, you should not use SUBMIT or CALL TRANSACTION in such a report after using CALL FUNCTION STARTING NEW TASK.
– You cannot use the CALL FUNCTION STARTING NEW TASK DESTINATION IN GROUP keyword to start external programs.
– In calls between systems, both systems must be of Release 3.0A or higher.
● SAP System resources: In order to process tasks from parallel jobs, a server in your SAP System must have at least three dialog work processes. It must also meet the workload criteria of the parallel processing system: Dispatcher queue less than 10% full, at least one dialog work process free for processing tasks from the parallel job.
The parallel processing system has built-in safeguards that eliminate the possibility that a parallel job can soak up all of the resources in an SAP System and cause performance problems for other jobs or other users.
In addition to these built-in safeguards, you can optimize the sharing of resources through RFC server groups. In the context of parallel processing, a group specifies the set of SAP application servers that can be used for executing a particular program in parallel. By default (CALL FUNCTION STARTING NEW TASK with the addition DESTINATION IN GROUP DEFAULT), the group is all servers that meet the resource criteria. But you can also create your own more limited groups. You can view and maintain groups with transaction RZ12 (Tools → Administration → Administration → Network → RFC destinations and then RFC → RFC groups).
You must specify the group to use in both the SPBT_INITIALIZE function module (if used) and in the ABAP CALL FUNCTION STARTING NEW TASK keyword. Only one group is allowed per parallel ABAP report or program (job step).
In the function modules that you call, you should use exceptions for any error reporting, and not the MESSAGE keyword. Exception handling is fully under your control in asynchronous RFC. You simply add the exceptions generated by the function module to the reserved SYSTEM_FAILURE and COMMUNICATIONS_FAILURE exceptions of the CALL FUNCTION keyword. You can then handle the exceptions in the program that launches the parallel programming tasks.
If the system profile parameter auth/rfc_authority_check is set (value 1), then the system automatically checks at the CALL FUNCTION keyword whether the user of the background job has the required RFC authorization. The RFC authorization object is S_RFC Authorization check at RFC access. The authorization checks access to function modules by function module group. That is, whether a user has the right to run function modules that belong to a particular group.
You can test a user’s RFC authorization with the function module AUTHORITY_CHECK_RFC. This function module returns RC = 0 if the user is authorized for the group that you name. The function module does not check whether an authorization check actually takes place.
This sample program shows how to create a report that would execute in parallel if started interactively or in the background processing system. It is based upon the online documentation for the ABAP CALL FUNCTION STARTING NEW TASK documentation.
SPBT_INITIALIZE: After declaring data, the report calls the SPBT_INITIALIZE function module. The call lets the program check that the parallel processing group is valid and that resources are available. This call is optional. If you do not call the function module, then the SAP System itself calls SPBT_INITIALIZE to initialize the RFC server group.
Since the function module returns the number of available work processes, the call could be used to decide how to size the work packets that are to be processed. If 10 work processes were available, you could, for example, divide the data into 10 packets for processing. However, there’s no guarantee that the number of free work processes will not change between the time of the call and the time that your program sends off its work packets unless you are working with a group of servers that are reserved for your job.
It’s of course also possible to run through the data making one parallel processing call for each record that is to be processed. In this case, of course, no attempt is made to optimize the sizing of work packets.
CALL FUNCTION... Loop The heart of the report is a DO loop in which the function module that is to be processed in parallel is called (CALL FUNCTION STARTING NEW TASK DESTINATION IN GROUP). The loop in this example is controlled by a simple count-down mechanism. In a production report, you’d repeat the loop until all of your data has been sent off for processing with CALL FUNCTION.
For purposes of simplicity, the loop in this example calls a function module that requires no data (RFC_SYSTEM_INFO). In a production report, you would include logic to select a record or group of records for processing. You would package these recorts in an internal table and include them with the parallel processing call using the EXPORTING or TABLES additions of CALL FUNCTION STARTING NEW TASK.
For purposes of recovery, a production program should also include logic for logging the progress of processing. Should the program terminate abnormally in the middle of processing, it is essential that you be able to determine which data has already been processed and with which data the report should resume. In simplest form, this logic should log the data that has been dispatched for parallel processing and should note the completion of the processing of each unit of data.
Task management: A 0 return code (SY-SUBRC) from CALL FUNCTION indicates that your parallel processing task has been successfully dispatched. It’s now important that your job keep track of these parallel processing tasks. Your task management should take care of two assignments:
● Generating unique task names for each CALL FUNCTION task.
Use the taskname that you specified in CALL FUNCTION to identify parallel processing tasks. Each such task that you dispatch should have a unique name so that you can correctly identify returns from the task.
● Waiting for resources to become available, should all parallel processing resources (dialog work processes) temporarily be in use. See “Handling the CURRENTLY_NO_RESOURCES_AVAIL exception,” below.
● Determining when all tasks have been completed. Only then can your program terminate. See “Waiting for job completion”, below, for more information.
In the example, the task management increments a counter as the task name and maintains a table of the tasks that have been dispatched. As an optional feature, the task management uses SPBT_GET_PP_DESTINATION to find out where each parallel task is being processed.
Handling the RESOURCE_FAILURE exception: As each parallel processing task is dispatched, the SAP System counts down the number of resources (dialog work processes) available for processing additional tasks. This count goes up again as each parallel processing task is completed and returns to your program.
Should your parallel processing tasks take a long time to complete, then the parallel processing resources may temporarily run out. In this case, CALL FUNCTION returns the exception RESOURCE_FAILURE. This means simply that all dialog work processes in the RFC group that your program is using are in use.
Your program must now wait until resources become available and then re-issue the CALL FUNCTION that failed. In the sample program, we use a simple, reasonably failsafe wait mechanism. The program waits for parallel processing tasks to return, freeing up resources. The WAIT also specifies a initial timeout of 1 second. If the CALL FUNCTION again fails, the WAIT is repeated with a longer time-out. You can increase the time-outs if you expect that your parallel tasks will take longer to complete. You should also add code to exit from the retry loop after a suitable number of iterations.
Receiving replies: The CALL FUNCTION keyword triggers processing of the form RETURN_INFO when each parallel processing task completes. This form uses the RECEIVE keyword to capture the results of the parallel processing. In this case, the structure RFCSI_EXPORT from RFC_SYSTEM_INFO is collected into the internal structure INFO.
RECEIVE is needed to gather IMPORTING and TABLE returns of an asynchronously executed RFC function module.
Example: Assume that your report generates a list. You would use a form like RETURN_INFO and RECEIVE to collect the list entries generated by each parallel processing task. After all parallel tasks have returned, your report could then sort or otherwise further process the list entries before presenting them.
Waiting for job completion: As part of your task management, your job must wait until all of the parallel processing tasks have been completed. To do this, the sample program uses the WAIT keyword to wait until the number of completed parallel processing tasks is equal to the number of tasks that were created. Independently of this WAIT; the RETURN_INFO form is triggered. RETURN_INFO keeps track of the number of completed parallel processing tasks, so that the WAIT condition can be correctly evaluated.
* Data Declarations
DATA: GROUP LIKE RZLLITAB-CLASSNAME VALUE ' ',
"Parallel processing group.
"SPACE = group default (all
WP_AVAILABLE TYPE I, "Number of dialog work processes
"available for parallel processing
"(free work processes)
WP_TOTAL TYPE I, "Total number of dialog work
"processes in the group
MSG(80) VALUE SPACE, "Container for error message in
"case of remote RFC exception.
INFO LIKE RFCSI, C, "Message text
JOBS TYPE I VALUE 10, "Number of parallel jobs
SND_JOBS TYPE I VALUE 1, "Work packets sent for processing
RCV_JOBS TYPE I VALUE 1, "Work packet replies received
EXCP_FLAG(1) TYPE C, "Number of RESOURCE_FAILUREs
TASKNAME(4) TYPE N VALUE '0001', "Task name (name of
"parallel processing work unit)
BEGIN OF TASKLIST OCCURS 10, "Task administration
TASKNAME(4) TYPE C,
RFCDEST LIKE RFCSI-RFCDEST,
RFCHOST LIKE RFCSI-RFCHOST,
END OF TASKLIST.
* Optional call to SBPT_INITIALIZE to check the
* group in which parallel processing is to take place.
* Could be used to optimize sizing of work packets
* work / WP_AVAILABLE).
CALL FUNCTION 'SPBT_INITIALIZE'
GROUP_NAME = GROUP
"Name of group to check
MAX_PBT_WPS = WP_TOTAL
"Total number of dialog work
"processes available in group
"for parallel processing
FREE_PBT_WPS = WP_AVAILABLE
"Number of work processes
"available in group for
"parallel processing at this
INVALID_GROUP_NAME = 1
"Incorrect group name; RFC
"group not defined. See
INTERNAL_ERROR = 2
"SAP System error; see the
"system log (transaction
"SM21) for diagnostic info
PBT_ENV_ALREADY_INITIALIZED = 3
"Function module may be
"called only once; is called
"automatically by SAP System if you
"do not call before starting
CURRENTLY_NO_RESOURCES_AVAIL = 4
"No dialog work processes
"in the group are available;
"they are busy or server load
"is too high
NO_PBT_RESOURCES_FOUND = 5
"No servers in the group
"met the criteria of >
"two work processes
CANT_INIT_DIFFERENT_PBT_GROUPS = 6
"You have already initialized
"one group and have now tried to
"initialize a different group.
OTHERS = 7..
"Everything’s ok. Optionally set up for optimizing size of
"Non-existent group name. Stop report.
MESSAGE E836. "Group not defined.
"System error. Stop and check system log for error
"Programming error. Stop and correct program.
MESSAGE E833. "PBT environment was already initialized.
"No resources: this may be a temporary problem. You
"may wish to pause briefly and repeat the call. Otherwise
"check your RFC group administration: Group defined
"in accordance with your requirements?
MESSAGE E837. "All servers currently busy.
"Check your servers, network, operation modes.
* Do parallel processing. Use CALL FUNCTION STARTING NEW TASK
* DESTINATION IN GROUP to call the function module that does the
* work. Make a call for each record that is to be processed, or
* divide the records into work packets. In each case, provide the
* set of records as an internal table in the CALL FUNCTION
* keyword (EXPORT, TABLES arguments).
CALL FUNCTION 'RFC_SYSTEM_INFO' "Function module to perform
STARTING NEW TASK TASKNAME "Name for identifying this
DESTINATION IN GROUP group "Name of group of servers to
"use for parallel processing.
"Enter group name exactly
"as it appears in transaction
"RZ12 (all caps). You may
"use only one group name in a
"particular ABAP program.
PERFORMING RETURN_INFO ON END OF TASK
"This form is called when the
"RFC call completes. It can
"collect IMPORT and TABLES
"parameters from the called
"function with RECEIVE.
COMMUNICATION_FAILURE = 1 MESSAGE msg
"Destination server not
"reached or communication
"interrupted. MESSAGE msg
"captures any message
"returned with this
"exception (E or A messages
"from the called FM, for
"example. After exception
"1 or 2, instead of aborting
"your program, you could use
"exclude this server from
"further parallel processing.
"You could then re-try this
"call using a different
SYSTEM_FAILURE = 2 MESSAGE msg
"Program or other internal
"SAP error. MESSAGE msg
"captures any message
"returned with this
RESOURCE_FAILURE = 3. "No work processes are
"currently available. Your
"program MUST handle this
YOUR_EXCEPTIONS = X. "Add exceptions generated by
"the called function module
"here. Exceptions are
"returned to you and you can
"respond to them here.
"Administration of asynchronous RFC tasks
"Save name of task...
TASKLIST-TASKNAME = TASKNAME.
"... and get server that is performing RFC call.
CALL FUNCTION 'SPBT_GET_PP_DESTINATION'
RFCDEST = TASKLIST-RFCDEST
OTHERS = 1.
WRITE: / 'Started task: ', TASKLIST-TASKNAME COLOR 2.
TASKNAME = TASKNAME + 1.
SND_JOBS = SND_JOBS + 1.
"Mechanism for determining when to leave the loop. Here, a
"simple counter of the number of parallel processing tasks.
"In production use, you would end the loop when you have
"finished dispatching the data that is to be processed.
JOBS = JOBS - 1. "Number of existing jobs
IF JOBS = 0.
EXIT. "Job processing finished
WHEN 1 OR 2.
"Handle communication and system failure. Your program must
"catch these exceptions and arrange for a recoverable
"termination of the background processing job.
"Recommendation: Log the data that has been processed when
"an RFC task is started and when it returns, so that the
"job can be restarted with unprocessed data.
"Remove server from further consideration for
"parallel processing tasks in this program.
"Get name of server just called...
CALL FUNCTION 'SPBT_GET_PP_DESTINATION'
RFCDEST = TASKLIST-RFCDEST
OTHERS = 1.
"Then remove from list of available servers.
CALL FUNCTION 'SPBT_DO_NOT_USE_SERVER'
SERVERNAME = TASKLIST-RFCDEST
INVALID_SERVER_NAME = 1
NO_MORE_RESOURCES_LEFT = 2
"No servers left in group.
PBT_ENV_NOT_INITIALIZED_YET = 3
OTHERS = 4.
"No resources (dialog work processes) available at
"present. You need to handle this exception, waiting
"and repeating the CALL FUNCTION until processing
"can continue or it is apparent that there is a
"problem that prevents continuation.
MESSAGE I837. "All servers currently busy.
"Wait for replies to asynchronous RFC calls. Each
"reply should make a dialog work process available again.
IF EXCP_FLAG = SPACE.
EXCP_FLAG = 'X'.
"First attempt at RESOURCE_FAILURE handling. Wait
"until all RFC calls have returned or up to 1 second.
"Then repeat CALL FUNCTION.
WAIT UNTIL RCV_JOBS >= SND_JOBS UP TO '1' SECONDS.
"Second attempt at RESOURCE_FAILURE handling
WAIT UNTIL RCV_JOBS >= SND_JOBS UP TO '5' SECONDS.
"SY-SUBRC 0 from WAIT shows that replies have returned.
"The resource problem was therefore probably temporary
"and due to the workload. A non-zero RC suggests that
"no RFC calls have been completed, and there may be
IF SY-SUBRC = 0.
ELSE. "No replies
"Endless loop handling
* Wait for end of job: replies from all RFC tasks.
* Receive remaining asynchronous replies
WAIT UNTIL RCV_JOBS >= SND_JOBS.
LOOP AT TASKLIST.
WRITE:/ 'Received task:', TASKLIST-TASKNAME COLOR 1,
30 'Destination: ', TASKLIST-RFCDEST COLOR 1.
* This routine is triggered when an RFC call completes and
* returns. The routine uses RECEIVE to collect IMPORT and TABLE
* data from the RFC function module.
* Note that the WRITE keyword is not supported in asynchronous
* RFC. If you need to generate a list, then your RFC function
* module should return the list data in an internal table. You
* can then collect this data and output the list at the conclusion
* of processing.
FORM RETURN_INFO USING TASKNAME.
DATA: INFO_RFCDEST LIKE TASKLIST-RFCDEST.
RECEIVE RESULTS FROM FUNCTION 'RFC_SYSTEM_INFO'
IMPORTING RFCSI_EXPORT = INFO
COMMUNICATION_FAILURE = 1
SYSTEM_FAILURE = 2.
RCV_JOBS = RCV_JOBS + 1. "Receiving data
IF SY-SUBRC NE 0.
* Handle communication and system failure
READ TABLE TASKLIST WITH KEY TASKNAME = TASKNAME.
IF SY-SUBRC = 0. "Register data
TASKLIST-RFCHOST = INFO_RFCHOST.
MODIFY TASKLIST INDEX SY-TABIX.