The Data Archiving Process (Archiving Workbench) 

Purpose

You can use the data archiving process to archive business object data. This process is implemented by an archiving object related to the business process, and the structure and composition of the data to be archived is defined in this archiving object. During the data archiving process, the system copies the data to archive files, after various checks, then test reads the archived data and removes it from the operational database if successful. There are also subprocesses, in addition to the core process, to display and reload archived data and form package profiles (pre-step).

Prerequisites

You must make all the required settings in the implementation guide or area menu of your application system:

...

      1.      Archiving ® Create Logical File Path:
The path and convention for forming archive file names must be maintained for each archiving object in the FILE basic settings (logical file path and names).

      2.      Archiving ® Maintain Basic Settings for Archiving Objects:
The archiving objects must be fully maintained in the AOBJ basic settings.

      3.      Archiving ® Maintain Global Control of Archiving:
The global control of archiving must be maintained for all archiving objects implemented.

      4.      Archiving ® Archiving Objects ® Define Residence Time for….:
The residence times for the archiving objects must be maintained in the object-specific Customizing settings.

To check the Customizing settings, you can use the AFX_ARCHIVING_CUST_CHECK report. This is part of the area menu.

      5.      Tools ® Parallel Processing ® Maintain Job Distribution:
The job distribution for parallel processing must be maintained for the application types assigned to the archiving objects.

You are authorized to perform the archiving process: The system checks the S_ARCHIVE authorization object for all activities. For Customizing activities, the system checks the S_TABU_DIS authorization object.

 

Process Flow

You can schedule the archiving process using the normal SAP job control (transaction SM36) or the Archive Administration (transaction SARA). The archiving process is structured so it can run in parallel to the live operations.

The basic archiving process consists of three subprocesses, namely analysis, write and delete. These can be coupled together if you make the corresponding Customizing settings. The analysis and write subprocesses can be processed in parallel if you use the parallel processing tool. This is achieved by forming suitable data packages that are processed in parallel by different jobs.

The pre-step is an optional subprocess used to form package profiles and must run before the actual archiving process.

All subprocesses record their current execution status in the archiving activity log. You can display this log and other important information (such as the application log or the job log) using the archiving monitor. For more information on this Archiving Workbench tool, refer to the Archiving Monitor section of the SAP Library.

 

1. Pre-Step

The pre-step subprocess firstly examines the dataset of the archiving object and then generates corresponding package profiles defined by the package limits for parallel processing.  The profiles are stored persistently in the database and then used later by the analysis andwrite subprocesses, if set in the global Customizing settings for the archiving object.

The aim of the package profile generation is to use the parallel package administrators to their full capacity in the analysis and write subprocesses in order to minimize the overall runtime of the archiving process. To this end, the dataset must be divided into separate packages of an equal size, as far as possible. Since the data distribution can change over time, you must repeat the pre-step at regular intervals to ensure suitable profiles exist.

The pre-step is an optional subprocess and is not implemented for all archiving objects.

2. Analysis

The analysis subprocess examines the archiving object data, then determines the archivability status and the resubmission date. To do this, the system makes two checks:

·       Has the data reached the residence times (minimum retention period of data in the operational database) defined in the Customizing settings?

·       Have all the business-related prerequisites for archiving data been met?

The system makes this test with a check module from the application that can, in turn, enable user-defined checks to be made by a customer enhancement (in the form of a BADI, for example).

If both checks are successful, the analysis program sets the archivability status to Can Be Archived. If at least one of the checks fails, the analysis program sets the archiving status to Cannot Be Archived and the resubmission date to the system date + resubmission period.

The resubmission period refers to the period of time for resubmitting the data after the residence time elapses. You define one for each archiving object in the global control of archiving, which ensures that the system does not analyze the data again until the resubmission period elapses. This can reduce the amount of data to be processed, thereby saving time in the analysis process.

To increase efficiency, the analysis program divides the data into separate packages. It either forms these packages dynamically beforehand or using package profiles defined in the pre-step program. These packages are then analyzed by mutually-independent jobs. Each package contains a partial amount of data to be processed, precisely defined by key limits.

When parallel processing starts, the parallel processing tool takes control. This calls up callback routines to execute the individual steps of the process. These routines are provided by archiving for each parallel processing application type of an archiving object (analysis and write, in other words). You can set the number of jobs to be implemented for each application type in the parallel processing Customizing settings. Each job processes different packages, taken sequentially from a worklist shared with all jobs. You can find more information on parallel processing in the Parallel Processing of Mass Data (CA-GTF-TS-PPM) section of the SAP Library.

The following diagram shows the process of parallel processing:

On the lefthand side of the diagram, you can see the time sequence of the process status. For more information on the status, refer to the data element documentation for the Status field in the activity log. Use the archiving monitor to call this up.

3. Write

In the standard set-up, the analysis program starts the write program automatically. The write program copies the data that was identified as being archivable in the analysis subprocess from the operational database to new archive files. It is therefore in this step that the data is actually archived.

The data is written in parallel, mutually-independent jobs, as in the analysis process (see diagram). Each job processes different subpackages taken from the joint worklist. Precisely one archive is created for each parallel processing job; this can consist of one or more archive files.

4. Delete

Once the data has been copied to the archive files, the delete subprocess removes it from the operational database. To this end, the archived data is read from the archive files and only deleted from the database if it is read successfully from the archive. This procedure guarantees that data is only removed from the database if the system is able to archive it in accordance with the rules and settings.

If the write process successfully completes an individual archive file, a separate delete process starts automatically in the standard set-up. The number of delete processes created is always identical to the number of archive files created by the write program.

If the test reading of an archive file fails, the data remains in the operational system with the status Can Be Archived and is then processed again by the write process in the next archiving run. You can either manually delete the archive files that have already been created or leave them in the archive. The latter option is harmless since the files are not removed from the operational system: Archive information structures are only created if a delete process is completed successfully.

Result

Successful Conclusion

If the data is archived in accordance with the rules, the write program stores the archivable data from the operational database in archive files. You can display this data using various read programs.

The options are as follows:

·       Integrate the display of archived data in the application itself.

·       Use separate read programs to display the data.

·       Use other tools to display the data, such as the Archive File Browsers.

Reload

You can recover the archived data using reload programs. During the process, the reload program reads the data from the archive files and then reinserts it into the operational database. However, this is only permitted in an emergency (for example, if the Customizing settings are incorrect or there are technical problems). To avoid misuse of these functions and the risk of inconsistencies occurring through bringing old data into the operational system, you can only reload archived data to the operational database within a period of five days after successful archiving. The reload program sets the archivability status of the reloaded data records to Cannot Be Archived. The resubmission date is not adjusted.

Note that any reload process (including those within the 5 day limit) can lead to database inconsistencies.

Also note that the reload program does not update the archive information structures (AISs). If you use AISs, you must start transaction SARI after reloading individual archive files and then deconstruct the AIS data for these archive files.

The analysis program rewrites the reloaded data records with the status Can Be Archived to the archive file(s) during the next archiving run. To prevent this, you must adjust the residence time Customizing settings for the archiving object in question.

Archive File Storage

It is generally not sufficient simply to write the data to be archived to archive files and then remove it from the database. The archive files themselves must be stored securely and managed to permit later access if required. You are offered the following alternatives:

·       HSM systems (Hierarchical Storage Management)

HSM systems simulate infinitely large file systems. This means that the file system in which the files are archived is included in the memory hierarchy of the HSM system. You must define the file path for this accordingly in the archiving Customizing settings. You do not need to communicate via the SAP ArchiveLink.

·       Storage system via SAP ArchiveLink

If a third-party provider’s storage system is connected via the SAP ArchiveLink, this storage system stores the processed archive file after a delete program is successfully completed.

·       Manual management

If you do not want the files to be stored in a storage system, you can manage the archive files independently.