Reorganization

Clean up process of the database - deletes transactions, precalculated values etc. based on configuration.

The reorganization process is divided into separately configured steps, so that transactions can be deleted for an hour, and then the report archive can be deleted for 10 minutes.

Currently, reorganization only contains a cleanup of expired data; there is no maintenance of database statistics, etc.

Each step logs relevant information about its progress and result. All delete steps have the same properties, but with a different prefix, which allows each individual step to be fine-tuned.

Delete processes are possible independently of day-end closing. This allows database entries to be deleted depending on defined criteria and periods.

Note:
Delete steps may only be performed after consultation with the responsible project engineer because it is necessary to determine which entries are to be deleted in which tables.

The following Delete steps are configurable:

  • Delete old transactions from the Enterprise Transaction Pool
  • Delete old unfinished transactions from the Enterprise Transaction Pool
  • Delete old transaction process protocols from the Enterprise Transaction Pool
  • Delete old suspended transactions from the Enterprise Transaction Pool
  • Delete old suspended scale transactions from the Enterprise Transaction Pool
  • Delete old suspended empties transactions from the Enterprise Transaction Pool
  • Delete old reports from the archive
  • Delete old reporting periods
  • Delete old invoice number cache

Configuration of the reorganization process is split up between SDC and POS Server, depending on the reorganization step (process). This means that reorganization-service.properties exists in both applications and the parameters of the processes are divided in the following way between the applications:

  • SDC:
    • gkr.tx-pool.deleteOldTransaction
    • gkr.tx-pool.deleteOldUnfinishedTransaction
    • gkr.tx-pool.deleteOldSuspendedTransaction
    • gkr.tx-pool.deleteOldSuspendedScaleTransaction
    • gkr.tx-pool.deleteOldSuspendedEmptiesTransaction
    • gkr.tx-pool.deleteTransactionProtocols
    • gkr.report.deleteOldArchive
    • Delete old temporary report data and report export entries
  • POS Server:
    • gkr.calculation-service.deleteOldReportingPeriod
    • gkr.accounting-service.deleteOldInvoiceNumberCache
    • gkr.reporting-service.deleteOldReports

In addition, it is possible to configure the storage period ('...backupTimePeriod') of some processes at store level, so that the retention period of old transactions, for example, can be defined differently for different stores (see reorganization-service.properties at store level on SDC).

Global configuration file reorganization-service.properties on SDC:

#
# Configuration file for process specified at https://confluence.gk.gk-software.com/display/GKRSMDOC/.Reorganization+-+01250202406+v5.0
#
# ... backupTimePeriod - defines how old data entries should be deleted. Each of affected data types has its own method of determining its age
# ... processing - its possible to disable deletion of certain data type by this property
# ... incrementalDeletePoolRange - Data are processed in some chunks. This defines size of that chunk. This property affects performance.
# ... incrementalDeleteStopAfterRecord - Stop condition defining total amount of data deleted of certain type
# ... incrementalDeleteStopAfterPeriod - Stop condition defining duration of deletion for one data type
# ... incrementalDeleteStopOnTime - Stop condition defining that data type should be processed until some time is reached.
# ... correlatedTxBackupTimePeriod. - deletion condition for correlating entries.Applicable only to ceratin tx types.
# If transaction is being correlated to by some child transaction, this defines expiration period of this correlation.
# Generally should be set to maximum backupTimePeriod used among other transaction deletion tasks.
# Defaults to corresponding ..backupTimePeriod
#Global setting that terminates whole process(not specific to data type) defined time
gkr.reorganization.stopAfterPeriod=2h
#for transaction deletion it might make performance-sense(must be measured) to trigger deleting multiple times for
#different data chunks.
gkr.tx-pool.deleteOldTransaction.selectChunksParallelismLevel=10
gkr.tx-pool.deleteOldTransaction.paraleismLevel=1
gkr.tx-pool.deleteOldTransaction.backupTimePeriod=180d
gkr.tx-pool.deleteOldTransaction.processing=true
gkr.tx-pool.deleteOldTransaction.chunkSize=200
gkr.tx-pool.deleteOldTransaction.incrementalDeleteStopAfterRecord=12000
gkr.tx-pool.deleteOldTransaction.incrementalDeleteStopAfterPeriod=60m
gkr.tx-pool.deleteOldTransaction.incrementalDeleteStopOnTime=07\:00
gkr.tx-pool.deleteOldUnfinishedTransaction.selectChunksParallelismLevel=10
gkr.tx-pool.deleteOldUnfinishedTransaction.backupTimePeriod=30d
gkr.tx-pool.deleteOldUnfinishedTransaction.processing=true
gkr.tx-pool.deleteOldUnfinishedTransaction.chunkSize=200
gkr.tx-pool.deleteOldUnfinishedTransaction.incrementalDeleteStopAfterRecord=12000
gkr.tx-pool.deleteOldUnfinishedTransaction.incrementalDeleteStopAfterPeriod=60m
gkr.tx-pool.deleteOldUnfinishedTransaction.incrementalDeleteStopOnTime=07\:00
gkr.tx-pool.deleteOldSuspendedTransaction.selectChunksParallelismLevel=10
gkr.tx-pool.deleteOldSuspendedTransaction.backupTimePeriod=180d
gkr.tx-pool.deleteOldSuspendedTransaction.correlatedTxBackupTimePeriod=180d
gkr.tx-pool.deleteOldSuspendedTransaction.processing=true
gkr.tx-pool.deleteOldSuspendedTransaction.chunkSize=200
gkr.tx-pool.deleteOldSuspendedTransaction.incrementalDeleteStopAfterRecord=12000
gkr.tx-pool.deleteOldSuspendedTransaction.incrementalDeleteStopAfterPeriod=60m
gkr.tx-pool.deleteOldSuspendedTransaction.incrementalDeleteStopOnTime=07\:00
gkr.tx-pool.deleteOldSuspendedEmptiesTransaction.selectChunksParallelismLevel=10
gkr.tx-pool.deleteOldSuspendedEmptiesTransaction.backupTimePeriod=180d
gkr.tx-pool.deleteOldSuspendedEmptiesTransaction.correlatedTxBackupTimePeriod=180d
gkr.tx-pool.deleteOldSuspendedEmptiesTransaction.processing=true
gkr.tx-pool.deleteOldSuspendedEmptiesTransaction.chunkSize=200
gkr.tx-pool.deleteOldSuspendedEmptiesTransaction.incrementalDeleteStopAfterRecord=12000
gkr.tx-pool.deleteOldSuspendedEmptiesTransaction.incrementalDeleteStopAfterPeriod=60m
gkr.tx-pool.deleteOldSuspendedEmptiesTransaction.incrementalDeleteStopOnTime=07:00
gkr.tx-pool.deleteOldSuspendedScaleTransaction.selectChunksParallelismLevel=10
gkr.tx-pool.deleteOldSuspendedScaleTransaction.backupTimePeriod=180d
gkr.tx-pool.deleteOldSuspendedScaleTransaction.correlatedTxBackupTimePeriod=180d
gkr.tx-pool.deleteOldSuspendedScaleTransaction.processing=true
gkr.tx-pool.deleteOldSuspendedScaleTransaction.chunkSize=200
gkr.tx-pool.deleteOldSuspendedScaleTransaction.incrementalDeleteStopAfterRecord=12000
gkr.tx-pool.deleteOldSuspendedScaleTransaction.incrementalDeleteStopAfterPeriod=60m
gkr.tx-pool.deleteOldSuspendedScaleTransaction.incrementalDeleteStopOnTime=07:00
gkr.tx-pool.deleteOldTransactionProtocol.selectChunksParallelismLevel=10
gkr.tx-pool.deleteOldTransactionProtocol.paraleismLevel=1
gkr.tx-pool.deleteOldTransactionProtocol.backupTimePeriod=30d
gkr.tx-pool.deleteOldTransactionProtocol.processing=true
gkr.tx-pool.deleteOldTransactionProtocol.searchBugsInTrTrnTable=false
gkr.tx-pool.deleteOldTransactionProtocol.chunkSize=200
gkr.tx-pool.deleteOldTransactionProtocol.incrementalDeleteStopAfterRecord=999999999
gkr.tx-pool.deleteOldTransactionProtocol.incrementalDeleteStopAfterPeriod=60m
gkr.tx-pool.deleteOldTransactionProtocol.incrementalDeleteStopOnTime=07\:00
gkr.report.deleteOldArchive.backupTimePeriod=180d
#you can add properties defining backupTimePeriod for certain report types
gkr.report.deleteOldArchive.drawer-report.backupTimePeriod=180d
gkr.report.deleteOldArchive.processing=true
gkr.report.deleteOldArchive.incrementalDeletePoolRange=3000
gkr.report.deleteOldArchive.incrementalDeleteStopAfterRecord=12000
gkr.report.deleteOldArchive.incrementalDeleteStopAfterPeriod=60m
gkr.report.deleteOldArchive.incrementalDeleteStopOnTime=07\:00

Store configuration file reorganization-service.properties on SDC:

#
# Configuration file for process store specified at https://confluence.gk.gk-software.com/display/GKRSMDOC/.Reorganization+-+01250202406+v5.15
#
# ... backupTimePeriod - defines how old data entries should be deleted per store. Each of affected data types has its own method of determining its age
gkr.tx-pool.deleteOldTransaction.backupTimePeriod=180d
gkr.tx-pool.deleteOldUnfinishedTransaction.backupTimePeriod=30d
gkr.tx-pool.deleteOldSuspendedTransaction.backupTimePeriod=180d
gkr.tx-pool.deleteOldSuspendedEmptiesTransaction.backupTimePeriod=180d
gkr.tx-pool.deleteOldSuspendedScaleTransaction.backupTimePeriod=180d
gkr.tx-pool.deleteOldTransactionProtocol.backupTimePeriod=30d

Configuration file reorganization-service.properties on POS Server :

#
# Configuration file for process specified at https://confluence.gk.gk-software.com/display/GKRSMDOC/.Reorganization+-+01250202406+v5.0
#
# ... backupTimePeriod - defines how old data entries should be deleted. Each of affected data types has its own method of determining its age
# ... processing - its possible to disable deletion of certain data type by this property
# ... incrementalDeletePoolRange - Data are processed in some chunks. This defines size of that chunk. This property affects performance.
# ... incrementalDeleteStopAfterRecord - Stop condition defining total amount of data deleted of certain type
# ... incrementalDeleteStopAfterPeriod - Stop condition defining duration of deletion for one data type
# ... incrementalDeleteStopOnTime - Stop condition defining that data type shoudl be processed untim some time is reached.
#Global setting that terminates whole process(not specific to data type) defined time
gkr.reorganization.stopAfterPeriod=2h
gkr.calculation-service.deleteOldReportingPeriod.backupTimePeriod=180d
gkr.calculation-service.deleteOldReportingPeriod.processing=true
gkr.calculation-service.deleteOldReportingPeriod.incrementalDeletePoolRange=3000
gkr.calculation-service.deleteOldReportingPeriod.incrementalDeleteStopAfterRecord=12000
gkr.calculation-service.deleteOldReportingPeriod.incrementalDeleteStopAfterPeriod=60m
gkr.calculation-service.deleteOldReportingPeriod.incrementalDeleteStopOnTime=07:00
gkr.accounting-service.deleteOldInvoiceNumberCache.backupTimePeriod=10d
gkr.accounting-service.deleteOldInvoiceNumberCache.processing=true
gkr.accounting-service.deleteOldInvoiceNumberCache.incrementalDeletePoolRange=3000
gkr.accounting-service.deleteOldInvoiceNumberCache.incrementalDeleteStopAfterRecord=12000
gkr.accounting-service.deleteOldInvoiceNumberCache.incrementalDeleteStopAfterPeriod=60m
gkr.accounting-service.deleteOldInvoiceNumberCache.incrementalDeleteStopOnTime=07\:00
# Deletes old temporary report data and report export entries, not report archive
gkr.reporting-service.deleteOldReports.backupTimePeriod=1d
gkr.reporting-service.deleteOldReports.processing=true
gkr.reporting-service.deleteOldReports.incrementalDeletePoolRange=3000
gkr.reporting-service.deleteOldReports.incrementalDeleteStopAfterRecord=-1
gkr.reporting-service.deleteOldReports.incrementalDeleteStopAfterPeriod=60m
gkr.reporting-service.deleteOldReports.incrementalDeleteStopOnTime=07:00