Backtesting 
Use
Backtesting is the process in which value-at-risk values are checked ex post with the actual changes in the value of the portfolio. Backtesting checks how accurately the value-at-risk calculation predicted the actual risk.
· The Basel Committee on Banking Supervision requires banks to carry out an ex post comparison for the in-house models that they use to calculate the capital requirement. In this process, the exceptions (realized gain or loss > VaR) from the last 250 trading days (holding period of 1 day, 99% confidence level) are added together and compared with the forecasts.
The Basel backtesting procedure enables you to assess in-house models using a traffic light display that is based on the number of outliers:
Traffic Light Display |
Description |
|
The number outliers is less than or equal to 4. The model is considered accurate by the banking supervisory body. |
|
The number of outliers is less than or equal to 9. The bank can continue to use the model if they use a higher scaling factor. |
|
The number of outliers is greater than 10. The scaling factor is 4 in this instance and the banking supervisory body can refuse the use of the bank's in-house model for the calculation of the capital requirement. |
Other methods, such as Q-Q plots and P-P plots, check the distribution of the forecasts.
Integration
· Displaying Backtesting Results (Results Database)
You can use the system to carry out backtesting for results data as per the Basel Committee's requirements. It also contains additional functions in the results database, such as Q-Q plots and P-P plots.
· Support for Backtesting (Drilldown Reporting)
· The system still contains functions the functions for backtesting from previous releases. However, these have been superceded by the functions that are part the results database:
¡ Calculation of profit or loss in drilldown reporting (values are not shown as a time series)
¡ Calculation of the NPV for saved datasets
Prerequisites
Backtesting requires that the ex post check accesses the same transaction data as that used for the value-at-risk forecast. This means that any changes to the portfolio that have taken place in the meantime must not be included in backtesting.
Since historical transaction data is not stored in SEM Banking, the system contains a function for saving, at any point in time, the data that is used in an analysis. This data is referred to as asaved dataset, and is stored in the database. In order that the backtesting functions can be used, a saved dataset must exist, regardless of whether the analysis involves the results database or drilldown reporting.
To generate saved datasets, in the SAP Easy Access screen, choose Accounting ® Bank Applications ® SEM Banking ® Market Risk Analysis ® Tools®Additional Functions for Backtesting ® Generate New Dataset.
The system saves the transaction data as risk objects for the selection date specified, and on the basis of a view and a portfolio hierarchy. It assigns the dataset an identification number, which is used in the backtesting function to access this dataset.
Future-style transactions are saved and analyzed differently from other types of transaction. When the NPV is being calculated, futures have, by definition, an NPV of zero. However, during backtesting, they may well have an NPV risk. For these reasons, when futures are saved at time point t0, the system saves the current price at the same time. When the transaction is analyzed at time point t1, the price valid at that time is determined, and the future (or option on the future) is valued using the price difference.
You can also archive datasets. When datasets are archived, they are stored both on the database and in the SAP Archive until they can be deleted explicitly from the database.
See also: