Displaying Backtesting Results (Results Database) 

Use

The Analyzer Information System contains functions for reporting backtesting results using the results database.

The following analyses are available:

·       Traffic light display for the requirements from the Basel Committee

·       Tables containing the results of calculations

·       Graphs showing the value at risk, and the values of the profit and loss key figures required for purposes of comparison

·       Q-Q plot for generating graphs to show the extent to which key figures deviate from their average values

·       P-P plot to check to what extent a profit and loss key figure is evenly distributed

In the analysis functions, you can navigate in the portfolio hierarchy and risk hierarchy, which enables you to display the results at different aggregation levels, and for different risk factors.

Prerequisites

In Customizing for market risk analysis, which is under SAP Banking ® SEM Banking ® Market Risk Analysis ® Results Database ® Define Initial Layout, you need to have stored the following information:

·       One value-at-risk key figure, and at least one profit and loss key figure that you want the system to display.

·       A profit and loss key figure that you want the system to compare with the value at risk.

·       Size of the sample for backtesting

You also need to have carried out the relevant valuation runs for value at risk and the profit and loss key figures.

Procedure

...

      1.      In the SAP Easy Access screen, choose Accounting ® Bank Applications ® SEM Banking ® Market Risk Analysis ® Information System ® Analyzer Information System.

The system displays a selection screen.

      2.      Specify the view, portfolio hierarchy, and the key dates for the data that you want to display, and choose an initial layout for the display of the results.

      3.      Choose Execute.

The portfolio hierarchy is shown in the top part of the screen, and risk hierarchy is shown in the lower part of the screen.

To analyze the backtesting results, you first select a node in the portfolio hierarchy, and then a node in the risk hierarchy.

Choosing a Node in the Portfolio Hierarchy

There are two ways in which you can choose a node in the portfolio hierarchy:

·       Choose Determine Outliers.

The system opens a dialog box that shows the section of the portfolio hierarchy that contains the outlier. Any portfolio hierarchy nodes that contain outliers are shown by a red indicator.

To analyze an outlier, highlight the respective portfolio hierarchy node by choosing the relevant field in the Outliers column, and then choose Display Details.

·       Open the portfolio hierarchy down to the level of the required node. Then choose the node by double clicking the field containing the key figure.

The system highlights the portfolio hierarchy node in the upper part of the screen, and selects its backtesting data.

Choosing a Node in the Risk Hierarchy

...

      1.      Choose the Backtesting tab page.

On the left, the system displays a navigation structure containing the risk hierarchy.

      2.      To display the backtesting results, double click the node in the risk hierarchy.

The system highlights the risk hierarchy node, and displays the associate backtesting results in the right-hand part of the screen next to the navigation structure.

Reporting

      3.      The system displays the backtesting results on the Backtesting tab page. This tab page is divided into the following areas:

·       Table Containing Backtesting Results

For each maturity band, the system compares the value-at-risk key figures with the profit or loss that was realized. In the right-hand column, the system displays the difference from backtesting, which is the difference between the estimated loss from the value-at-risk analysis, and the key figure defined in Customizing for comparison with the actual profit or loss.

If the actual loss is larger than the value at risk, the backtesting different is negative. Negative differences from backtesting are shown in red.

·       Summary of Backtesting Results

Field	Description
Number of Outliers	Number of key dates on which the backtesting key figure is negative The system shows the number of outliers in green, yellow, or red, depending on the number of outliers and as per the Basel Committee regulations.
Number of times limit was exceeded positively	Number of key dates on which the profit is larger than the value at risk
Number of Losses	Number of key dates on which the key figure for the comparison for profits and losses is negative
Size of Sample	Number of key dates that have been included in the analysis
Holding Period	The holding period specified in the value-at-risk calculation
Confidence Level (Plan) in Percentage	Range for the confidence level, which is stored in the value-at-risk key figure
Confidence Level (Actual) in Percentage	Confidence level P calculated from the set of values that were determined: where x is the number of outliers, and N the scope of the sample.
Value at Risk on key date
Green, yellow, and red traffic lights	Number of outliers in the green, yellow or red areas, as defined by the Basel Committee. In a sample of 250, the ranges are defined as follows: ·       Green: up to 4 outliers ·       Yellow: 5 to 9 outliers ·       Red: more than 9 outliers If the sample is not 250, the system calculates the ranges for the traffic lights as per the Basel regulations.

 

·       Distribution of Value at Risk and Profits and Losses

The system shows the profits and losses as bars on the time axis. This graph is based on the the key figure that you defined for comparing the profit and loss. The outliers are shown in red. In order to make it easier to compare the profit and loss with the value at risk, the system also shows the data for the value at risk.

To display the value at risk and the profit and loss key figure for a key date, double click a bar in the diagram. In the status bar, the system displays the relevant key date with the values for the value at risk and the profit and loss key figure.

·       Q-Q Plot

For each key date, the system calculates the extent to which the value-at-risk key figure deviates from its average, and the extent to which the profit and loss key figure deviates from its average. It then sorts the values of the key figures by size (separately for the profit and loss key figure, and for the value-at-risk key figure). The system enters the values in the sorted lists accordingly, so that they can be compared.

The deviations are measured as follows:

,

where n is the scope of the sample, and σ is the standard deviation.

The graph shows how close the distribution of the forecast VaR values is to the distribution of the actual profit and loss values. To make the analysis easier, the system displays a line from the origin with a slope of 1. The actual distribution is equal to this line if the deviations of the value at risk and the profit and loss key figure are identical.

To display the value at risk and the profit and loss key figure for a key date, double click a bar in the diagram. In the status bar, the system displays the relevant key date with the values for the value at risk and the profit and loss key figure.

·       P-P Plot

The system places the values of the profit and loss key figure in 100 equal quantiles. In the graph, the system shows how frequently the values for the profit and loss key figures occur in the respective quantiles. It also shows the respective cumulative frequency across the quantiles.

To make the analysis easier, the system displays a line from the origin with a slope of 1. If the profit and loss key figure is equally distributed, the graph of the actual distribution has this form.

To display a pair of values of the actual distribution, double click on a point in the graph. In the status bar, the system displays the relevant quantile and cumulated frequency.

You can change the size of the areas in the tab page by using Drag&Drop to move the gray borders.