Comparison of Block Maxima and Peaks Over Threshold Value-at-Risk models for market risk in various economic conditions

The BM consists in dividing the set of data into M (m = 1, 2, ..., M) time intervals of length n each. Values used for estimation are the minimums or maximums observed in subsequent M time intervals. In other words, if X_1,m, X_2,m, ..., X_n,m is a sequence of independent and identically distributed random variables from a time interval m, the maximum values can be defined as M_m = max(X_1,m, ..., X_n,m). The minimum values can be defined analogously by reversing their sign. To find a non-degenerated cumulative distribution function (cdf), the maximum values M_m are standardised by the scale parameter (variance) σ_m and the expected value μ_m (S_m = (M_m − μ_m) / s_m).

According to the Fisher and Tippett (1928), if such a non-degenerate cdf exists (FM), it must belong to one of the Gumbel, the Frechet or the Weibull distributions. Those distributions can be interpreted as special cases of generalised extreme value (GEV) distribution. The cdf of this distribution is defined as follows (with a shape parameter x): (2)GEV(x;ξ,μ,σ)={exp[−(1+ξ(x−μσ))−1ξ]if ξ≠0 and (1+ξ(x−μ)σ)>0exp[−exp(−(x−μσ))]if ξ=0{\rm{GEV}}(x;\xi ,\mu ,\sigma ) = \left\{ {\eqalign{{\exp \left[ { - {{\left( {1 + \xi \left( {{{x - \mu } \over \sigma }} \right)} \right)}^{ - {1 \over \xi }}}} \right]} \hfill & {if\;\xi \ne 0\;{\rm and}\;\left( {1 + {{\xi (x - \mu )} \over \sigma }} \right) > 0} \hfill \cr {\exp \left[ { - \exp \left( { - \left( {{{x - \mu } \over \sigma }} \right)} \right)} \right]} \hfill & {if\;\xi = 0} } } \right. The x sign determines which of the distributions has been selected. The Gumbel, Frechet or Weibull distribution is assumed for x = 0, x > 0 and x < 0, respectively (Da Silva & de Melo Mendes, 2003). In the research, a method of the maximum likelihood was used to estimate the distribution parameters.

From the GEV distribution, a VaR can be estimated as follows: (3)VaR(α)={μ^n−σ^nξ^n(1−(−nln(α))−ξ^n)to ξ>0 (Fréchet)μ^n−σ^n ln(−nln(α))to ξ=0 (Gumbel)VaR(\alpha ) = \left\{ {\eqalign{{{{\hat \mu }_n} - {{{{\hat \sigma }_n}} \over {{{\hat \xi }_n}}}\left( {1 - \left( { - nln (\alpha )} \right)^{- {{\hat \xi }_n}}} \right)} \hfill & {to\;\xi > 0\;\left( {Fr{\acute e}chet} \right)} \hfill \cr {{{\hat \mu }_n} - {{\hat \sigma }_n}\;ln \left( { - nln (\alpha )} \right)} \hfill & {to\;\xi = 0\;\left( {Gumbel} \right)} } } \right. where VaR(α) is the measure of the Value at Risk at the significance level α, μ^\hat \mu is the estimated location parameter, σ^\hat \sigma is the estimated scale parameter and ξ^\hat \xi is the estimated shape parameter.

The POT method distinguishes two approaches: the approach based on the Hill estimator and the approach based on the assumption that the tail of the return rate distribution is derived from the generalised Pareto distribution (GPD). In the following research, the method based on the GPD (Balkema & de Haan, 1974; Pickands, 1975) will be used and described. The GPD cumulative distribution function is as follows: (4)GPD(x;ξ,μ,σ)={1−(1+ξx−μσ)−1ξξ≠01−exp(−x−μσ)ξ=0{\rm{GPD}}\left( {x;\xi ,\mu ,\sigma } \right) = \left\{ {\eqalign{{1 - {{\left( {1 + \xi {{x - \mu } \over \sigma }} \right)}^{-{1 \over \xi }}}} \hfill & {\xi \ne 0} \hfill \cr {1 - \exp \left( { - {{x - \mu } \over \sigma }} \right)} \hfill & {\xi = 0} } } \right. where m is the location parameter, s is the scale parameter and x is the shape parameter.

In this case, a series of random variables X₁, X₂, ..., Xc (i.i.d.) and certain threshold level u are considered. Assuming that right tail of a distribution is of interest, for all realisations xi above the threshold u the values of exceedances y₁, y₂, ¼, yn are calculated (y_i = x_i − u). The distribution of exceedances above the u threshold is defined as: (5)Fu(x;u)=P(X=u+y|X>u)=F(y+u)−F(u)1−F(u){F_u}(x;u) = P\left( {X = u + y|X > u} \right) = {{F\left( {y + u} \right) - F(u)} \over {1 - F(u)}}

Assuming that for a certain threshold u the distribution of observations being above the threshold is the GPD (y;σ,μ,ξ), the tail of the distribution of the return rates above the assumed cut-off point can be written as follows: (6)F(x)=F(y+u)=[1−F(u)] GPD(y;σ,μ,ξ)+F(u)F(x) = F\left( {y + u} \right) = \left[ {1 - F\left( u \right)} \right]\;{\rm{GPD}}\left( {y;\sigma ,\mu ,\xi } \right) + F\left( u \right) where F (.) is a cumulative distribution function, u is the cut-off threshold, y is a loss level above the cut-off threshold u and G_ξ,σ (y) is the cumulative distribution function value of the GPD.

Value at Risk at the a level is calculated from the following formula: (7)VaR(α)=u+σ^ξ^[[nNu(1−α)]−ξ^−1]VaR(\alpha ) = u + {{\hat \sigma } \over {\hat \xi }}\left[ {{{\left[ {{n \over {{N_u}}}\left( {1 - \alpha } \right)} \right]}^{ - \hat \xi }} - 1} \right] where VaR(α) is Value at Risk at the significance level a, u is the cut-off threshold, ξ^\hat \xi , σ^\hat \sigma are GPD parameters, n is the total number of the analysed return rates, N_u is the number of return rates below the cut-off point u.

Also, in this case, the maximum likelihood method was used to estimate the parameters. A mean excess plot was used for determining the cut-off threshold u (Embrechts, Kluppelberg, & Mikosch, 1997).

In the study, the Value at Risk forecasts were compared on the basis of the number of losses exceeding the estimated VaR, traffic light backtest, Kupiec test (1995), backtesting criterion statistics (Abad et al., 2013), Christoffersen test (1998) and the cost functions: the absolute cost function of Abad and Benito (Abad et al., 2013), the Caporin function (2008) measuring the absolute cost of the forecast and the function of the excessive cost (see Chlebus, 2014).

The Kupiec test compares the expected and observed share of exceeded VaR forecasts, and the zero hypothesis is the equality between the expected and observed share of exceedances. This test, however, is not able to determine the direction of error (overestimation or underestimation). Backtesting criterion statistics allows verification of the error direction. Strongly negative values suggest an overestimation of VaR forecasts, while positive ones indicate an underestimation of forecasts. Christoffersen test extends the Kupiec test with the test of the independence of exceedances – when the intervals between exceedances are too small, it means that the model incorrectly estimates the risk during volatility clustering periods.

The applied cost functions do not have the character of a formal test. Lopez (1999) described square cost function which increases the weight of severe exceedances, but hinders the interpretation of the results. The cost function of Abad and Benito is more straightforward in interpretation as it takes into account the absolute value of the difference between forecasts and real observed values when exceedance occurred. The sum of these differences divided by the number of periods at the observed VaR exceedance is the average severity of the exceedance per VaR forecast. In this method, the average is taken into account, and not the sum of the severity of the exceedances, so that this measure does not take into account the number of exceedances. The Caporin function takes into account both the exceedances and the underestimation of the forecast. The result of this function is the mean absolute error of the VaR forecast. In the work a function similar to the Caporin function is also used, but distinguishing the measurement of effectiveness in three variants: in the case of exceeding the VaR forecast by the observed return rate, in the case when the VaR forecast is smaller than the observed return rate and at the same time the observed return rate is higher than zero and in the case when the VaR forecast is lower than the observed return rate but the observed return rate is smaller than zero. The average value of the excessive cost function is used to compare the models. The higher the result of the mean value of the excessive cost function, the more conservative the model, that is, VaR forecasts are too high in relation to the needs related to the coverage of possible losses.

Summing up, the Kupiec test, backtesting criterion and Christoffersen test are used to examine whether the forecasts perform correspondingly to Basel postulates. The loss functions, such as the absolute cost function of Abad and Benito, the Caporin function and the function of the excessive cost, check if the costs of using such techniques are economically reasoned for financial institution.

The first discussed out-of-sample period is the pre-crisis period that is between 01 January 2007 and 31 December 2007. The results are presented in Table 2. Forecasts for all indices, with the exception of the Lithuanian one, can be considered as precise based on the results of the Basel traffic light test. Only the OMXV index has semi-precise forecasts for the POT method. Similar conclusions can be drawn from the analysis of unconditional coverage tests (Kupiec and Backtesting criterion statistics) and conditional coverage test (Christoffersen). Based on the results obtained from them, it can be noticed that for all three tests the null hypothesis is simultaneously rejected only for the Lithuanian index in the POT method – when the number of exceedances is relatively large. The Kupiec test indicates that the total absence of exceedances should also be considered as a statistical discrepancy between the expected and observed share of exceedances in most of the cases – null hypothesis should not be rejected in 5 of 30 discussed time series.

The results of the Abad and Benito functions indicate that even though the exceedances occur, their magnitude is rather not severe. The Caporin and CAE functions describe the average absolute error in relation to the observed return rate. The results in each country are similar and it is noticeable that both BM₂₁ and BM₄₂ methods are more conservative than POT method and therefore they can be considered to be more expensive for financial institutions to implement.

The obtained results indicate that the unconditional EVT models for the pre-crisis period allow to obtain satisfying VaR forecasts in terms of their quality but it may involve higher costs of applying the selected models. No quintessential differences were observed between individual countries within emerging and developing groups. The distinction between both groups is also rather imperceptible based on the results shown in Table 2.

In the second period, one can observe a clear deterioration of results. The models are not able to quickly adjust data to the observed drops in the market. Satisfactory results are obtained only by the Slovak market for each method and by all indices using the BM methods, excluding Czech and Hungarian markets. The rest of the results, according to the Basel recommendations, can be classified into semi-precise or imprecise. The Kupiec test indicates that the hypotheses about the correctness of the forecast should be rejected in the Slovakian and German markets for the BM method with both 1- and 2-month blocks and also for all POT cases. The test statistics of the Backtesting criterion statistics is definitely positive in most of the POT cases, which confirms the underestimation of the forecasts.

The cost function of Abad and Benito indicates differences between the return rates and the VaR forecasts at the point of exceedance especially for POT method. The Caporin and CAE functions are similar to the results obtained from Table 2. They suggest an overestimation of the forecast in moments of calm, and underestimation of forecasts in moments of intensified market movements. The Caporin function most often indicates the smallest mean absolute error of the VaR forecast for POT methods.

During the crisis period, the unconditional EVT models fail. On the one hand, both BM methods are able to limit number of exceedances, but on the other it is highly uneconomic from bank's perspective. POT method leads to significant underestimation of VaR due to the slow adaptation to new market conditions.

In this case, differences between countries are noticeable. The Czech Republic, Slovakia and Hungary are characterised with similar VaR levels, that is, around 7%–8% for BM methods and around 3.5% for POT method. The similarity between these indices can be explained with the fact that these three countries are neighbours and their trading connections are strongly related. This observation could lead to expansion of the research on VaR forecasts, using additional information of connections and correlations between two or more examined parties.

Another observed remark is connected with Table 1. Even though the capitalisation between emerging and developed countries is clearly different, Polish market distinguishes in its group and in terms of capitalisation it is the closest to developed countries group. According to the results from Table 3, the similarities are also noticeable. Number of exceedances is especially similar to English and French market. Simple mean average for CAE function in developed countries is 0.036 for BM₂₁, 0.042 for BM₄₂ and 0.021 for POT methods. These scores are prominently close to the scores obtained by the Polish market.

The results from the period III are presented in Table 4. They are clearly improved as every forecast can be considered to be precise recording not more than four exceedances, despite of still increased volatility on the markets in 2009. Therefore, all the methods can be considered as equally good in terms of its conservativeness. On the ground of cost functions, the POT method is again less demanding than both BM methods as Caporin and CAE functions are often two or three times smaller for POT method. Results obtained with those functions are however still significantly higher than in periods I and II which stem from extreme observations included in the analysis of period III.

Similarly to the period I, no evident differences were observed between individual countries within emerging and developing groups. The distinction between both groups is also rather imperceptible. It stems from the fact that the volatile data from previous period were used to forecast VaR and now, in the calmer period, there is rather small chance to extend the VaR level for each index.

The results obtained in period III can be summarised noting that in the post-crisis period the number of exceedances was decidedly limited compared with the crisis period. However, the VaR estimations can also be considered as too high to conclude that the obtained forecasts are of high quality in each of the considered cases.