Are the European Commission's forecasts of public finances better than those of national governments?

This paper aims to compare the forecasting performance of the European Union's (EU's) national governments and the European Commission (EC), based on the estimates published in the years 2014–2019 during the convergence programme update. The academic literature frequently emphasizes the supremacy of international institutions in accurate forecasting of the fiscal situation, thanks to its neutrality. Presently, given the recent conflicts between the EC and several governments regarding, for example, the Polish, Hungarian rule of law (Article 7 procedure) or fiscal discipline and migration policy with the former M5S-Lega Nord government of Italy, this argument is debatable.

First, contrary to previous research, we found evidence that EC estimates have similar accuracy compared with national governments in the short-term horizon (i.e. current year). In the case of the next year's forecasts, EC estimates do not strongly differ regarding both revenues and expenditures. However, it has the worse performance to correctly forecast the joint effect of policy decisions – its budget balance estimates are less accurate. Our research suggests that governmental estimates became more professional compared to those in the previous decade (e.g. Brück & Stephan, 2006; Jonung & Larch, 2006; Pina & Venes, 2011).

Second, our research highlights different forecasting accuracies between the countries. The major discrepancies are visible especially in the case of small economies. The weak performance of the EC in forecasting the fiscal situation in such countries creates an incentive for multinational companies and governments to allow for tax evasion. In the case of slightly more populated countries, there is consistent evidence that the estimates of the Danish and Slovakian governments underperform the forecasts of the EC. Furthermore, we do not find significant evidence of fiscal forecasts being used as a soft power tool against governments in conflict with the EC (i.e. Poland or Hungary) – there are no meaningful deviations between the estimates of the organisation and national governments.

Finally, we identified a worse performance of the EC's forecasts in the case of indebted countries, that is, Spain and Portugal. This problem is likely related to the conditions of the Stability and Growth Pact – forecasts may be overly optimistic not to trigger panic regarding the stability of public debt in those countries. Unfortunately, a lenient approach of the EC creates the incentive for national governments to deliberately delay fiscal consolidation, as there will be no punishment for the hazardous behaviour.

This paper is structured as follows: Section 2 presents a literature review on fiscal forecasting, Section 3 discusses the fiscal variables forecasted in the update of the Convergence programme by the national governments and the EC, Section 4 describes our methodology, Section 5 presents the model outcomes and, finally, Section 6 interprets the results and concludes the paper.

The problem regarding the accurate comparison of fiscal forecast between those of the national governments and internationa l organisations came into the literature two decades ago. The most pronounced papers (Brück & Stephan, 2006; Jonung & Larch, 2006) analysed the forecast errors of national governments via fixed effect panel regressions. These authors highlighted two problems: (1) excessive optimism and (2) existence of a so-called electoral cycle, which resulted in bigger forecast errors during the election year. According to the authors both the problems were inevitable – each political faction has strong motivation to depict the success story of its rule and maximise voting outcomes. Therefore, they recommended increase competences of international supervision to shape fiscal policy. Theoretically, such institutions should be free of motivational biases.

The next generation of papers was more critical towards the accuracy of forecasts by international institutions (Pina & Venes, 2011; Merola and Pérez, 2012; Frankel & Schreger, 2013). The authors, basing their comparisons on root mean squared errors (RMSE) and panel models, highlighted that the estimates were also prone to errors typical for national government estimates. However, they also sustained the view that the forecast accuracy of international organisations is better than those of the national governments. Second, research on fiscal policy frequently recommended the introduction of a new generation of fiscal rules to improve the predictability of policies (Schaechter et al., 2012; Schick, 2010).

Introduction of a stricter fiscal framework and reform of the Stability and Growth Pact created new incentives in economics forecasting. Researchers were reporting special country-specific biases related to the use of fiscal rules visible in the case of both national governments’ estimates and forecasts of international organisations (Baldi, 2016; Gilbert & de Jong, 2017; Rybacki, 2019).

A new and relatively uninvestigated phenomenon is the recent clash between the EC and several countries regarding the rule of law (i.e. Hungary and Poland) or fiscal discipline (Italy and Romania). Our study attempts to verify whether such conflicts have an impact on the forecasting performance.

This section presents the datasets used during the research and data transformation. All of the used raw information comes from statistical annexes of the institutional paper series, The YYYY Stability & Convergence Programmes: An Overview and Assessment of the Euro Area Fiscal Stance. YYYY denotes the year of publishing. The series started in 2014, is published up to date (2019) and is an annual publication. The paper series discusses various topics related to the fiscal policy, for example, review of the previous year's performance, medium-term objectives for the stability of public finances and differences between the EC and national government forecasts.

The statistical appendixes publish projections of the following:

Three fiscal indicators: general government balance (surplus or deficit), revenues and expenditures. Each statistic is expressed as a percentage of gross domestic product (% GDP). Our aim is to compare the forecasting performance in the case of each series. All of the mentioned indicators are the most crucial variables for the fiscal policy. The amount of expenditure presented in the budget bill typically sets the limit for a social and healthcare system spending or the government's investments. The projected revenues shape the tax policy – it justifies the introduction of new levies or a change in their rates. Finally, the fiscal balance is a difference between these two variables, which is widely scrutinised by the financial markets and international organisations, such as the International Monetary Fund.

The analysed sample consists of 27 countries – EU members and the United Kingdom. Greece is not represented. During the years 2014–2018, the Greek economy was under the surveillance of the EC, the European Central Bank and the International Monetary Fund. At this time, the Greek government did not publish their forecasts. Regular publications started only in 2019.

The forecasts presented in the paper offer probably the greatest opportunity for a comparison. National governments submit the reform programmes to receive notification from the EC. Therefore, both sides should have similar information sets available during the preparation of forecasts.

Second, the projected variables are equally defined by the ESA 2010 methodology. Some discrepancies may occur if the national government and EC disagree on whether an activity constitutes valid ESA revenue or expenditure. However, to limit the influence of such phenomena we compared the cumulative changes in revenues and expenditure from the starting point of forecast, rather than their levels about the GDP.

National governments may use different approaches to construct their estimates. Yet, our exercise should answer whether unified procedures from the EC can perform better, at least on the average.

The problem of accuracy comparison of fiscal forecasts is not straightforward. Although there exist plenty of recognised methods for testing the forecast accuracy, they apply mainly to the long time series with frequent estimations. Unfortunately, this is not the case for fiscal forecasts – given the fact that estimates are produced annually or semi-annually, the panel contains a high number of cross-sections and a low number of time periods.

Therefore, the application of tools such as Diebold Mariano test (Diebold & Mariano 1995; Harvey et al, 1997; further DM) would end up with problems of estimating forecast variance,

Formally the spectral density of the loss function differential The DM test compares expected value of the difference between loss function applied to the two forecasts (numerator) and its spectral density (denominator). DM statistics follows a standardised normal distribution.

Thus, we decided to use a different approach. We run the fixed effects panel regression, which explains the difference between the squared errors of two forecasts (numerator in the forecast accuracy test). The advantages of such an approach include simplicity and straightforward interpretation. The major drawback is the lack of possibility to present a formal test comparable to DM, with strong theoretical foundations.

Based on such methodology, we attempt to perform two tests, which will allow us to identify whether there exist statistically significant differences between the forecast accuracy of national governments and the EC.

Let us define the following notation:

e_t.h,gov denotes national government's forecast error for the year t with horizon h. For example, e_2005,1,XX denotes the forecast for the year 2005, published 1 year earlier (in 2004) by the government XX.

The forecast errors are described as the difference between the realised value of a variable and its forecasts. The calculation is as follows: (1)et.h=realizationt−forecastt,h{e_{t.h}} = {\rm{realizatio}}{{\rm{n}}_t} - {\rm{forecas}}{{\rm{t}}_{t,h}}

First, we would estimate the panel model, which explains the difference between squared forecast errors of national governments and the EC. The model formula is as follows: (2)et.h,gov2−et.h,EC2=β0+μi+θt+εt{e_{t.h,gov}}^2 - {e_{t.h,EC}}^2 = {\beta _0} + {\mu _i} + {\theta _t} + {\varepsilon _t}

Second, based on the following models, we verify the following hypotheses: 1)

EC forecasts are generally more accurate than national governments. Therefore parameter β₀ should be statistically significant and its values should exceed zero.

This section presents the results of our estimation. The estimated parameters β₀ and the corresponding model outputs are presented in Table 1. Tables 2 and 3 show the numerical values of the estimated period and the cross-sectional fixed effects. The results of redundant fixed effects tests are presented in Table 4.

Contrary to previous research, our study shows no meaningful differences in the forecasts between the national governments and the EC. The Stability and Growth Pact introduced the European Semester, wherein each country provides its fiscal estimates for international supervision. All the government estimates thus became more accessible – and, therefore, the differences between forecasts were easier to compare. This mechanism likely limited the incentives for national governments to present unrealistically optimistic fiscal forecasts.

There is also no convincing evidence of stigmatizing governments, which have an open conflict with the Institution. In such a case, the negative assessment of polices by the EC could be used as a soft power tool to oppress cabinets in conflict. The potential identification of such an issue would indicate that the EU authorities abuse their position in the process of fiscal supervision to influence the political landscape in the country with conflict. The presented methodology does not present evidence of such dishonest behaviour.

However, we identified numerous examples of small economies where the forecasting accuracy of the EC was weak. Although the mentioned problem does not pose a threat to financial the stability of the EU as a whole organism, it may backfire. The weak performance of the EC in forecasting the fiscal situation in such countries creates an incentive for multinational companies and governments to allow for tax evasion. For example, Ireland reported 26.3% GDP growth related to intellectual property in 2016, attracting companies with low corporate tax. The loss of fiscal revenues in the EU was so great that this phenomenon started a debate about whether the activity of digital entities should be taxed in the country of origin (so-called digital tax). Such problems are likely to repeat in the future.

We have also highlighted some evidence that the EC is less accurate in the case of long-term forecast for indebted countries. Again, this phenomenon may be related to motivational problems. Analysts of the EC may be overly optimistic to avoid triggering fears of public debt instability. Yet, such an approach has consequences. It creates incentives for national governments to deliberately delay fiscal consolidation, as there will be no punishment for hazardous behaviour.