Specifics of sources for Russian macroeconomic forecasts

¹Sources are specified that have published forecasts in table format on their websites at least once a year over the past three years. Consequently, Table 1 does not include the Accounts Chamber of the Russian Federation, Expert RA or the Center for Macroeconomic Analysis and Short-Term Forecasting, which either started publishing forecasts recently or publish them at intervals of more than a year. A group of indicators is shown is present in a forecast if at least one of the indicators of the group has been disclosed over the past three years.

ACRA talks in detail about the specifics of constructing and applying macroeconomic forecasts at the following workshops: Forecasting in credit analysis. Course 1: macroeconomic and industry forecasting fundamentals and Forecasting in credit analysis. Course 2: practical aspects of economic modeling.

Most of the publicly available macroeconomic forecasts for Russia include at least one indicator that characterizes the level of economic activity (GDP growth rate, industrial production, etc.) and inflation. The other sets of indicators are different. In almost all cases, forecasts provide annual data, and the forecast horizon is usually less than five years (most often, it is two to four years). The majority of forecasts are updated twice a year or more often. Scenarios other than the baseline are very rarely provided, and pessimistic scenarios are designed to take into account the uncertainty in preconditions rather than the likelihood of a full-blown crisis (i.e. they usually do not provide an opportunity to conduct stress testing).

The only way to determine the quality of a macro forecast’s source and the degree of confidence in it is by looking at the history of previous forecasts. Besides historical accuracy, frequency of updates and the selection of indicators, sources of forecasts also differ in terms of other characteristics (including according to internal ideas about the purposes of forecasting; more on this below), and the indicators themselves are predictable to varying degrees.

Table 2. Examples of using macroeconomic indicators when forecasting the performance of sectors or companies

Source: ACRA

Systemic bias in forecasters’ expectations

The preconditions and methods which the analyst choses when forecasting are influenced not only by market incentives or the specifics of the organization in which the analyst works, but also by his or her psychological characteristics. In a number of cases, this leads to the fact that for some indicators, the forecast steadily shifts either in the positive or negative direction. During this study, we will conditionally identify stable “optimists and “pessimists” among forecasters and explain why this factor must be taken into account when using their forecasts for practical purposes.

MED – Ministry of Economic Development of Russia

Bias of expectations relative to the fact (absolute bias). The MED’s forecasts of the growth of real GDP and household income can be seen as typical examples of this sort of bias as they are historically overestimated over a forecast horizon of more than two years (Fig. 1). This may be largely related to the fact that these indicators are targets for the MED, and therefore, assumptions about the effectiveness of stimulus or structural measures cannot be too conservative and should imply a certain improvement in dynamics. Other analysts’ forecasts for similar horizons are usually not biased or biased to a lesser extent. This observation does not at all mean that all forecasts of the MED are biased relative to the fact (for example, in terms of the nominal value of retail turnover at all forecast horizons, on average, deviations from the fact are historically close to zero), nor does it mean that all conditionally target indicators of all forecasters will definitely be subject to bias.

Figure 1. Bias of the forecasts of the Russian Ministry of Economic Development for economic growth rates and real disposable income of the population, 2005–2020*

* In the right graph, the dotted lines represent the average deviations from the fact, excluding forecasts for crisis periods (2009, 2015, and 2020).
Sources: Russian Ministry of Economic Development, ACRA

Inflation targeted by the Bank of Russia is also an important target indicator. Over the period from 2015 (when the inflation-targeting regime formally began to operate) to 2020, deviations of the medium-term inflation forecast at different time intervals from the actual indicator were both positive and negative. However, in general the Bank of Russia’s forecasts are more conservative than they are optimistic: the middle of the range of forecasts of the Bank of Russia for the next year is on average 0.36 pps higher than actual inflation (calculated December to December). At the same time, unlike the example with the growth rates of real GDP and real disposable income of the population, in this case, the bias can hardly be called systemic, since the width of the interval given in the forecast was up to 0.5 pps, and the historical spread of deviations from the fact considerably exceeded 0.36 pps (Fig. 2, left graph).

The balance of the federal budget is a kind of cross between a plan and a forecast for the Ministry of Finance of Russia as it is determined by target expenditures and projected revenues. From 2005 to 2020, in the December version of the budget for the following year, revenues and the federal budget balance were more frequently underestimated than overestimated. Consequently, the established practice of short-term planning can be called systemically conservative. This is not surprising as an unexpected sequestration is associated with significantly higher costs for the ministry than additional revenues at the end of the year (Fig. 2, right graph).

Figure 2. Deviations of forecasts for inflation and parameters of the federal budget from the actual value of the following year

* In the right graph, the deviation from the actual value was calculated in nominal monetary terms. Conversion to GDP shares is done using actual nominal GDP (actual estimate).
Sources: Bank of Russia, Russian Ministry of Finance, ACRA

Over these years, analysts from about 60 Russian and foreign organizations have participated in the survey, with more than 30 of them being permanent responders. These organizations include, in approximately equal proportions, commercial banks, investment companies, and think tanks. The survey results are not public and available only to the survey company's members and clients.

Forecast biases are not only related to the targets of their authors. Some analysts may encounter the impact of the systematic revision of statistics by Rosstat, which may lead to conservative short-term forecasts of economic activity. More details about this effect can be found in ACRA’s analytical commentary “A significant positive revision of industry data is the norm for Russia and many other countries” dated October 20, 2020.

With the exception of the examples listed above, the bias of the predictions relative to the fact is, as a rule, not absolute, but relative.

Bias relative to the consensus (relative bias). Most public and non-public sources of forecasts for a particular indicator do not demonstrate consistently positive or negative deviations from the fact; however, they are often persistently “pessimistic” or “optimistic” in comparison with other forecasters.

The monthly consensus surveys of FocusEconomics for the period from 2017 to 2021 reveal the following patterns in the forecasts concerning Russian macroeconomic indicators (Fig. 3):

- In more than 80% of surveys, analysts from investment companies are, on average, more optimistic than others in their expectations for economic growth and the federal budget balance in the next year.

- In more than 85% of surveys, analysts from commercial banks are, on average, more pessimistic than others in their expectations for economic growth in the next year.

- In more than 80% of surveys, sectoral consensus forecasts prepared by think tanks contain a lower key interest rate and weaker budget balance than the consensus forecast of all survey participants.

Figure 3. Share of surveys that showed more “optimistic” consensus forecast that the consensus forecast of all survey participants (2017–2021, forecast for the next year)*

* Explanations for the chart: the left column should be interpreted so that in 43 out of 50 surveys (86% of cases), the sectoral consensus forecast of investment companies for expected GDP growth next year was more optimistic than the consensus forecast of all survey participants; the second column means that in eight out of 50 surveys (16% of cases), the sectoral consensus forecast of commercial banks for the same indicator was more optimistic than the consensus forecast of all survey participants (and more pessimistic in 84% of cases, respectively).
Sources: FocusEconomics, ACRA

The sign of deviation of forecasts from the consensus that corresponds to the “optimistic” approach is as follows: GDP growth - plus, ruble exchange rate - upward, inflation - minus, key rate - plus, budget balance - plus.

Surveys conducted in other countries often have similar results, but the question of whether the forecaster’s industry influences the bias of its expectations against the consensus forecast remains uncertain in Russia. For a more complete understanding of the situation, the analyzed time interval should include multiple cycles of growth/decline in indicators and increase/decrease in uncertainty, and a deeper analysis of the characteristics of survey participants is also required.

A practically useful conclusion is that, regardless of the sector of forecasters, there seem to be persistent “pessimists” and “optimists” among them, and there are quite a few of them. For example, in the aforementioned consensus survey, the forecasts of 25–40% of analysts for each of the indicators are consistently above or consistently below the consensus most of the time (>80% of periods). Moreover, the degree of an analyst’s “optimism” in one indicator correlates with that in all other indicators, which theoretically makes it possible to present the degree of “optimism” as a specific feature of a forecaster that is independent of the selected indicator.

The good news is that, with rare exceptions, public forecasters’ forecasts (outside of consensus surveys) for the next year do not have a systemic bias relative to the consensus; and they almost never represent extreme viewpoints (Appendix 2).

Those analysts who are users of external forecasts should be aware of potential systematic biases in outlooks and adjust macroeconomic prerequisites expertly or choose a forecast source that allows them to obtain the required degree of conservatism in estimates.

Predictability of selected macroeconomic indicators

A longer history of forecasts for less numerous sources is shown in Table 4.

Even if forecasts contain no systematic bias relative to the fact, that is, the average deviation of a forecast for any indicator over several years is close to zero, each individual period is predicted with varying accuracy. Macroeconomic indicators are characterized by differing volatilities, which, as a rule, determines the level of their predictability: the distribution of forecast biases for each indicator depends more on the indicator itself than on the forecast source. This is confirmed, for example, by the closeness of the mean and maximum deviation moduli for different public forecasters (Table 3).

Tables 3 and 4 should not be taken as a comparison of the forecast accuracy of certain forecasters, since (1) such a comparison is highly dependent on the chosen accuracy metrics and time period, and, as shown on pages 4–5, (2) there are additional goals set by forecasters from different organizations and internal quality criteria. Rather, these tables reflect the degree of uncertainty faced by all macroeconomic forecasters in Russia, and, as a consequence, users of their forecasts.

Table 3. Deviation distribution characteristics of forecasts for certain indicators, 2017–2020, forecast for next calendar year

* Deviations of indicators measured as percentage are calculated as a simple difference between the factual and forecasted values (absolute difference, A). For other indicators, the relative difference is calculated as a percentage of the factual value (R).
** The “Official sources” column shows the range of deviations for the entire set of Russian ministries and agencies. For certain indicators, it includes the Ministry of Economic Development, the Ministry of Finance, and the Bank of Russia.
Sources: organizations’ publications, Rosstat, ACRA

The mean deviation moduli given in the tables allow the forecast user to understand the level of uncertainty of the macroeconomic assumptions used by them and, for example, to build confidence intervals for indicators depending on these assumptions. In addition, these numbers can serve as a reference point for analysts who build predictive models for such indicators. To be considered applicable to real problems, the model must show at least comparable characteristics of deviations from the fact in out-of-sample forecasts for one year ahead. Table 3 shows, in a sense, a typical period (four-year period with one crisis year), while Table 4 characterizes the period with as many as two crises. The set of organizations in Table 4 is significantly smaller, which is primarily due to the availability of a long history of forecasts.

Table 4. Deviation distribution characteristics of forecasts for certain indicators, 2012–2020, forecast for next calendar year

Indicator	Calculation type*	Mean deviation modulus		Maximum deviation modulus		Average deviation (with sign)
				MED**	IMF	MED	IMF	MED	IMF
Real GDP growth rate, %	A	2.1	2.2	5.0	4.9	1.4	1.1
Real disposable income growth rate, %	R	2.7	-	5.9	-	2.6	-
Real industrial production growth rate, %	A	2.0	-	5.2	-	0.7	-
Retail trade turnover, RUB bln	R	2.3	-	6.8	-	1.7	Print version Download PDF Analysts Dmitry Kulikov Senior Director, Sovereign and Regional Ratings Group +7 (495) 139 04 80, ext. 122 dmitry.kulikov@acra-ratings.ru Comment Last name Required field Required field First name Required field Required field E-mail Enter your e-mail in the format user@site.com Required field Phone Enter your phone number, at least 7 characters Required field Place of work Required field Required field Position Required field Required field Message Required field Required field Add file By pressing Send you confirm that you have reviewed and agree to the User agreement Reset Send We protect the personal data of users and process cookies only to personalize services. You can prevent the processing of cookies in your browser settings. Please read the terms of use of cookies on this website by clicking on more information. Accept SUBMIT FEEDBACK Last name Required field Required field First name Required field Required field E-mail Enter your e-mail in the format user@site.com Required field Phone Enter your phone number, at least 7 characters Required field Feedback type E-mail Phone Does not require treatment Required field Request type Complaint Appeal Corruption and fraud Countering UUOII Required field Message Required field Required field By pressing Sign up you confirm that you have reviewed and agree to the User agreement Reset Send SERVICE PROVISION REQUEST Last name Required field Required field First name Required field Required field Place of work Required field Required field Position Required field Required field E-mail Enter your e-mail in the format user@site.com Required field Phone Enter your phone number, at least 7 characters Required field TIN Required field Required field Service type Rating services Research Automatic data downloads Sustainable development services Trainings Other Required field By pressing Send you confirm that you have reviewed and agree to the User agreement Reset Send