Thesis: Given the Michigan survey of inflation expectations and the Survey of Professional Forecasters, I test the forecast ability of the surveys to predict inflation.
Since the 1970s, macroeconomists had to change the way they do economic analysis. Thanks to Robert Lucas, he showed that one could not predict the effects of a change in macro policy, given only historical data. Economists then understood that expectations, had to be a contributing factor to conducting better monetary policy. Not long after, the Michigan survey of consumer sentiment was born. Here I consider the part of the survey pertaining to inflation expectations, where participants give input on their predictions for the inflation rate for the following year. In essence, I test the forecast ability of the survey to predict “inflation” against other competing models, such as the (SPF) survey of professional forecasters (which began three years after the Michigan survey), and a random-walk no change forecast as a benchmark model.
In a recent blog, I do a similar process in testing the predictive power of futures contracts against a no-change model and a percent spread (future spot price and today’s spot price) model. Although we were able to conclude that futures contracts had the most predictive power, I did not specifically test (nor would I have been able to) if the superiority of the futures relative to the other models was statistically significant. Luckily this is precisely what we can do for testing the forecast ability of the inflation surveys, and the no change model above. (Note that doing this kind of test is very special, given that usually we are only able to do it with survey data).
So we want to test whether the Michigan survey is a good predictor of today’s inflation, and whether or not it is better than the SPF and the no-change. However the measure of inflation is undefined. Survey participants may have an idea of what kind of price changes they are predicting, but we cannot know for sure, because there are so many measures of inflation. Therefore I do a quick analysis of what inflation measure survey participants are predicting. I calculate the PMSE‘s below:
After testing for 5 different measures of inflation, choosing the measure that minimizes the PMSE, it seems clear that both the Michigan Survey and the SPF are forecasting the Median CPI. Also plotting a graph of the Michigan Survey and the SPF vs the Median CPI, we see that both surveys do a reasonably good job of predicting inflation. (Note: I have double checked, and made sure that I plotted the inflation expectations with the corresponding CPI, 12 months ahead).
We can look at this graph and the PMSE from the table above that the SPF may do a slightly better job of predicting the inflation rate, defined by the median CPI. Formally this requires a Diebold-Mariano (DM) test, where I test the null hypothesis that both models have equal predictive power, against the alternative that the SPF is superior. I use the following equation, where the DM statistic follows a standard normal distribution and the values u0t, and u1t, represent
sequences of squared prediction errors, corresponding to the two surveys we are trying to test. Intuitively the DM test, is almost like a t-test, except we are testing whether or not the PMSEs of the survey forecasts are equal.
We arrive at the following conclusion:
The DM test of SPF vs Michigan Survey, gives a p-value of .3783, and the corresponding p-values for the Michigan Survey and SPF relative to the No-change forecast are .4032 and .4468 respectively. What we conclude for now is that the predictive power of the all the forecasts compared to each other is not statistically significant. What is nice though, is that we were able to conclude with a formal test that we cannot say the SPF is better than the Michigan Survey beyond reasonable doubt, which means there is some merit to the Michigan Survey. This leads me to the next step, that it may be possible to construct a model of survey participants from the Michigan survey that could be superior to the SPF and statistically significant with a DM test. This would lead us to consider demographics of participants such as income, education, and age, and see whether these have substantial effects on people’s ability to predict inflation. I shall consider this question in a future blog post.
(Matlab Code and Data)