Working Paper: CEPR ID: DP15217
Authors: Florens Odendahl; Barbara Rossi; Tatevik Sekhposyan
Abstract: We propose a novel forecast comparison methodology to evaluate models’ relative forecasting performance when the latter is a state-dependent function of economic variables. In our bench¬mark case, the relative forecasting performance, measured by the forecast loss differential, is modeled via a threshold model. Importantly, we allow the threshold that triggers the switch from one state to the next to be unknown, leading to a non-standard test statistic due to the presence of a nuisance parameter. Existing tests either assume a constant out-of-sample forecast performance or use non-parametric techniques robust to time-variation; consequently, they may lack power against state-dependent predictability. Importantly, our approach is applicable to point forecasts as well as predictive densities. Monte Carlo results suggest that our proposed test statistics perform well in finite samples and have better power than existing tests in selecting the best forecasting model in the presence of state dependence. Our test statistics uncover “pockets of predictability” in U.S. equity premia forecasts; the pockets are a state-dependent function of stock market volatility. Models using economic predictors perform significantly worse than a simple mean forecast in periods of high volatility, but, in periods of low volatility, the use of economic predictors may lead to small forecast improvements.
Keywords: state dependence; forecast evaluation; pockets of predictability
JEL Codes: C52; C53; G17
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| increased stock market volatility (G17) | poorer predictive performance of economic models (E17) |
| decreased stock market volatility (G17) | improved predictive performance of economic models (E17) |
| stock market volatility (G17) | forecasting performance of economic models (E17) |