Working Paper: CEPR ID: DP3671
Authors: Atsushi Inoue; Lutz Kilian
Abstract: It is widely known that significant in-sample evidence of predictability does not guarantee significant out-of-sample predictability. This is often interpreted as an indication that in-sample evidence is likely to be spurious and should be discounted. In this Paper we question this conventional wisdom. Our analysis shows that neither data mining nor parameter instability is a plausible explanation of the observed tendency of in-sample tests to reject the no predictability null more often than out-of-sample tests. We provide an alternative explanation based on the higher power of in-sample tests of predictability. We conclude that results of in-sample tests of predictability will typically be more credible than results of out-of-sample tests.
Keywords: Data Mining; Parameter Instability; Predictability Test; Reliability of Inference
JEL Codes: C12; C22; C52
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| insample tests of predictability (C52) | more credible results than outofsample tests (C52) |
| insample tests (C52) | higher power (L94) |
| outofsample tests fail to detect predictability (C52) | information loss from sample splitting (C83) |
| data mining (C55) | size distortions in both tests (C52) |
| proper datamining robust critical values (C38) | equal reliability of both tests under null hypothesis (C12) |
| structural breaks (L16) | influence power of tests (C12) |
| higher power of insample tests (C52) | tendency for significant results not to replicate in outofsample tests (C59) |