Working Paper: NBER ID: w20592
Authors: Campbell R. Harvey; Yan Liu; Heqing Zhu
Abstract: Hundreds of papers and hundreds of factors attempt to explain the cross-section of expected returns. Given this extensive data mining, it does not make any economic or statistical sense to use the usual significance criteria for a newly discovered factor, e.g., a t-ratio greater than 2.0. However, what hurdle should be used for current research? Our paper introduces a multiple testing framework and provides a time series of historical significance cutoffs from the first empirical tests in 1967 to today. Our new method allows for correlation among the tests as well as missing data. We also project forward 20 years assuming the rate of factor production remains similar to the experience of the last few years. The estimation of our model suggests that a newly discovered factor needs to clear a much higher hurdle, with a t-ratio greater than 3.0. Echoing a recent disturbing conclusion in the medical literature, we argue that most claimed research findings in financial economics are likely false.
Keywords: cross-section of expected returns; multiple testing framework; statistical significance; asset pricing
JEL Codes: C01; C58; G0; G1; G12; G3
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| traditional significance criteria (e.g., a t-ratio greater than 20) (C52) | credibility of factors in asset pricing (G12) |
| newly discovered factor needs to achieve a t-ratio of 30 (C38) | credibility of factors in asset pricing (G12) |
| adjusted threshold for significance levels (C24) | reliability of factors in asset pricing (G12) |
| multiple testing corrections (C52) | spurious results in asset pricing (G19) |
| historical analysis of significance levels (C12) | adjustment of significance criteria (C52) |