Working Paper: CEPR ID: DP17927
Authors: Zuzana Irsova; Pedro R.D. Bom; Tomas Havranek; Heiko Rachinger
Abstract: Meta-analysis upweights studies reporting lower standard errors and hence more precision. But in empirical practice, notably in observational research, precision is not given to the researcher. Precision must be estimated, and thus can be p-hacked to achieve statistical significance. Simulations show that a modest dose of spurious precision creates a formidable problem for inverse-variance weighting and bias-correction methods based on the funnel plot. Selection models fail to solve the problem, and the simple mean can beat sophisticated estimators. Cures to publication bias may become worse than the disease. We introduce an approach that surmounts spuriousness: the Meta-Analysis Instrumental Variable Estimator (MAIVE), which employs inverse sample size as an instrument for reported variance.
Keywords: publication bias; phacking; selection models; metaregression; funnel plot; inverse-variance weighting
JEL Codes: C15; C26; C83
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| manipulation of standard errors (C20) | biased results (J15) |
| lower reported standard errors (C20) | more weight in meta-analyses (C83) |
| traditional bias-correction methods (C51) | inadequate address of spurious precision (C49) |
| MAIVE method (C36) | correction of bias (C83) |
| MAIVE method (C36) | reliable estimation of true effects (C51) |
| spurious precision (C59) | inflated statistical significance of estimates (C51) |
| spurious precision (C59) | distorted results of inverse-variance weighting (C46) |