Working Paper: NBER ID: w23298
Authors: Isaiah Andrews; Maximilian Kasy
Abstract: Some empirical results are more likely to be published than others. Such selective publication leads to biased estimates and distorted inference. This paper proposes two approaches for identifying the conditional probability of publication as a function of a study’s results, the first based on systematic replication studies and the second based on meta-studies. For known conditional publication probabilities, we propose median-unbiased estimators and associated confidence sets that correct for selective publication. We apply our methods to recent large-scale replication studies in experimental economics and psychology, and to meta-studies of the effects of minimum wages and de-worming programs.
Keywords: Publication Bias; Empirical Research; Statistical Inference
JEL Codes: C12; C13; C18
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| Selective publication (Y30) | Biased estimates (C51) |
| Selective publication (Y30) | Distorted inference (D80) |
| Median-unbiased estimators (C51) | Valid inference on study parameters (C20) |
| Negative significant effects (D62) | More likely to be published (C46) |
| Results showing no significant impact (C52) | More likely to be included in metastudies (C90) |