Working Paper: NBER ID: w26716
Authors: Stefano Dellavigna; Nicholas Otis; Eva Vivalt
Abstract: Forecasts of experimental results can clarify the interpretation of research results, mitigate publication bias, and improve experimental designs. We collect forecasts of the results of three Registered Reports preliminarily accepted to the Journal of Development Economics, randomly varying four features: (1) small versus large reference values; (2) whether predictions are in raw units or standard deviations; (3) text-entry versus slider responses; and (4) small versus large slider bounds. Forecasts are generally robust to elicitation features, though wider slider bounds are associated with higher forecasts throughout the forecast distribution. We make preliminary recommendations on how many forecasts should be gathered.
Keywords: forecasting; experimental results; elicitation strategy
JEL Codes: O10; O17
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| Elicitation features (e.g., slider bounds) (C51) | Forecast accuracy (C53) |
| Wider slider bounds (J62) | Higher forecasts (E37) |
| Wider slider bounds (J62) | Shift forecasts to the right (C69) |
| Reference value used in examples (Y20) | Forecast accuracy (C53) |
| Format of elicitation (raw units vs standard deviations) (C83) | Average forecasts (E17) |
| Consistency of slider ranges (C20) | Comparability of forecasts across studies (C53) |