Working Paper: NBER ID: w21399
Authors: Bruce D. Meyer; Wallace K.C. Mok; James X. Sullivan
Abstract: Household surveys, one of the main innovations in social science research of the last century, are threatened by declining accuracy due to reduced cooperation of respondents. While many indicators of survey quality have steadily declined in recent decades, the literature has largely emphasized rising nonresponse rates rather than other potentially more important dimensions to the problem. We divide the problem into rising rates of nonresponse, imputation, and measurement error, documenting the rise in each of these threats to survey quality over the past three decades. A fundamental problem in assessing biases due to these problems in surveys is the lack of a benchmark or measure of truth, leading us to focus on the accuracy of the reporting of government transfers. We provide evidence from aggregate measures of transfer reporting as well as linked microdata. We discuss the relative importance of misreporting of program receipt and conditional amounts of benefits received, as well as some of the conjectured reasons for declining cooperation and survey errors. We end by discussing ways to reduce the impact of the problem including the increased use of administrative data and the possibilities for combining administrative and survey data.
Keywords: household surveys; government transfers; survey quality; measurement error; nonresponse
JEL Codes: C42; C81; D31; H53; H55; I32; I38
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| rising rates of nonresponse (C83) | downward bias in reported transfer income (H31) |
| item nonresponse (C83) | downward bias in reported transfer income (H31) |
| measurement error (C20) | downward bias in reported transfer income (H31) |
| underreporting of benefits received (H55) | misestimations of poverty and inequality levels (I32) |
| measurement error (C20) | bias in transfer income reporting (H23) |
| measurement error increases over time (C22) | reliability of survey data decreases (C83) |
| biases in reporting have risen over time (J70) | understatement of incomes (H26) |
| biases in reporting have risen over time (J70) | overstatement of poverty rates (I32) |
| measurement error (C20) | economic data accuracy (C82) |