Working Paper: NBER ID: w29184
Authors: Pablo A. Celhay; Bruce D. Meyer; Nikolas Mittag
Abstract: We document the extent, nature, and consequences of survey errors in cash welfare and SNAP receipt in three major U.S. household surveys. We find high rates of misreporting, particularly failure to report receipt. The surveys inaccurately capture patterns of multiple program participation, even though there is little evidence of program confusion. Error rates are higher among imputed observations, which account for a large share of false positive errors. Many household characteristics have significant effects on both false positives and false negative errors. Error rates sharply differ by race, ethnicity, income and other household characteristics. The errors greatly affect models of program receipt and estimated effects of income and race are noticeably biased. We examine error due to item non-response and imputation, as well as whether imputation improves estimates. Item non-respondents have higher receipt rates than the population conditional on covariates. The assumptions for consistent estimates in multivariate models fail both when excluding item non-respondents and when using the imputed values. In binary choice models of program receipt, linked data estimates favor excluding item non-respondents rather than using imputed values. Biases are well predicted by the error patterns we document, helping researchers make informed decisions on whether to use imputed values.
Keywords: survey errors; government benefits; imputation; SNAP; public assistance
JEL Codes: C81; D31; I32; I38
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| survey errors (C83) | biased estimates of program receipt (C83) |
| misreporting of SNAP and public assistance (I38) | significant biases in understanding program participation (I24) |
| household characteristics (race, income, employment status) (R20) | error rates (C52) |
| imputed values (D46) | exacerbate biases (D91) |
| item nonresponse and imputation errors (C83) | biased estimates (C51) |