Working Paper: NBER ID: w31751
Authors: Emily Aiken; Suzanne Bellue; Joshua Blumenstock; Dean Karlan; Christopher R. Udry
Abstract: Do non-traditional digital trace data and traditional survey data yield similar estimates of the impact of a cash transfer program? In a randomized controlled trial of Togo’s COVID-19 Novissi program, endline survey data indicate positive treatment effects on beneficiary food security, mental health, and self-perceived economic status. However, impact estimates based on mobile phone data – processed with machine learning to predict beneficiary welfare – do not yield similar results, even though related data and methods do accurately predict wealth and consumption in prior cross-sectional analysis in Togo. This limitation likely arises from the underlying difficulty of using mobile phone data to predict short-term changes in wellbeing within a rural population with fairly homogeneous baseline levels of poverty. We discuss the implications of these results for using new digital data sources in impact evaluation.
Keywords: cash transfers; impact evaluation; digital data; machine learning; randomized controlled trial
JEL Codes: C55; I32; I38
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| Mobile Phone Data (L96) | Welfare Outcomes (I38) |
| Mobile Phone Data Predictions vs Survey Data Predictions (C83) | Discrepancies in Impact Estimates (C80) |
| Cash Transfers (F35) | Food Security (Q18) |
| Cash Transfers (F35) | Mental Health (I19) |
| Cash Transfers (F35) | Self-Perceived Economic Status (D31) |
| Cash Transfers (F35) | Composite Index of Welfare (I31) |