Working Paper: NBER ID: w20955
Authors: Patrick Bajari; Denis Nekipelov; Stephen P. Ryan; Miaoyu Yang
Abstract: We survey and apply several techniques from the statistical and computer science literature to the problem of demand estimation. We derive novel asymptotic properties for several of these models. To improve out-of-sample prediction accuracy and obtain parametric rates of convergence, we propose a method of combining the underlying models via linear regression. Our method has several appealing features: it is robust to a large number of potentially-collinear regressors; it scales easily to very large data sets; the machine learning methods combine model selection and estimation; and the method can flexibly approximate arbitrary non-linear functions, even when the set of regressors is high dimensional and we also allow for fixed effects. We illustrate our method using a standard scanner panel data set to estimate promotional lift and find that our estimates are considerably more accurate in out of sample predictions of demand than some commonly used alternatives. While demand estimation is our motivating application, these methods are likely to be useful in other microeconometric problems.
Keywords: demand estimation; machine learning; model combination; promotional lift; econometrics
JEL Codes: C14; C53; C55
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| promotional activities (M31) | demand outcomes (P47) |
| machine learning models (C45) | demand prediction accuracy (C53) |
| promotional activities (M31) | promotional lift (J62) |
| larger datasets (C55) | reduced bias in causal estimates (C21) |
| machine learning methods (C45) | marketing effectiveness insights (M31) |