Working Paper: CEPR ID: DP15418
Authors: Eric Ghysels; Andrii Babii; Xi Chen; Rohit Kumar
Abstract: The importance of asymmetries in prediction problems arising in economics has been recognized for a long time. In this paper, we focus on binary choice problems in a data-rich environment with general loss functions. In contrast to the asymmetric regression problems, the binary choice with general loss functions and high-dimensional datasets is challenging and not well understood. Econometricians have studied binary choice problems for a long time, but the literature does not offer computationally attractive solutions in data-rich environments. In contrast, the machine learning literature has many computationally attractive algorithms that form the basis for much of the automated procedures that are implemented in practice, but it is focused on symmetric loss functions that are independent of individual characteristics. One of the main contributions of our paper is to show that the theoretically valid predictions of binary outcomes with arbitrary loss functions can be achieved via a very simple reweighting of the logistic regression, or other state-of-the-art machine learning techniques, such asboosting or (deep) neural networks. We apply our analysis to racial justice in pretrial detention.
Keywords: No keywords provided
JEL Codes: No JEL codes provided
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| application of logistic regression or machine learning techniques (C35) | valid predictions of binary outcomes (C25) |
| application of logistic regression or machine learning techniques (C35) | improvement in fairness in predictions (C52) |