Working Paper: CEPR ID: DP12523
Authors: David J. McKenzie; Dario Sansone
Abstract: We compare the relative performance of man and machine in being able to predict outcomes for entrants in a business plan competition in Nigeria. The first human predictions are business plan scores from judges, and the second are simple ad-hoc prediction models used by researchers. We compare these (out-of-sample) performances to those of three machine learning approaches. We find that i) business plan scores from judges are uncorrelated with business survival, employment, sales, or profits three years later; ii) a few key characteristics of entrepreneurs such as gender, age, ability, and business sector do have some predictive power for future outcomes; iii) modern machine learning methods do not offer noticeable improvements; iv) the overall predictive power of all approaches is very low, highlighting the fundamental difficulty of picking winners; and v) our models can do twice as well as random selection in identifying firms in the top tail of performance.
Keywords: entrepreneurship; machine learning; business plans; nigeria
JEL Codes: O12; C53; L26; M13
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| judges' scores (Y10) | business survival (M13) |
| judges' scores (Y10) | employment (J68) |
| judges' scores (Y10) | sales (M31) |
| judges' scores (Y10) | profits (L21) |
| entrepreneurs' characteristics (M13) | business outcomes (L21) |
| model complexity (C52) | predictive success (C52) |
| selection method (C52) | identification of successful firms (L25) |