Working Paper: NBER ID: w31422
Authors: Nikhil Agarwal; Alex Moehring; Pranav Rajpurkar; Tobias Salz
Abstract: Full automation using Artificial Intelligence (AI) predictions may not be optimal if humans can access contextual information. We study human-AI collaboration using an information experiment with professional radiologists. Results show that providing (i) AI predictions does not always improve performance, whereas (ii) contextual information does. Radiologists do not realize the gains from AI assistance because of errors in belief updating – they underweight AI predictions and treat their own information and AI predictions as statistically independent. Unless these mistakes can be corrected, the optimal human-AI collaboration design delegates cases either to humans or to AI, but rarely to AI assisted humans.
Keywords: Artificial Intelligence; Radiology; Human-AI Collaboration; Diagnostic Quality
JEL Codes: C50; C90; D47; D83
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| biases in belief updating (D83) | effectiveness of AI predictions (C52) |
| AI predictions (C45) | diagnostic quality (L15) |
| AI predictions (high confidence) (C45) | diagnostic quality (L15) |
| AI predictions (low confidence) (C45) | diagnostic quality (L15) |
| contextual information (D80) | diagnostic quality (L15) |