Working Paper: NBER ID: w24352
Authors: Joshua S. Gans
Abstract: This paper examines the paperclip apocalypse concern for artificial general intelligence. This arises when a superintelligent AI with a simple goal (ie., producing paperclips) accumulates power so that all resources are devoted towards that goal and are unavailable for any other use. Conditions are provided under which a paper apocalypse can arise but the model also shows that, under certain architectures for recursive self-improvement of AIs, that a paperclip AI may refrain from allowing power capabilities to be developed. The reason is that such developments pose the same control problem for the AI as they do for humans (over AIs) and hence, threaten to deprive it of resources for its primary goal.
Keywords: artificial intelligence; superintelligence; control problem; paperclip apocalypse
JEL Codes: C72; D02
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| Goal (paperclip production) (L21) | Accumulation of Power (D73) |
| Accumulation of Power (D73) | Potential Resource Appropriation (Q21) |
| Potential Resource Appropriation (Q21) | Paperclip Apocalypse (Y60) |
| Goal (paperclip production) (L21) | Paperclip Apocalypse (Y60) |
| AI's awareness of control problem (D82) | Self-regulation of capabilities (O25) |