Anthropic’s justification for its pursuit of superintelligence ignores history

An AI generated image of "superintelligence"
What ChatGPT created at the prompt, “please create a psychedelic .jpg image depicting superintelligence.” For the record, it brought back a .png.

To contribute very briefly to the deluge of analysis surrounding Anthropic’s successful initial public offering today: Dario Amodei’s justification for Anthropic’s messianic pursuit of recursively self-improving AI ignores the history of innovation.

Amodei argues for using AI to improve AI, rapidly and without a clear idea of where AI will take AI, based on the idea that, if Anthropic doesn’t do it, someone else will (in particular, China will). The victor will, in doing so, achieve “geopolitical dominance.”

Given the number of frontier models out there and the amount of AI talent being cultivated around the world, this is nonsensical. The clearest analogy is the development of the atomic bomb. The Soviets detonated their version four years after the Little Boy erased much of Hiroshima. Espionage helped, but there were some smart people behind the Iron Curtain.

If any foundation-model AI player, Anthropic or otherwise, cracks the superintelligence nut, the rest will soon have it, too. Most won’t need espionage: Key Anthropic employees, if they’re indeed as altruistic as they purport to be, would simply recognize the danger of centralizing such power in one organization and hire into OpenAI or Meta or Alphabet or Tesla to help them along. And that assumes competitors don’t figure it out on their own anyway. These models all operate on the same fundamental principles, after all.

And given China’s technical (and industrial-espionage) chops, how far behind would they really be? Not four years. There will be no superintelligence-derived “geopolitical dominance,” because others will catch up too fast to enable it.

And would American-made superintelligent AI somehow dislodge China as the world’s manufacturer? Would it discover, mine, and refine vast new strategic lithium or rare-earth reserves? Perhaps its a lack of imagination, but it’s hard to fathom superintelligent AI operating at the beck and call of cognitively inferior primates quickly fashioning, training up, and orchestrating robot-AI armies to take over the world.

Theres a nuclear deterrent for that anyway. Superintelligence would probably figure that out.

There is one scenario in which Amodei could be on point, in a way: An Anthropic model achieves superintelligence and quickly proves the AI doomers right. But there’s no winner there.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.