Oxford’s AI Chair: The Singularity is Bullshit

Marcus du Sautoy, Oxford’s Simonyi Chair for the Public Understanding of Science, argues that the popular “singularity” narrative — the idea that AI will rapidly surpass human intelligence, recursively self-improve, and spiral out of control — is deeply implausible and distracts from real, present-day AI risks.

• The singularity story is compelling because it taps into a primal fear: creating something that turns on you, like Frankenstein’s monster or Skynet in Terminator.
• This narrative dominates AI risk discourse, funding, and policy, crowding out more immediate concerns under the umbrella of “existential risk” (X-risk).
• Du Sautoy sees the appeal as almost religious — a psychological drive toward apocalyptic thinking, similar to Christian end-times movements or climate doom — not necessarily because the threat is likely, but because it feels total and world-orienting.

Why the singularity argument is flawed

• The idea that AI will rapidly and uncontrollably self-improve ignores how fragile and patchwork current AI systems actually are.
• There are two main versions of the existential risk argument:
  • The paperclip argument: A superintelligent AI follows a goal (e.g., “make paperclips”) in unintended, harmful ways (e.g., enslaving humanity). But for this to happen, humans would have to give the AI dangerous levels of control — like access to nuclear weapons — which is unlikely and avoidable.
  • The misaligned goals argument: AI develops its own goals that conflict with human interests. This is even more speculative, especially since current large language models (LLMs) show no signs of agency or independent goal formation.
• Du Sautoy stresses there is no clear roadmap from today’s AI to either scenario, making the singularity narrative more science fiction than science.

The real risks we should be focusing on

• Du Sautoy and colleagues coined the term “existential risk risk” — the danger that society becomes so fixated on far-fetched, apocalyptic AI scenarios that it ignores tangible, current harms.
• These include issues like bias in AI systems, job displacement, misinformation, erosion of privacy, and the concentration of power in a few tech companies.
• Shifting focus away from X-risk allows for more productive conversations about how to govern, regulate, and deploy AI responsibly today.