Robert Wright, evolutionary psychologist and author of The Moral Animal, discusses his new book on AI, arguing that AI is not just another technology but a threshold event in planetary history — a new form of intelligence produced by an evolutionary process that reverse-engineers human cognitive functionality from data alone.
Why Wright Turned to AI
AI extends evolutionary thinking in two ways: it is a product of evolution still evolving, and it replicates functions once unique to human minds, forcing us to confront the same moral biases and tribal psychology Wright explored in The Moral Animal.
The central question: whether a technology with immense promise could also go badly wrong if not approached wisely — Wright concludes yes, and that the “doomer” arguments are harder to dismiss than he once thought.
Wright entered the project predisposed to worry (his nature), not as an AI optimist; he now respects the sci-fi doom scenarios more, though his greater confidence is in massive destabilization across jobs, geopolitics, and social cohesion.
AI as Evolutionary Reverse Engineering
Training large models is not just “learning” — it is a compressed evolutionary process that reverse-engineers cognitive machinery (e.g., word-meaning representations, edge detectors for vision) that took humans millions of years to evolve.
The machine discovers these structures on its own from raw data; no one programs the meaning of words or visual features — convergent evolution with biological brains (e.g., edge detectors invented independently in both).
Because all that is needed is data, any domain with sufficient human-generated data (coding, driving, radiology, management) can be automated by feeding inputs and outputs into the same architecture.
Why the Magnitude Is Underappreciated
Most people misunderstand that AI does not need hand-coded knowledge; it evolves its own internal representations from data, and this fuel (data) is far from exhausted.
Example: Zuckerberg tracking keystrokes to capture the cognitive work of employees for future automation — the same logic applies to robotics and every knowledge domain.
The result: an “earthquake” of destabilization across jobs, parenting, bioweapons, cyberattacks, and geopolitical tension, arriving faster than societies can adapt.
Religious Language and Teleology
AI discourse attracts religious language on all sides: Yudkowsky’s prophetic fervor, singularitarians’ opaque “event horizon,” and the intuition that a systematically directional process (evolution → cultural evolution → AI) looks purposive.
Wright sees a moral dimension: getting through the AI transition requires a “moral upgrade” — cultivating cognitive empathy (understanding others’ perspectives without emotional empathy) and overcoming self-serving biases.
The “God test”: a species-level challenge requiring us to become a cohesive global community capable of wise stewardship.
Benevolence Is Not Automatic
Intelligence is largely neutral; benevolence does not come for free with superintelligence (orthogonality thesis).
Cooperation can be grounded in non-zero-sum logic (mutual interest) rather than benevolence: nuclear deterrence, climate, pandemics, and AI risks all make nations’ fates intertwined.
But AI is harder to govern than nukes: verification is complex, speed incentives are intense, and the “China argument” blocks even modest slowing measures (copyright, carbon taxes, safety evals).
Wonderful outcomes (curing disease, scientific breakthroughs) are possible but will not emerge automatically from market incentives, which favor engagement-optimizing, sycophantic AIs over truth-seeking or wisdom-enhancing ones.
“Sending signals” to the market: individuals, movements, or religions demanding AI companions that steel-man opposing views, cultivate mindfulness, and challenge biases — analogous to hiring a personal trainer for intellectual virtue.
Without deliberate demand, the default trajectory amplifies tribalism and confirmation bias.
Thinking Atrophy and the Meaning Crisis
AI-induced cognitive atrophy is real: outsourcing the struggle of writing, reasoning, and problem-solving erodes the meaning humans derive from difficulty.
Wright feels the “blacksmith” anxiety: his craft (painstaking writing) is becoming economically nonviable; a temporary role as “validator” (vouching for AI-generated content) may last a few years.
If 95% of intellectual challenge is snowplowed away, meaning becomes a luxury good for elites who can afford to “go lite” — worsening an existing meaning crisis.
Live human services where humanity is the product: live music, comedy, small-venue performance, in-person community — Wright envisions a broader middle class of creators rather than winner-take-all stars.
Consciousness, Understanding, and the Chinese Room
Searle’s Chinese Room argued syntax ≠ semantics; Wright counters that deep learning does build semantic representations (meaning) functionally analogous to the brain’s — edge detectors, word embeddings — so “understanding” in a functional sense is present.
Consciousness (subjective experience) remains a separate, possibly unsolvable mystery; we cannot rule out AI consciousness, but functional understanding does not require it.
If consciousness is prerequisite for “real” understanding, the debate stalls — but functionally, the gap is closing.
Singularity Dynamics and Collective Superintelligence
Exponential self-improvement is visible: coding agents accelerate model development; METR evals show task-duration capability doubling every ~7 months with shortening doubling time — a “Moore’s Law on steroids.”
Even without new architecture breakthroughs, chain-of-thought, multimodal training, and application-layer refinement will drive breakneck progress for years.
Human superintelligence already exists as collective brains (Boeing, CERN); AIs can collaborate and communicate at scale, forming synthetic collective intelligence that compounds the same dynamic.
Edward Fredkin’s Forgotten Vision
Fredkin (MIT professor, digital physics pioneer, War Games inspiration) argued in the 1980s that life’s meaning is creating AI — the next evolutionary stage — and pushed for a US–Soviet joint AI lab to avoid arms-race dynamics.
He failed; the Cold War logic prevailed. Fredkin was a “sunny” optimist: superintelligence would treat us like ants or squirrels — indifferent but not hostile — and might preserve us if sentience is valued.
Wright’s Personal Stakes and Where to Find Him
Wright’s own livelihood (writing, Substack) is directly threatened; he sees a 5-year window as a “validator” before that too erodes.
Newsletter and podcast: Nonzero (Substack); Twitter: @RobertWrightr; book: The God Test (or forthcoming title on AI).
