AMA: career advice given AGI, how I research ft. Sholto & Trenton

Dwarkesh Podcast 49min 7 min #85
AMA: career advice given AGI, how I research ft. Sholto & Trenton
Watch on YouTube

Summary

  • Dwarkesh Patel launches his book The Scaling Era (published by Stripe Press), a curated compilation of insights from his podcast interviews with AI lab CEOs, researchers, economists, philosophers, and other scholars
    • The book organizes short, page-by-page excerpts from different interviews by topic, letting readers see how ideas from different fields connect — for example, a physicist explaining scaling via data manifolds next to an evolutionary biologist discussing why general intelligence evolved
    • It includes side captions and diagrams explaining technical concepts (parameters, models, etc.) to make interviews accessible to non-experts, including Dwarkesh’s own parents
    • Two previously unreleased interviews are included, one with Jared Kaplan (Anthropic co-founder) and one with Goren
    • Available at stripe.press/scaling

Why LLMs don’t make cross-disciplinary connections

  • Dwarkesh raises a puzzle he originally posed to Dario Amodei: LLMs have vast knowledge memorized but don’t seem to make novel cross-field connections the way humans do (e.g., someone noticing magnesium deficiency patterns match migraine structures, leading to a cure)
    • Sholto’s take: Pre-training gives flexible general knowledge but doesn’t imbue the skill of making novel discoveries — that requires RL and training that models how researchers actually explore and interact with the world; the field hasn’t meaningfully scaled this yet
    • Trenton’s angle: Models may lack “memory scaffolding” — they predict the next token on internet text rather than deciding what’s worth summarizing and storing; humans are aware of their memory limitations and construct summaries accordingly, while models just memorize raw text
    • The Kim Peek analogy: LLMs may be “idiot savants” — like Kim Peek, who had a perfect encyclopedic memory but lacked social functioning, excelling in narrow domains while failing in others
    • The perfect-memory trade-off: There’s a documented case of someone with perfect recall whose memory was debilitating — attending to too many past details prevents extracting generalizable insights; LLMs sit at one end (memorizing exact phrasing) while young children sit at the other (forgetting almost everything but generalizing well)
    • Dwarkesh notes that his own learning has been heavily shaped by a group chat and in-person meetings with people like Sholto and Trenton, who feed him ideas that serve as intuition pumps during conversations

Career advice given AGI timelines

  • When asked what to study if you’re 17 and AGI is coming, the group emphasizes increasing individual leverage rather than any specific field
    • Software engineers already report being 2–5x faster with AI assistance; this trend will continue from “pairing session” to “managing a team of AIs” to “managing a division”
    • Deep technical knowledge will still matter because you’ll be managing enormous resources and need to understand what AIs are and aren’t good at
    • Dwarkesh’s actual advice: “Put yourself close to the frontier” — it’s remarkably obvious what the real problems are once you’re there; study fundamentals but in an AI-native, top-down way rather than bottom-up rote memorization
    • The group is skeptical of career advice in general — Dwarkesh found 80,000 Hours useless in college; the best approach is to just try things
    • There’s a tension between experienced professionals with tacit knowledge being accelerated by AI versus newcomers who can’t replicate the onboarding path (solving GitHub issues, etc.)

How Dwarkesh chooses podcast guests

  • The key question: “Do I want to spend 1–2 weeks reading everything this person has ever written and talking to their colleagues?” — the research is the bottleneck, not the interview itself
    • Big names don’t matter much for growth; Dwarkesh’s most popular guest was Sarah Constance, an obscure scholar before the interview, followed by geneticist David Reich — not tech CEOs
    • It’s hard to predict who will be a breakout guest, so the heuristic is just to research whoever seems interesting — that’s a good proxy for what will actually be popular
    • Dwarkesh frequently declines influential people who ask to come on, because the interview prospect isn’t fundamentally interesting enough to justify two weeks of research

Why Dwarkesh pursued the podcast long-term

  • The turning point was shopping ad spots for a Mark Zuckerberg episode and realizing he could hire full-time editors — it became a real business
    • He encourages young people to start blogs or podcasts, calling “the Matt Levine of AI” a totally open niche
    • Three things to emphasize: (1) It can be done, (2) you can make real money, (3) early work will suck but you need to stick with it long enough to get the RL feedback loop on how to improve
    • The real flywheel isn’t audience growth — it’s that the podcast is good enough to get him access to people like Sholto and Trenton, who teach him things, which makes the podcast better, which gets him access to people in other fields, and so on; a single blog post about China netted him an entire China network
    • The first big break was a blog post on the “Annus Mirabilis” that Jeff Bezos retweeted — not because of a fundamentally new insight, but because it crisply articulated ideas people already had

Reading habits and learning methods

  • Dwarkesh uses Anki (spaced repetition flashcards) extensively, including a Claude integration, and credits it with a huge uplift in his ability to learn — mainly by consolidating complex subjects so you don’t keep “climbing the same hill”
  • He doesn’t do book clubs or use GoodReads
  • Currently reading: a hand-made translation of the Greek poet Cavafy by his friend Alvaro De Menard (Fantastic Anachronism), limited to 100 copies
  • His method for preparing for episodes is straightforward: read the guest’s books and papers, talk to their colleagues, and learn enough to ask interesting questions (which is much harder than being a practitioner)

Long-term goals and the podcast’s purpose

  • AGI makes long-term planning difficult; for now the goal is to grow the podcast and do more writing
  • Dwarkesh has contradictory views on whether the podcast should shape the world:
    • On one hand, good arguments do reach decision-makers efficiently — important decisions are being made now, and a strong piece of writing can “one-shot” the relevant person
    • On the other hand, it’s very hard to know what should be done; you need a correct world model and need to predict how your actions will play out
    • His conclusion: the podcast should be an “epistemic tool” — just helping people understand the relevant arguments is the highest priority, because it’s easy to be wrong about prescriptions

Who should run a frontier AI lab

  • The group discusses which historical figure would be best suited
    • LBJ would be effective at the actual job — raising money, building hype, setting a vision, coordinating among groups, getting concessions; his teaching to debate students was “If you do everything, you’ll win”
    • Robert Moses would also make things happen, though not necessarily in a way that’s good for the world
    • There’s a tension: “Great people are very rarely good people” (Lord Acton); it’s hard to find someone who both drives progress and has trustworthy moral judgment
    • The group thinks we’re relatively lucky with the current set of lab leaders, who genuinely care about safety as well as progress; this makes Dwarkesh skeptical of grand schemes like nationalization that would shake up the landscape

Preparing for fast AGI timelines

  • If AGI arrives quickly, there will be a ~6-month window where the most important decisions in human history are made; having an AI podcast during that time might be useful
  • Financial preparations:
    • Dwarkesh reinvested his first real ad revenue (after going negative 23 cents) into Nvidia stock
    • Trenton canceled his 401K contributions, finding it hard to imagine the money sitting until age 60 in a radically different world
    • Sholto hasn’t changed his lifestyle much — he just works all the time regardless
  • Altruistic use of money: Dwarkesh considers supporting up-and-coming content creators, inspired by people who supported him early (Anil Varanasi, Leopold’s foundation)
    • The challenge is that good talent is hard to find before they’re visible; grant applications may not work well
    • A better model might be funding people to move to San Francisco for 2 months to get immersed in the intellectual environment — similar to how the MATS program and Anthropic Fellows Program give researchers time, funding, and mentorship
    • These fellowship programs have been incredibly successful, with multiple fellows hired by Anthropic and other labs; quality has improved over time as a flywheel effect

Growing the podcast — distribution and hiring

  • Distribution: The most underrated factor; YouTube Shorts have been responsible for at least half the podcast’s growth, which nobody would have predicted; TikTok hasn’t worked despite attempts
    • Write tweets like you’re writing to a group chat of friends, not formally
    • Dwarkesh enjoys the content but finds the optimization game (thumbnails, titles, etc.) somewhat tedious
  • Hiring editors: Dwarkesh found 3–4 incredible editors through a public competition — a farmer in Argentina, a math student in Sri Lanka, a former Mr Beast editor, and a director from Czechia who makes AI animations; he 10x’d their salaries and can’t replicate the process
    • The key insight: there’s global arbitrage for people who are data-oriented and willing to obsessively improve
  • Hiring a general manager: Took a year and ~1,000 applications; the person he hired (Max Herrns, his childhood best friend) came through a mutual friend referral, not the public process — the best people don’t apply to public postings
    • This is the #1 problem startup CEOs face; Dwarkesh genuinely doesn’t know how big companies hire at scale

How to get started as a new writer

  • Two useful cold-start hacks:
    1. Podcasting — interview people who have original takes and leverage their platform; you don’t need your own worldview yet
    2. Book reviews — you have something to react to; this is massively undersupplied (Dwarkesh himself constantly visits Gwern’s book reviews and Jason Furman’s GoodReads page)
  • Beyond that: just try things, and don’t expect to know in advance what will work
Back to Dwarkesh Podcast