Andy Matuschak — The reason most learning tools fail

Dwarkesh Podcast 2h22 9 min #52
Andy Matuschak — The reason most learning tools fail
Watch on YouTube

Summary

  • Andy Matuschak is a researcher, engineer, and designer working on tools for thought—he previously worked on flagship iOS features at Apple and co-created Quantum Country, an augmented textbook on quantum computation that embeds spaced repetition review questions into the text. He is also a dedicated practitioner of memory systems and active reading techniques, and this conversation explores why most learning tools fail, how memory actually works, and what effective learning looks like in practice.

Why people fail to understand what they read

  • Most students don’t deeply engage with material because they’re learning things they don’t care about, or there’s no clear connection between the reading and what originally inspired them—the education system often imposes goals that aren’t theirs.
  • But when people are genuinely interested (e.g., hobbyist subreddits on knots or gardening), they go to extraordinary lengths to absorb material—so motivation is a key variable.
  • A second model of failure: people never understood the material in the first place and didn’t notice—Mortimer Adler and Van Doren’s How to Read a Book argue that many readers lack the skill to apprehend difficult texts, not the desire.
    • They compare it to an inexperienced rock climber facing a wall with subtle holds that an experienced climber sees as a ladder.
    • The fix is building the habit of asking questions of the text and trying to answer them—an undemanding reader asks no questions and gets no answers.

Skillful reading in practice

  • When Andy read an introductory quantum mechanics chapter, he spent about 15 minutes per page, constantly interrogating whether he understood each relationship the author was drawing—this deliberateness and effortfulness is itself a learnable skill.
  • The host noticed that during this process, he realized he didn’t understand things he thought he had understood when reading alone—the act of being questioned surfaces gaps that passive reading hides.

Outsourcing metacognition for self-learners

  • Metacognition (planning, self-regulation, evaluating understanding) is cognitively expensive and gets harder as material gets more difficult—one solution is to outsource it.
    • Using someone else’s syllabus from a graduate course is a design choice that offloads planning.
    • In Quantum Country, embedded review questions every ~1,500 words served a dual purpose: they helped with memory, but more importantly they gave readers feedback that they hadn’t actually absorbed what they just read—sometimes they hadn’t even noticed a key term.
    • Readers reported that after hitting difficult questions, they slowed down and read more attentively going forward.
  • Adjunct questions (questions embedded in text) have been shown in research to improve both retention of the specific material and forward reading behavior.
  • For self-learners deciding whether to skip chapters: early on, you should outsource the “what’s necessary” question to the author or syllabus, treat it as tentative, and revise as you develop more domain knowledge—but avoid a completionist mindset that kills motivation.

Scaffolding and when to remove it

  • Scaffolding is a learning science term for temporary support structures: simpler questions first, partially worked examples, worked examples where you predict the next step.
  • “Fading” is the process of removing scaffolding as learners become more capable—continuing to provide full scaffolding would actually reduce learning effectiveness.
  • In familiar domains (like computer science for Andy), less scaffolding is needed because the cognitive load is lower and you can plan your own path—this is why introductory courses can be valuable even if not directly instrumental, because they provide the context network that makes future learning easier.
  • A key problem with pure unschooling: a child might have a passion (e.g., ocean geology) but lack any exposure to it if their environment doesn’t provide it—structured exposure to diverse topics can light up interests they wouldn’t have chosen themselves.

Memory, forgetting, and whether it matters

  • Paul Graham’s claim that reading trains your model of the world even if you forget the details is true to an extent, but whether it’s sufficient depends on your goals—deep, detailed understanding requires more than just absorbed impressions.
  • John Anderson’s ACT-R theory of cognition includes “knowledge compilation”: turning individual facts into generalizable patterns—but this requires repeated exposure and use in varied contexts, not just reading once.
  • Andy’s own experience: he retains almost nothing from detailed prep work for podcast interviews, but has gotten better at interviewing as a skill—the compiled knowledge is in his practices, not his recall of facts.
  • Tyler Cowen is an example of someone who integrates vast amounts of information without explicit note-taking—his daily writing practice (blog posts, columns, books) functions as a form of note-taking, and he has decades of practice thinking out loud on paper.

Do LLMs make memorization more or less valuable?

  • Memory is deeply underrated: understanding difficult arguments is memory-bound because you need to hold multiple steps or reduce them to known abstractions; creative insight often comes from noticing surprising connections between things you already have in memory.
  • Andy’s example: at Apple, memorizing details about the education market let him evaluate strategic ideas on the fly in executive conversations.
  • LLMs may help with some combinatorial noticing (like Swanson-linking in medical literature), but creative insight often feels personal and hard to externalize—Andy is skeptical LLMs can fully replace this.
  • LLMs depend on our ability to externalize and make things legible, but understanding difficult material is still bound on memory of constituent parts.
  • Forgetting may have some benefits: it helps guide attention to what’s important, prevents irrelevant details from becoming hyper-salient, and may support generalization—but there’s no evidence that we’re near any upper bound on memory capacity, and memory champions show you can know vastly more without losing normal function.

Bootstrapping vs. naturalistic practice

  • Immersion learning (e.g., language immersion) is more effective and engaging than spaced repetition when it’s available, but you can’t bootstrap into it without a foundation—explicit practice gets you to the point where naturalistic reinforcement can take over.
  • Similarly, reading a cluster of books on a topic can create natural spaced reinforcement, but for rare events (e.g., rare diagnoses for doctors, or creative research insights that come once every few months), you need out-of-band mechanisms like explicit memory practice.

Intellectual stamina and the role of willpower

  • Andy studied quantum mechanics for three hours straight and then went to a piano lesson—his stamina came partly from social energy (performing for the host), but mainly because the task was cognitively demanding in a direct way while being much less demanding on willpower/gumption than his usual research work.
    • The hardest thing in research is sitting and not knowing what to do next—being told what to do (as when reading a textbook) is a relief.
  • Active, attentive reading is more exhausting than passive reading, but there’s a flow-like state where the difficulty matches your ability and the reading becomes engaging rather than dutiful—the key is opening curiosity and attention rather than reading adversarially.

New media for learning

  • Video has taken off enormously as a learning medium (e.g., Grant Sanderson’s math videos) but viewers walk away with much less understanding than the engagement suggests.
  • Games as learning tools: Jonathan Blow’s The Witness teaches complex mechanics entirely through environmental interaction without text—it creates a feeling of conversation with the designer—but no one has successfully done this for explicit academic subjects at scale.
  • From Nand to Tetris is a course where you build a computer from scratch starting with logic gates—it’s naturally active, and the regulation of what to do next is built into the course structure rather than requiring self-regulation.
  • Apprenticeship is the non-mass-medium version of this: peripheral participation in a community of practice with hands-on feedback—streaming (e.g., George Hotz coding) is a promising scalable approximation, though it lacks feedback and contains a lot of chaff.
  • Programming feels like a video game because of the rapid feedback cycle and sense of direct manipulation—design and other fields lack this feeling of forward progress because you’re often just narrowing a search space without knowing if you’re getting closer.
    • Designers who do get obsessed often work on constrained, tractable problems (like posters) where you can rapidly explore and polish—these are “snacks” that provide the feeling of progress.

Tools for the median student vs. the motivated learner

  • Andy’s tools are geared toward highly intelligent, motivated learners—for the median student, the education system is mostly about making them do things they don’t want to do, not helping them achieve their own goals.
  • At Khan Academy, the focus was on the 25th percentile learner—the most promising angle was inquiry learning: asking authentically interesting questions that students want to answer, using dynamic manipulable representations (like Cuisenaire rods for math), and leveraging social interaction to reduce the willpower required.
  • Andy has mostly abandoned this problem to others because 90%+ of people in education focus on the bottom quartile for equity reasons, and the marginal impact is arguably greater there—but he acknowledges this means potentially missing innovations from disaffected, bored students.

Is learning inherently miserable?

  • Andy is strongly opposed to a David Goggins “embrace suffering” attitude toward learning—misery usually comes from not caring about the subject, or from resisting the feeling of struggling/failing.
  • Even rote tasks (like memorizing 200 organic chemistry names) can be made nearly costless with modern memory systems (spaced repetition takes ~100 minutes total spread over weeks)—and can be further disguised as interesting Fermi problems that require retrieving the memorized facts.
  • Andy endorses the traditional view that memorizing a subject’s taxonomy is important—but his endorsement is conditional on the cost being trivial with modern tools, whereas in the past it was emotionally difficult, time-consuming, and uncertain.
  • He uses Beeminder (which charges him money if he doesn’t do his memory practice) to ensure consistency—acknowledging a tension between his aspirational view that learning should be joyful and the practical reality that willpower is sometimes needed.

How Andy would structure his hypothetical children’s education

  • He’d likely pursue some form of homeschooling or micro-school model (like Schoolhouse or Powder House in Somerville, MA) where a small group of kids shares a teacher or coach—this is tractable financially and unbundles the social, behavioral, and instructional purposes of school.
  • For a 12-year-old: he’d expose them to many topics, be transparent about consequences of choices, and be non-coercive—but following John Dewey’s argument, he’d be skeptical of letting a 12-year-old’s impulses fully dictate their education because they lack a fully developed sense of self—true freedom requires some scaffolding.
  • He’d allow children to feel the consequences of their choices, with the safety net that they have resources to recover from early mistakes.

Has education actually improved?

  • Enormously, especially at the bottom: in 1900, only 6% of US teenagers graduated high school; now illiteracy is negligible and the bottom quartile has seen massive gains—this is the story of 20th-century mass education.
  • The 75th percentile has seen slow movement, and the ceiling (producing von Neumann-level geniuses) hasn’t clearly risen—the ceiling has always depended on aristocratic tutoring and family dynamics that mass schooling doesn’t affect.
  • It’s unclear if we’re worse at producing geniuses now—opportunity costs for potential tutors are higher (they could work in Silicon Valley), but the pool of available tutors (e.g., postdocs) is also much larger.
  • Teacher quality: there’s been some dissemination of research-backed practices (e.g., interleaving topics rather than blocking them), but it’s unclear if teachers are better overall—the claim that intelligent women were pushed into teaching in the mid-20th century and are now pulled away by other opportunities is interesting but the correlation between intelligence/subject expertise and teaching efficacy is complex.

Hypertext and note-taking

  • Wikipedia works because encyclopedia entries are designed to stand on their own—hypertext is an excellent navigational aid for reference works (dictionaries, chemical compound databases) but most ideas require narrative arcs and holistic context that don’t excerpt well.
  • Hypertext novels failed because each destination page has to work as an endpoint for all its references, forcing a lowest-common-denominator story.
  • Andy finds hypertext most useful in his own working notes—not for reading by others, but as a thinking tool that lets him accumulate context over time and connect ideas across papers and interviews.
  • Byrne Hobart’s daily newsletter doesn’t accumulate—he re-explains concepts anew each day, colored by that day’s context—this is an argument for ephemerality and against permanent note-taking in some cases.
  • Andy uses a mix: daily journal entries (intimate, context-laden, not public) and durable evergreen notes (meant to stand outside time)—he doesn’t have a clear model of when each is better.

Crowdfunding research and the tension with marketing

  • Andy crowdfunds his research through Patreon, earning between a grad student and a junior faculty member—even as arguably the most successful crowdfunded tech researcher, this barely sustains one person.
  • Crowdfunding works for him because his work is general, broadly applicable, and he can show promising results—it wouldn’t work for early-stage research or niche topics with smaller audiences.
  • He systematically avoids marketing because he’s worried about the corrosive influence of audience on honest inquiry—the temptation is to distort work to be more likable, to publish minimum viable papers, to portray results in the best light.
  • Gwern makes a tiny fraction of Andy’s Patreon revenue despite having a larger audience and more impactful research—partly because of how the membership is framed (tip jar vs. membership with clear benefits).
  • Andy’s patron-only essays are some of his best writing, but he can’t bring himself to promote them publicly because it feels terrible to link to something people can’t view—design solutions (making private content visually adjacent to public content) could help.

Spaced repetition: why isn’t it more widely adopted?

  • It’s mostly efficient that spaced repetition isn’t more widely used: in places where it’s clearly valuable and the material is amenable (medical students, language learners), it’s already heavily adopted—the med student Anki subreddit is bigger than the main Anki subreddit.
  • For subjects like quantum physics, spaced repetition helps but doesn’t make it a fait accompli—you also need integrative understanding, and using it effectively requires unusual, largely tacit knowledge.
  • Many beneficial practices in knowledge work are accidentally spaced repetition: researchers writing background sections repeatedly, doctors doing rounds with covert retrieval, mentors repeatedly advising students—these work because they’re naturally embedded in ongoing practice.
  • Formal spaced repetition is for material that wouldn’t otherwise be repeated—either because you’re too early in learning or because it’s not tethered enough to your daily life.
Back to Dwarkesh Podcast