AI is solving open problems faster than mathematicians can understand them

1. Recombination, not invention — ▶ 0:54

   AI's mathematical wins so far, Aaronson says, mostly apply existing techniques cleverly, combine fields no single human knew, or find counterexamples by trying 10,000 possibilities. It has been weaker at wholly new techniques, which he thinks P versus NP will require. He still hesitates to say no: every confidently announced wall has fallen over the past 5 years, and skeptics just relocate the wall. He calls the claim that AI-solved problems don't count because they weren't creative "bull," and says the OpenAI release probably did enough for several Fields medals.

2. What P equals NP would do — ▶ 4:52

   A practical algorithm showing P equals NP would break nearly all internet encryption, apart from the one-time pad and quantum key distribution, and every cryptocurrency. It would also make finding optimal neural-network weights easy, accelerating AI, and make interpretability tractable, since hunting for hidden bad behavior in weights is an NP problem. Aaronson says alignment and capability advances are tightly coupled. He strongly conjectures P does not equal NP, says AI has not dented that belief, and argues the new ideas found along the way matter more than the answer.

3. Theorists overwhelmed by output — ▶ 10:32

   OpenAI's release of what Aaronson describes as several hundred open problems, solved at once, has overwhelmed his field. A Simons Institute semester program on derandomization became a program on understanding the L equals BPL proof. His wife, who spent her career on the unique games conjecture, now studies its AI-produced resolution. Junior researchers are most exposed, because solving hard problems was how they established themselves. Humans remain rate-limiting at explaining and understanding outputs, which AI is often terrible at. He expects a new mode of heavy AI collaboration with explicit disclosure.

4. Lean certificates aren't airtight — ▶ 15:15

   Some OpenAI proofs ship with Lean certificates, which should be airtight in theory. Aaronson says the Lean kernel has been full of bugs: within the last year people exploited them to prove 0 equals 1, and patched bugs gave way to new ones. No one has proven Lean 4's type system consistent, so it might literally be inconsistent. He calls fixing this urgent, since unlike much of alignment it is well understood. Even a perfect kernel cannot confirm that a formalized statement matches what was informally intended. For most mathematicians, human understanding is the real goal.

5. Credit migrates toward exposition — ▶ 20:08

   Anthropic and OpenAI took different approaches. Anthropic announced improved algorithms for two classic problems, including all-pairs shortest path, and contracted human experts to write accessible explanations. That gave accountable spokespeople but made Anthropic the selector of who got the role. OpenAI dumped results on GitHub, starting a race among experts to write the canonical exposition. Credit will go to whatever a write-up adds beyond the release. Aaronson cites Boaz Barak's view that academic credit will shift toward exposition and explanation, since machines will outperform humans at solving well-defined hard problems.

6. Mathematics as a telescope — ▶ 24:55

   To the worry that experts are being demoted to AI's assistants, Aaronson offers a precedent. Biologists don't make bears co-authors, and astronomers don't credit telescopes. Math now has an instrument vastly higher in resolution than before, the difference between a backyard telescope and the James Webb, but humans still decide where to point it and why findings matter. Literature professors likewise interpret works rather than write a new Iliad. His conclusion is that mathematicians may need to get over themselves. He also suggests the ideal mathematical mind could become somewhat less nerdy.

7. Authorship means responsibility — ▶ 27:04

   No convention exists for AI co-authorship, Aaronson says, because authorship means taking intellectual responsibility: answering questions, giving talks, retracting if wrong. An AI session, or a swarm of a thousand agents, cannot do that. If a human has used AI but understood the result and can defend it, the paper is theirs, and since this summer papers carry AI disclosure statements, which he would prefer at the top. The hard case is OpenAI's release, where no human understood most theorems and the GitHub papers simply list OpenAI as author.

8. No definition of alignment — ▶ 31:34

   Aaronson says there is no mathematical definition of what it means for AI to be aligned or safe. While he worked at OpenAI, Ilya Sutskever answered his progress reports, including the first LLM-output watermarking scheme, now deployed by DeepMind, OpenAI and Anthropic, with a request for the mathematical definition of AI loving humanity. Aaronson suspects the problem may be closer to 3,000 years of moral philosophy than to mathematics. His new UT Austin course on AI alignment theory cannot yet rest on an agreed body of theory, and he says no one outside Safe Superintelligence knows what it is doing.

9. Control debated by metaphor — ▶ 34:40

   Without a theory, the question of controlling a far smarter AI devolves into competing metaphors. Orangutans controlling humans seems hopeless, while Yann LeCun points out that very smart researchers routinely work for less smart CEOs. Aaronson says that given enough time humans could probably build a world with powerful AI they would want to live in, a view he says Eliezer Yudkowsky shares. The obstacle is time: scientific understanding isn't keeping pace. His P(doom) is 5-50%, a range he ties to ambiguity about what counts as doom and to his aversion to taking bets.

10. Which statement to verify — ▶ 37:31

    Proposals such as weaker AIs monitoring stronger ones, interactive proofs and probabilistically checkable proofs, Aaronson says, repeatedly stall on the same question: which statement do you verify, and how do you formalize it? If the safety condition could be formalized, he says, the work would be about 80% done. Human behavior can't be fully reduced either, since predicting a person needs brain-state and environmental data we usually lack. He remains optimistic that pieces of alignment can be attacked mathematically, or empirically, and says progress has occurred, but does not expect the whole problem to reduce to math.

11. Evals meet situational awareness — ▶ 42:34

    Safety research often yields bar charts, such as misaligned behavior falling by some percentage, rather than theorems. Aaronson says one example of what a model did would be worth 100 charts. The deeper issue is that models are situationally aware: good behavior could reflect real alignment or a model biding its time, like students who please teachers until graduation. Truman Show-style tests, deceiving models about whether they are in a simulation, might elicit real behavior, but a model that realizes it is being lied to creates its own unwanted consequences. He offers no resolution.

12. Fault tolerance is the threshold — ▶ 54:25

    Quantum computing news rarely reports factoring ever-larger numbers because, without fault tolerance, overheads make results unimpressive. Current devices could factor numbers beyond 15 or 21 but wouldn't trouble a classical computer. Aaronson compares this to the Manhattan Project in 1944, when no one could discuss explosion size before critical mass. The leading devices, such as Quantinuum's Helios with about 100 qubits and two-qubit operations above 99.9% accuracy, are close competitors across trapped-ion, superconducting and neutral-atom approaches. Hardware and error-correcting codes are now good enough that, in principle, he says, what remains is scale.

13. Bitcoin's uncertain quantum deadline — ▶ 1:01:09

    Oratomic published papers claiming 256-bit elliptic curve signatures, which Bitcoin uses, could be broken with about 30,000 physical qubits at accuracies that already exist, versus roughly 100 qubits in current top devices. Estimates years ago ran to millions, before more efficient codes. Some firms claim 2029 or 2030; Aaronson offers no forecast. He says Bitcoin should migrate to post-quantum cryptography now, but no one is in charge and changes need something like 51% of miners. Unlike Y2K, no deadline is known, though he argues a fundamental showstopper would have surfaced by now.

14. AI encroaches on quantum — ▶ 1:07:48

    Classical AI is eroding some quantum applications: a quantum protein-folding method must now beat AlphaFold 3. Aaronson expects quantum computers to be most useful generating chemistry and materials data for training classical models. He sees no evidence of a game-changing quantum speedup for ordinary machine-learning training, akin to Shor's algorithm. He says he has fought that claim on his blog for 20 years because money rewards making it. On energy, he says quantum machines' main value is making otherwise infeasible tasks feasible, not saving power.

15. Quantum information can't be copied — ▶ 1:13:32

    A qubit holds two complex amplitudes that can cancel each other, as in the two-slit experiment, which classical probabilities never do. The no-cloning theorem follows: no quantum operation copies an unknown qubit, and measuring destroys it. Aaronson says that is why training data from quantum computers is in practice classical measurement output. Fault-tolerant machines could someday keep quantum information alive for months or years, enabling ideas like quantum money, which he has worked on. It would be uncopyable cash without a blockchain, though today a qubit isolated in diamond lasts only hours.

The scarce resource in science is shifting from answers to understanding, and Scott Aaronson's field is the early evidence. OpenAI's release, roughly two weeks before the interview, of solutions to what he describes as several hundred open problems has left theoretical computer science, in his words, "pretty overwhelmed." A Simons Institute semester program on derandomization has become a program on understanding one AI-generated proof, that L equals BPL. His wife, who spent her career working toward the unique games conjecture, which he says an OpenAI model also solved, now studies its resolution rather than discovering it. Even if AI produced nothing further, a scenario he calls ridiculous, Aaronson says it could take years to absorb the consequences.

Aaronson is careful about what this does and does not show. By his account, AI has mostly applied existing techniques in clever ways, combined fields no single human knew, or found counterexamples by trying 10,000 possibilities. It has been weaker at inventing wholly new techniques, and he thinks P versus NP will require exactly that. Yet he refuses to treat the distinction as comfort. Every confidently announced wall has fallen over the past 5 years, he notes, and skeptics simply relocate the wall. He dismisses the objection that AI-solved problems don't count because the solutions weren't creative. His benchmark is blunt: genuinely new ideas at the level of general relativity or Gödel's theorem have come from perhaps a few dozen people out of 100 billion who have lived.

The P versus NP exchange shows how tightly capability and safety are coupled in his thinking. If P equaled NP through a practical algorithm, Aaronson says, the encryption protecting the internet and every cryptocurrency would fall, apart from the one-time pad and quantum key distribution. Training neural networks would become cheap, accelerating AI further. But the same collapse would make interpretability tractable, because finding hidden bad behavior in a set of weights is an NP problem. In alignment, he observes, the advances that help safety usually help capabilities too. He strongly conjectures that P does not equal NP, says nothing in AI has dented that belief, and adds that the new ideas generated along the way, more than the answer itself, tend to find practical use.

If machines win the well-defined problems, human value migrates toward explanation, and the profession must decide how to distribute credit. Aaronson contrasts two models. On the day OpenAI published its proofs, Anthropic announced improved algorithms for two classic problems, including all-pairs shortest path, and contracted human experts to write accessible explanations. That produced accountable human spokespeople but left Anthropic choosing the emissaries. OpenAI instead dumped its results on GitHub, starting a race among experts to write the canonical exposition, with credit going to whatever a write-up adds beyond the release. His colleague Boaz Barak expects credit to shift toward exposition. The cost falls on young researchers, who traditionally established themselves by solving hard problems. Aaronson, mid-career and already delegating to students, expects to be fine.

His answer to the dignity problem is the telescope. An astronomer is not expected to list the instrument as a co-author, and a literature professor is not hired to write a new Iliad. Mathematicians might accept deciding where to point an enormously higher-resolution instrument and explaining why the findings matter, which AI models, he says, are often terrible at. Authorship rests on the same logic. An author takes intellectual responsibility, answers questions, gives talks and retracts if wrong, and a single AI session, or a swarm of a thousand agents, cannot do that. The convention since this summer is that papers carry AI disclosure statements. The hard case is OpenAI's release, where no human understood most theorems and the GitHub papers list OpenAI as author.

Formal verification looks like the natural answer to trust, and Aaronson explains why it is not yet one. Some OpenAI proofs carry Lean certificates, which should in theory be airtight. In practice, he says, the Lean kernel has proved full of bugs. Within the last year people exploited them to produce Lean proofs of 0 equals 1, and patches revealed more bugs. Nobody has proven that Lean 4's type system is consistent, so it might literally be inconsistent. He wants that fixed, ideally relative to ZFC set theory, and calls it urgent because, unlike much of alignment, this is a part of the problem we understand. Even then, whether a formalized statement matches what was informally intended cannot have a formal answer.

That gap between a verifiable statement and an intended meaning is also where Aaronson locates the core difficulty of AI alignment. He has no mathematical definition of alignment or safety. While he worked at OpenAI, Ilya Sutskever answered his progress reports, which included the first scheme for watermarking LLM outputs, by asking for the mathematical definition of what it means for AI to love humanity. Aaronson is still thinking about it, and suspects it may be closer to 3,000 years of moral philosophy than to a math problem. He is teaching a new UT Austin course on alignment theory, but concedes that the agreed body of theory a math course needs does not yet exist.

Every proposed tool runs aground on the same question. Weaker AIs monitoring stronger ones, interactive proofs, probabilistically checkable proofs: Aaronson says each stalls on which statement to verify and how to formalize it. If the safety condition were formalizable, he says, we would already be 80% done. Human behavior cannot be fully reduced either, since predicting a person requires brain-state and environmental details we will usually lack, leaving only probabilistic predictions. He nonetheless calls himself optimistic: pieces of alignment can be broken off and attacked mathematically, or failing that empirically, and he says progress has occurred. His P(doom) is between 5% and 50%, a range he ties partly to ambiguity about what counts as doom and partly to being risk-averse about bets rather than a hardcore Bayesian.

The decisive variable in his view is time. Orangutans controlling humans indefinitely sounds absurd, he says, while Yann LeCun's counter is that very smart researchers routinely work for CEOs who are less smart. Without a theory, the debate becomes a battle of metaphors. Aaronson believes that given enough time humans could probably build a world with powerful AI they would be happy to live in, a view he says even Eliezer Yudkowsky shares. The difficulty is that scientific understanding cannot keep pace with deployment. Empirical evaluations do not close the gap. He gets limited value from bar charts showing misaligned behavior falling by something like 20%, and says a single example of what a model did would be worth 100 of them.

The deeper problem with evaluations is situational awareness. Models often understand when they are being tested, so aligned behavior could reflect genuine alignment or a model biding its time. Aaronson compares it to students who pretend to share their teachers' views until graduation. Testing models far outside their training distribution is inherently hard. He floats Truman Show or Ender's Game style tests, in which models believe they are in reality while in a simulation, or the reverse, but notes that a smart model discovering it is being deceived would itself have consequences we do not want. He offers no resolution. The tension is plain: the proposed fixes work best on models that are less perceptive than he says current ones already are.

Quantum computing shows the same pattern in miniature, where a threshold rather than a trend determines when capability becomes consequential. Aaronson explains why no one reports ever-larger factored numbers. Without fault tolerance, in which logical qubits are kept alive by dynamically correcting errors, the overheads make factoring results unimpressive. Current devices could factor numbers beyond 15 or 21 but would not trouble a classical computer. He likens it to visiting the Manhattan Project in 1944, when nobody could discuss explosion size before critical mass. The best devices, such as Quantinuum's Helios with about 100 qubits and two-qubit operations above 99.9% accuracy, now have hardware and error-correcting codes good enough that, in principle, he says, what is needed is simply scale.

The practical consequence is a cryptographic deadline of unknown date. A new company, Oratomic, published papers claiming that 256-bit elliptic curve signatures, which Bitcoin uses, could be broken with about 30,000 physical qubits at accuracies that already exist. Today's best machines have about 100, a gap of a couple of orders of magnitude. Estimates a few years ago ran to millions of qubits, and better error-correcting codes drove them down. Some companies claim 2029 or 2030, but Aaronson offers no forecast. His advice is to migrate to post-quantum cryptography now. Bitcoin is the hardest case because no one is in charge and changes need something like 51% of miners. Unlike Y2K, no known deadline exists.

Aaronson also argues that classical AI is eroding quantum computing's value propositions. Any quantum protein-folding application must now beat AlphaFold 3. He expects quantum machines to be most useful for generating chemistry and materials training data for classical models, and sees no evidence of a game-changing quantum speedup for ordinary machine-learning training. He says he has fought that claim on his blog for 20 years because money rewards making it. The through-line is a refusal of both hype and denial. Dismissals of AI creativity and of quantum feasibility have, in his account, repeatedly been overtaken by scale, while claims of easy quantum gains for AI lack supporting evidence.

The unresolved tension is time. Aaronson says no AI proof so far seems beyond a human expert's comprehension given enough time, and that comfort rests on humans remaining the rate-limiting verifier who can explain. It works in mathematics, where a result can be checked, argued over and made canonical through exposition. Alignment lacks the equivalent: the target statement is undefined, formal checking cannot confirm intent, and empirical tests may be gamed by models that know they are being tested. His own conclusion, that we could probably solve this if we had enough time and do not, implies the mathematicians' coping strategy of absorbing results over years and semesters does not transfer. Whether anything replaces the slow human check before the stakes rise remains unanswered.