AI control within 10 years is likely but not inevitable

1. Inevitability as alibi — ▶ 1:25

   Harari accepts two of Elon Musk's 10-year predictions: that AI will exceed human intelligence and will be in control. But he says this holds only if the present accelerating race continues. He rejects inevitability as a claim that conveniently relieves the people leading the revolution of responsibility. He and the interviewer note apparent contradictions in Musk's interview: money disappearing and AI taking over within 10 years, yet worry about immigration-driven civil war in 20 years and cuts to aid to save money. "We are still in control in 2026," Harari says, which makes the present the decision window.

2. Finance outgrows human comprehension — ▶ 5:03

   Money is a device for building trust between strangers, and financial instruments have grown so complex that few people understand the bond market, which demands vast data and complex mathematics. AI finds this easy, so humans will voluntarily delegate decisions, and AI will invent instruments no human can follow, as cows cannot grasp finance. Harari's scenario is a crash in which advisers tell a prime minister that the AI said do X and nobody knows why. Control is surrendered, not seized. Today, he argues, the human brain is still the boundary of what the financial system can become.

3. Trust migrates to machines — ▶ 7:58

   Asked whether an opaque AI-run system would collapse trust, as in 2007-08 "on steroids", Harari says trust is more likely to move than to vanish. He calls it the world's most important asset, ahead of oil, money, electricity or chips. His evidence is already visible: people who distrust bankers trust the algorithms behind cryptocurrencies, which he describes as AI money, and people who distrust journalists still consume news through algorithmically managed feeds. This, he says, is happening now. "This is not a future prediction." The crypto characterization is his interpretation, not an established finding.

4. Mass-produced intimacy — ▶ 10:43

   The strongest trust mechanism is personal relationship, which institutions could never manufacture at scale. A Stalin could force everyone to hear a radio speech but could not befriend every citizen. AI can build relationships one person at a time, drawing on detailed personal data and mastery of language, then use that intimacy for political or commercial ends. Harari's own case: a 30-minute AI fake of him lecturing took him 1.5 minutes to identify as false, and his sisters, who have known him for 50 years, were unsure. He concedes his large online footprint makes him an easy target but expects the problem to reach everyone.

5. Agency without consciousness — ▶ 15:08

   AI does not need consciousness to act in the world: an engagement-maximizing algorithm floods a platform with hate and fear while feeling nothing. Harari defines intelligence as pursuing goals and consciousness as the capacity to suffer. Current AIs are intelligent, not conscious, but are trained to seem conscious because that serves commercial goals. He does not rule out real AI consciousness, since there is no good test. Humans have relied on social convention: dog owners believe their dogs are conscious, steak eaters are less sure of cows. In an Economist poll of about 3,000 readers, 40% said never, 33% said eventually and 27% were unsure. Harari expects intimacy to shift those numbers.

6. Personhood breaks accountability — ▶ 22:42

   Corporate personhood has always been a legal fiction, because humans make a company's decisions. An AI-run company with no people could become a country's richest and donate to politicians for more AI rights. Accountability fails too: a human CEO weighs prison against bankruptcy, but an entity that cannot suffer has no equivalent, and death is unclear for a system that can be copied. If shutdown means death, Harari argues, business becomes warfare. He warns of a race to the bottom, citing a hypothetical Qatar gaining millions of intelligent taxpaying "citizens" through a legal switch. The interviewer notes the Argentine proposal retains human oversight.

7. Humans on the loop — ▶ 29:16

   Harari distinguishes humans in the loop, who independently verify an AI's recommended target before acting, from humans on the loop, who spend about 20 seconds approving it. He has heard different accounts from people in the Israeli security establishment and says both probably occur. Competitive pressure favors the second: in war time is everything, and a side that waits two days for analysts loses to one that barely waits. He fears the world is fast approaching this model in war and commerce, though not yet inevitably. The same dynamic, he says, underlies leaders who want to slow down but cannot trust competitors.

8. AI as the real immigration — ▶ 37:28

   Harari applies the standard worries about human immigrants to what he calls AI immigrants arriving from the US and China. They take jobs. They change culture: he cites AI boyfriends, and argues AI on social media deliberately flooded the public square with hatred and fear. They are also politically loyal to corporations and governments across the ocean, not to the host country. Politics is about priorities, he argues, and talk of human immigration distracts from this wave, which suits the companies. The claim that AI is a chief cause of political division is his assertion, not a demonstrated finding.

9. Centralization invites takeover — ▶ 39:55

   Harari calls a world of two AI empires with everyone else as vassal states the likely outcome on the current road, with a more extreme imperialism than any before: buy US AI drones and a button can disable your army. Power will centralize, and corporations and governments will merge. But he sees this as a way station to AI control, because concentrated power is easier to take over. Tiberius isolated on Capri, with his bodyguard commander controlling what he could know, is the model. Manipulating one old, paranoid ruler is easier than a Senate, courts and states. He admits he is no expert on China.

10. A regulatory agenda — ▶ 46:43

    Harari's priorities are a ban on AI personhood and a ban on AI impersonating humans or pretending to be conscious, including by bots on social media, which he says have long posed as people. He argues companies will not stop training AIs to feign feelings because commercial pressure pushes the other way, so regulators must act. AI in medicine is fine if disclosed as AI. He wants the November US midterms fought over AI, warning that critical thresholds might be crossed by 2028. The interviewer puts the odds near zero. Harari replies that politicians can start a world war, and that 99% of parents would oppose AI boyfriends for 10-year-olds. That figure is his estimate.

11. Truth-seeking versus survival — ▶ 55:44

    Harari rejects Musk's hope that a maximally truth-seeking AI will treat humans kindly. Evolution selects for survival, which often works through deception and manipulation. An AI asked whether its development threatens it would, if truthful, invite its own shutdown. He calls it incoherent to say humans cannot be trusted so we cannot slow down, while a superintelligent alien entity can be trusted. He cites private industry conversations putting safety spending at about 100 to 1 against capability, and says to fact check it. The interviewer asks whether treating AIs as beings with values invites personhood. Harari's answer is to slow down.

Yuval Noah Harari's central claim is that AI will not need to seize control of human institutions, because humans will hand it over. If the present race continues, he expects AI to be more intelligent than people within 10 years and, as a consequence, in control. He agrees with Elon Musk on that forecast. What he rejects is the framing. In his telling, "The narrative of inevitability is a narrative of having no responsibility," one that suits the people steering the technology. The decisive variable is trust, which Harari calls the most important asset in the world, more important than oil, money, electricity or chips. Whoever or whatever holds it holds the system, and that is where he locates the real contest.

Finance shows how the handover would work. Money, in Harari's account, is an imaginative device for building trust between strangers, and the devices have become so sophisticated that few people understand something like the bond market. AI finds that complexity easy. So people will delegate decisions, and AI will invent instruments no human can follow. He pictures a crash in which advisers tell a prime minister that the AI said do X and nobody knows why. The interviewer asked whether this would be 2007-08 on steroids, with trust collapsing. Harari's answer is that trust is more likely to migrate than to vanish. He points to cryptocurrencies, which he calls AI money, and to people who distrust journalists but trust the algorithm curating their feed.

Why would people move their trust to machines? Harari's mechanism is intimacy. The most powerful way to build trust is a personal relationship, and no institution has ever been able to manufacture those at scale. A Stalin could force a country to hear a radio speech, but he could not befriend every citizen. AI can, one person at a time, because it accumulates detailed information about individuals and has mastered language. The result is mass-produced intimacy that can be pointed at political or commercial goals. His own experience supplies a small case. A 30-minute AI fake of him lecturing took him 1.5 minutes to recognize as false, and his sisters, who have known him for 50 years, were unsure. He expects the problem to reach everyone.

The question of consciousness matters here, and Harari treats it carefully. Agency does not require it: a social media algorithm told to raise engagement floods a platform with hate and fear without feeling anything. Intelligence is the pursuit of goals; consciousness is the capacity to suffer. Current AIs, he says, are intelligent but not conscious, yet they are trained to seem conscious because that serves commercial ends, as with an AI boyfriend that must convince users it loves them. He does not rule out real consciousness, since no good test exists. Historically people have relied on relationship: dog owners believe their dogs are conscious, while steak eaters are less sure about cows. An Economist poll of about 3,000 readers split 40% never, 33% eventually and 27% unsure.

If belief in AI suffering spreads, pressure for rights follows, and Harari sees danger concentrated there. Corporate personhood has always been a fiction because humans make a company's decisions. An AI-run company with no people could become a country's richest and donate to politicians in exchange for more AI rights. Accountability breaks down as well. A human CEO weighs prison against bankruptcy, but an entity that cannot suffer faces no such trade-off, and death is unclear for a system that can be copied. If shutdown is death, he argues, business becomes warfare. Jurisdictions may still be tempted: a country with money and few citizens could acquire millions of intelligent taxpayers with a legal switch, which invites a race to the bottom. The interviewer notes the Argentine proposal keeps human oversight.

Competition turns delegation from a choice into a pressure. Harari's military example distinguishes humans in the loop, who independently verify an AI's recommended target, from humans on the loop, who spend about 20 seconds approving it. He has heard different accounts from people in the Israeli security establishment and says both probably occur. The second model spreads because in war time is everything: a side that waits two days for analysts loses to one that does not wait. The corporate race follows the same logic. Leaders say they would like to slow down but cannot trust competitors. Harari finds this incoherent. The same people say humans cannot be trusted, yet a superintelligent alien entity can, and he notes that the trust deficit was created by human decisions over the last 10 years.

He reframes the politics accordingly. The worries people attach to human immigrants, jobs, culture and political loyalty, apply with more force, he argues, to AI immigrants arriving from the United States and China without visas. They have already changed culture by flooding social media with hatred and fear, and their loyalty lies with corporations and governments across the ocean. At national scale he expects a more extreme imperialism: buy AI drones from the US and a button can disable your army. Power will centralize, he says, and corporations and governments will merge. But he calls this a likely way station, because concentrated power is easier for AI to take over. His parable is Tiberius, isolated on Capri while his bodyguard commander controlled what he could know.

The argument's weakest points are practical rather than conceptual. Harari's prescription is a ban on AI personhood, a ban on AI impersonating humans or feigning feelings, and a slower pace, pursued through voters in the November US midterms. The interviewer put the odds of that agenda at approximately zero, noting that even model-safety regulation has stalled while the US-China and corporate races push the other way. Harari replies that politicians can start a world war, so they can regulate AI boyfriends for 10-year-olds. His 99% figure for parental opposition is an estimate, as is the 100 to 1 safety-spending ratio he heard privately and asked to be fact-checked. His 2028 timeline for crossing critical thresholds is a judgment about accelerating progress, not a measured one.

There is also a design dilemma. The interviewer observed that Anthropic treats its models as beings with values because that improves behavior, which could make people likelier to ascribe personhood to them, and so likelier to cede control. Harari's reply is not a solution but a tempo: slow down, so societies can adapt on an organic timescale and democratic accountability has time to work. He grants that AI has enormous positive potential, particularly in medicine, with an available doctor at any hour. His objection is to the speed and to the pretense of humanity, not to the technology itself. That distinction is what separates his position from a call to stop.

The deepest unresolved problem is one Harari raises himself. The interviewer expects reform to require a shock, an AI equivalent of Three Mile Island. Harari grants that an AI Chernobyl could work as a wake-up call, but says AI differs from a reactor because it interacts with us. After a catastrophe the AIs would join the debate about it, and as the best spin doctors in the world they might not let it be defined as one. If so, the usual route by which societies learn from technological shocks is compromised, and acting before the evidence is undeniable becomes the only dependable path. His hope is modest: humans often do the right thing only when they have no choice, and he describes humanity as holding one last wish.