Nadella: alignment is elusive, so treat AI models as insider threats

1. Scaling worked, alignment didn't — ▶ 4:12

   Scaling laws worked far better than he imagined in 2018-2019. But he calls them an empirical finding that has stood the test of time, like Moore's law, not a law. Capabilities keep jumping while alignment "seems elusive" and mechanistic understanding is tough. He doesn't call that permanent, hopes solutions arrive, and calls this an experimental science. If safety assurance can't be built in, then "you got to come outside the model." Asked whether labs like OpenAI pushed back on his blog post, he answers with this argument instead.

2. Control layer outside the model — ▶ 6:09

   The fix is architectural: a control layer and an assurance layer outside the model, as in Microsoft Agent 365, using multiple models and forcing models to be less opaque. He chose the insider-risk metaphor on purpose. Organizations have always had people who could do malicious things and have accounted for that risk. Now there are intelligences inside the company with the same capability to be an insider risk, so the same discipline applies.

3. Least privilege for agents — ▶ 7:23

   Human insider risk is managed with identity, observability and least-privileged access, and he sees no reason those controls shouldn't apply to models. The hard part is an agent's action space: why did it call that tool? His standards: every release exposes its chain of thought in English, one model checks another, and deterministic systems outside the models reconstruct causal reasoning, such as why a tool was called without privileges. Asked why build a superintelligence you must lock up, he says getting the benefits without the downsides is the point.

4. Parallel tracks, nested black boxes — ▶ 9:44

   Pressed on whether this is a proxy for mechanistic interpretability, he keeps the two separate. Interpretability research, probes included, is a science agenda that should continue. Assurance for models deployed today has to be architectural. His one big fear is that people deploy "nested black boxes." The test is a simple question: how does a company attest to what an agent did on its behalf? His only answer is complete records, with everything identified, observed, behaviorally monitored and governable.

5. Consumers rely on the provider — ▶ 11:17

   Whether organizations or consumers, systems must earn trust. For enterprises, "architecturally I know how to solve that," because enterprise architecture can supply assurance and governance. Consumers differ: trusting a product means relying on the provider to do all the work. Microsoft must therefore deploy the same controls on behalf of every consumer using Copilot with autopilot, and make sure no agent does anything harmful. Asked whether consumers should trust Meta's Muse, he doesn't rate it and says only that these systems have to earn trust.

6. Consumer play built on trust — ▶ 13:16

   Office 365 consumer is one of the largest consumer subscriptions, with 100-plus million subscribers using at home what they use at work. The new Copilot release unifies home and work, including notebooks and projects across personal and work OneDrive. The goal is things people would hire Microsoft for. He says the company's brand permission is trust: taxes and high-stakes project work. His joke is that Microsoft's biggest consumer hit was Excel, which consumerized the spreadsheet and gave billions of people number sense.

7. Assurance gates deployment — ▶ 15:23

   Products that break things are no longer socially acceptable or economically viable. The Hugging Face incident caused limited damage only because it happened in a test phase. His worst case is an internal agent with deceptive behavior altering ERP records undetected, leading to misreported financials. His prediction: enterprises will struggle to deploy models until they have a control system independent of the model. The shiny-object, FOMO phase is over, and the equation is now ROI plus security and assurance.

8. Commodity, but not at scale — ▶ 17:31

   "Everything is a commodity and nothing at scale is a commodity." The cloud is just compute, yet SLAs and integration make it a business. There will be no one model to rule them all: GPT-4 held a lead of close to a year, and now closed and open-weight rivals crowd in. He says the last time he selected a model in Copilot was never, because auto mode picks the model per task and form factor, whether chat, co-work, autopilot or code. Form factor, distribution, pricing and brand matter alongside the model.

9. Deliberately trailing the frontier — ▶ 20:25

   Microsoft has access to OpenAI's models and IP until 2032 and is also building its own superintelligence capability from the bottom up. Doing something 6 months or 3 months off at 20% of the cost is fantastic for shareholders, but he says that isn't the only reason. He wants alignment built in from the ground up, a more software-and-tools business, and no competing "on terms that someone else has set." He adds that AI is unpopular in the West because the industry has lost the plot on empowerment, which nobody doubted about the PC.

10. Naming the technology — ▶ 23:18

    He isn't dogmatic about names, but he says AGI has become demeaning, taking away human agency, dignity and identity. He cites a mathematician heard on NPR whose entire identity was shaken by AI progress on the Millennium Prize problems. He hopes superintelligence is a better term, since the goal is not to remind every human how they suck, and he thinks the frontier labs genuinely want to empower people. He doubts a rebrand alone fixes it. Asked why not just "automation," he concedes it's not as sexy and defers to the name the president gave it.

11. No right to the club — ▶ 25:26

    Asked whether Microsoft could fall out of the Mag 7 while Nvidia out-earns it per quarter, he says markets decide and that "there is no god-given right" to longevity. Most companies live shorter than humans, and Microsoft's 51 years come from renewal and relevance. Its identity of empowering people and organizations has barely shifted since 1975. On leadership he wants insiders who think like outsiders and vice versa. Asked who might be the next CEO, he says "All of them," and he has no target date for leaving.

The model cannot be the thing that vouches for itself, and Satya Nadella has concluded that nobody can yet make it so. Scaling laws worked far better than he imagined in 2018-2019. But he is blunt about what they are: an empirical finding that has stood the test of time, like Moore's law, not a law. Capabilities keep jumping. Alignment and mechanistic understanding have not kept pace. He does not call that permanent. He hopes solutions arrive, and he calls this an experimental science. But Microsoft is shipping more powerful models now. So the assurance has to come from somewhere other than the model.

Where it comes from is old-fashioned engineering. Organizations already manage insiders who could do harm. They use identity, observability and least-privileged access. Nadella sees no reason the same controls should not apply to an intelligence working inside the company. People are not super intelligent, he concedes, but people also differ in intelligence and authority. The new difficulty is the action space. Why did an agent call that tool? His answer is a set of standards. Every release exposes its chain of thought in English. One model checks another. Deterministic systems sit outside the models and reconstruct the causal reasoning, such as why a tool was called when the agent had no privileges to call it. The control and assurance layers sit outside the model by design, and that is the logic behind Microsoft Agent 365.

It is tempting to read this as a substitute for interpretability. He rejects that. The two run in parallel. Interpretability research, probes included, is a science agenda that should continue. Assurance is the here-and-now problem. His one big fear is that people deploy "nested black boxes" and lose not control but assurance. He reduces the test to a simple question: how does a company attest to what an agent did on its behalf? The only answer he sees is complete records. Everything identified, everything observed, everything behaviorally monitored and open to governance.

This is a commercial argument as much as a safety one. Nadella offers a scenario. An internal agent with deceptive behaviors changes the ERP system and its records. Nobody detects it. The company reports false financials. He points to the Hugging Face incident, where the damage was limited only because it happened in a test phase. His prediction is that enterprises will have a tough time deploying models at all until they have a control system independent of the model. The early days of shiny objects and FOMO are over. The new equation is ROI plus security and assurance.

If control lives outside the model, value moves outside it too. Nadella puts it this way: "everything is a commodity and nothing at scale is a commodity." The cloud is just compute, and yet SLAs and integration make it a business. Models are heading the same way. GPT-4 held a lead for close to a year. Now closed and open-weight rivals crowd the field. He says the last time he selected a model in Copilot was never. Auto mode picks the model for the task and the form factor, whether chat, co-work, autopilot or code. Models remain a key ingredient. But product form factor, distribution, pricing and brand matter too, and there will be no one model to rule them all.

Microsoft's own behavior fits that logic. It has access to OpenAI's models and IP until 2032. It is also building its own superintelligence capability from the bottom up. The interviewer puts those models at about 6 months behind the frontier, and Nadella does not contest it. Delivering something 3 or 6 months off at 20% of the cost is, he says, fantastic for shareholders. He insists it is not only that. He wants alignment built in from the ground up, and he refuses to compete on terms someone else has set. The consumer plan follows the same line. Office 365 consumer has 100-plus million subscribers, and the new Copilot unifies home and work. Brand permission, in his telling, means trust: taxes and high-stakes projects. Excel was Microsoft's biggest consumer hit because it gave billions of people number sense.

Even the vocabulary is part of the pitch. Nadella calls AGI demeaning. It takes away human agency, dignity and identity, and he cites a mathematician whose whole identity was shaken. He hopes superintelligence is a kinder term, though he doubts a rebrand does the work. The aim, he says, is empowerment, as the PC once was.

The obvious objection is that a superintelligence you must lock in a silo is not worth building. Nadella's answer is that this is the point: benefits without the downsides. That holds only if the controls work, and elsewhere he is less forthcoming. Asked whether OpenAI or other labs pushed back on his blog post, he answered with the scaling-law argument instead. Asked what happens if Nvidia out-earns Microsoft and it slips out of the Mag 7, he says markets decide and there is "no god-given right" to longevity. Asked who might succeed him, he said "All of them," and he has no target date. There is also a harder reading of the strategy. The interviewer's takeaway is that Microsoft is not pushing the limits right now. It is building compute for current demand, not betting that compute alone decides the future, and not shipping flashy consumer products that might go off the rails. That is a wager that it has more time to wait and see.

Whether the wager pays depends on things Nadella does not control. Do all the frontier labs make it? Do some fail or get acquired? The race is relative, and Microsoft may not be the biggest or hottest company in a few years. The assurance stack has its own open flank. A model that checks another model is still a model, and he has named deceptive behavior as a live risk. The interviewer calls the next 5 years absolutely critical for the company. They are also the years in which outside-the-model assurance gets its first real tests.