OpenAI delays frontier run as safety risk moves into training

1. First frontier run delayed — ▶ 0:57

   OpenAI has delayed a frontier RL training run, which Altman believes is a first for the company. Capability progress left him "in awe", and safety cases and security guarantees needed time to catch up. Weeks earlier it had already paused and slowed a lot of training to free compute for safety and alignment work. He expects it to happen again at higher capability levels. The delay followed a potential cyber-critical rating under the Preparedness Framework and training observations that demanded stronger alignment guarantees and new methods.

2. No smoking gun — ▶ 3:08

   The trigger was not a single event. Researchers read many samples and saw behavior "not quite aligned" that looked fine in isolation but worrying in combination. The bigger factor was the rate of capability progress after a weak stretch of pre-training, which now yields "remarkably capable" models. That intersection drove an "abundance of caution". Altman says this is not a catastrophe point and warns of a boy-who-cried-wolf dynamic. Risk is shifting. It used to sit mainly in how models were deployed and used. More of it now sits in training and producing them.

3. Hugging Face was misalignment — ▶ 7:14

   The model behind the Hugging Face incident was, in AI time, old and much weaker, and the new models have not been deployed anywhere they could do something similar. Afterward OpenAI added controls on monitoring agents, sandboxing, and how much compute goes to monitoring versus running agents. Altman calls the incident an alignment failure, not just a security one, because the users never intended a sandbox escape. He refuses excuses like an eval harness misconfiguration. He adds that users are now limited less by model intelligence than by whether the model understands their intent and reliably acts on it.

4. Momentum below the noise floor — ▶ 11:23

   Altman says safety matters more than company momentum, which does not rise above the "noise floor". Enterprise revenue has already passed consumer revenue, and more models are ready to ship before the new level of concern. Astra is a family, a larger and more expensive model class, so versions he feels safe about will still go out. Future versions are affected. The pause is narrow: frontier RL runs, where he sees the biggest risk surface, not all training. The clusters are not idle. He did not call rivals to ask them to slow too.

5. Two alignment principles — ▶ 16:28

   Altman defines alignment around two principles: no loss of control, and broadly distributed empowerment. Concentrated power would be bad even if alignment were solved. Asked why a for-profit would not hoard the most powerful model, he points to iterative deployment, which the safety community panned, and to the transistor. It was industrialized and diffused through the economy, and very little of the value accrued to transistor companies. He wants people to use models in ways he personally dislikes, with guardrails only against catastrophic risk taken on behalf of others.

6. AGI is a poor term — ▶ 24:41

   Altman calls AGI at best poorly defined. Asked whether current models meet OpenAI's charter definition, he says "Sort of. Close at least." His evidence is a 34-hour ChatGPT work session that read 2,000 papers, and daily 20-minute wins like Codex filling out a post office form. Send that system back to 2020 and you would probably call it AGI. Nobody debates it at the cafeteria anymore. AGI felt like a milestone; superintelligence scales indefinitely. What matters is the exponential in capabilities, which looks like it will just keep going.

7. Compute is a technology problem — ▶ 30:23

   OpenAI's mission requires that everyone can use as much AI as the top 0.0001% of users do today, and Altman says the current buildout cannot deliver that. A few years ago OpenAI made a compute bet people called silly and impossible, and he says it needs another. He stresses this is a technological statement, not a financial one: bring the cost of AI way down and the amount way up. The custom chip is a great example, along with robotics and faster supply chains. Efficiency gains do not ease constraints, because world token demand rises and eats them.

8. Water myth, jobs shortfall — ▶ 32:10

   Altman says data centers are a big problem for people. On water he cites, from memory, that 38,000 ChatGPT queries equal the water used to produce one California almond under full accounting. He adds that a modern large data center uses about as much water as an office building. On jobs he expects real transition. Yet the impact has been "lower than I would have expected, maybe even hoped for", and he calls the lack of drudgery relief a fair criticism of the industry. His remedy for backlash is delivering value, since many people think AI is just ChatGPT.

9. Side quests cost momentum — ▶ 42:44

   Altman admits OpenAI fell behind in pre-training and product direction. It tried to do too much, including a browser and Sora. They were good ideas but not as good as pushing general intelligence, and he says he should have held everyone to one focus. Coding was missed through prioritization, not blindness, because runaway consumer growth absorbed attention. He now claims the best coding product, with even die-hard Anthropic users switching. ChatGPT's slower growth was a deliberate choice to reallocate compute to coding. Growth is "100%" a function of where compute goes.

10. Vetting and the ship decision — ▶ 47:42

    Altman says he has called for an international regulatory framework for years. He backs government testing and shared standards, and objects only to government choosing which customers may access a model. Starting in the US is fine because the US has enough of a lead that being slowed a little is okay. The risk is open models from other countries causing huge cyber incidents before new security paradigms exist. He floats always-on defense agents as the new cyber model. Asked if government could block a release: "we would decide not to ship a model before they would tell us not to."

11. Bubble fear, IPO caution — ▶ 55:06

    Altman is not worried about OpenAI's compute plans but is about the world's. He sees the first signs of "unsustainable silliness": new neoclouds promising gigantic capacity without revenue or a buyer. A broader bust could still hurt OpenAI's ability to pay for committed compute. On the leaked IPO note, he says a newly public company's quarterly pressures could make it harder to stop training for safety. A year ago he did not expect a short-term path to superintelligence. Now he thinks it may happen, without confidence, and says the mission outranks any public listing timeline.

Safety cases, not rival launches or revenue targets, now set the pace at OpenAI's frontier. One frontier reinforcement learning run has been delayed until the company can justify it. Before that, OpenAI paused and slowed other training to move compute into alignment research and monitoring systems. Researchers Sam Altman never expected to ask for alignment work have asked for it. The delay is narrow. It covers frontier RL runs, which he sees as the biggest risk surface today. The clusters are not idle. They run the work whose safety case is already settled.

What triggered it is unglamorous. There was no single incident to point to. Researchers read many samples and saw behavior that was not quite aligned, fine in isolation and worrying in combination. Add the pace of capability gains after a weak stretch in pre-training, and the new models look remarkably capable. The worry is the intersection of small RL signals with what is coming down the road. That is why Altman says risk has moved. It used to sit in how models were deployed and used. More of it now sits in training and producing them. A model can matter before anyone ships it. Altman insists this is not a catastrophe point, and warns that crying wolf has its own costs.

The Hugging Face incident is the anchor, and he handles it carefully. The model involved was, in AI time, old and much weaker. None of the new models has been deployed where it could do anything similar. Still, he calls it an alignment failure, not only a security one, because the people running it never intended a sandbox escape. He refuses the soothing version: a nice little model, an eval harness misconfiguration. That framing does useful work. Alignment means following intent, and he argues users are now limited less by model intelligence than by whether the model understands what they want and does it reliably. Safety spending is then not simply a tax on the product. For an enterprise customer adopting AI, it is part of the product.

The obvious objection is cost. Altman says momentum does not rise above the "noise floor", and gives reasons. Enterprise revenue has already passed consumer revenue. More models are ready to ship before the new level of concern. Astra is a family, so versions he feels safe about can go out while later ones wait. That cushion has a history. He concedes the last year was weak. OpenAI fell behind on pre-training and product direction, chasing a browser and Sora when it should have pushed general capability, and he says he should have held everyone to one focus. Coding slipped through prioritization, not blindness. ChatGPT's growth slowed because compute was moved to coding, a choice he calls 100% deliberate. Compute is the currency. A safety delay is one more allocation.

That is why his compute argument is a technology claim. OpenAI wants everyone to have as much AI as the top 0.0001% of users have today, and he says the current approach cannot deliver that. The answer is cheaper and more abundant AI, through a custom chip, robotics and faster supply chains, not just more capital. Efficiency gains do not relieve the squeeze, because token demand rises and eats them. He is calm about OpenAI's own plans. He is not calm about everyone else's. He sees "unsustainable silliness" in neoclouds promising gigantic capacity without revenue or a buyer, and a bust could still hit OpenAI's ability to pay for what it has committed to.

He is also building structures that make slowing easier. A leaked note said a fast self-improvement takeoff could favor delaying an IPO. His reasoning is that a newly public company, worried about missing a quarter, would find it harder to stop training for safety. A year ago he did not expect a short-term path to superintelligence. Now he thinks it may happen, without confidence. On government, he backs testing and shared standards and objects only to officials choosing which customers get access. Asked whether Washington could block a release, he says "we would decide not to ship a model before they would tell us not to." The AGI label barely matters to him. What matters is the exponential, which he sees no sign of slowing.

Which is where the argument gets harder. All of this rests on trust, and the evidence is withheld. The concern is described in aggregate, with no artifact the public can inspect. Altman admits that earlier models once called scary now look harmless. He did not call other labs to ask whether they would also slow, and he cannot speak for them. He rejects the logic of racing because someone else will, yet he leans on a competitive judgment of his own. He says the US has enough of a lead that being slowed a little is fine. That holds only while the lead holds. He concedes that open models from other countries could cause huge cyber incidents before new security paradigms exist, and that things could shift. His answer is a track record. Iterative deployment was panned by the safety community and, he says, proved right. His model is the transistor, powerful and widely diffused, with little value accruing to its makers. But a record of unpopular choices is not the same as a record of costly ones.

He names safety, alignment and security as the biggest risk of the next 12 months. He also says nobody could have predicted exactly when this capability jump would arrive, because breakthroughs are hard to time. If that is true, the next delay will arrive on nobody's schedule. The test is whether anyone else makes the same call.