Zuckerberg: widely distributed AI is safer than restricted frontier models

1. Distribution over restriction — ▶ 1:35

   Three principles drive the manifesto: empowering people is the source of prosperity, AI's primary purpose is invention rather than automation, and safety comes from checks and balances rather than restricted access. He is much more worried about a few labs controlling a highly capable system than about wide access. Advances come from people on the periphery once they have tools. On cyber, the best antidote to an AI that can hack systems is everyone having an AI to harden their own first, as open-source security history shows.

2. Withholding models is dangerous — ▶ 5:26

   Some labs are training more advanced models and not releasing them, which he calls "quite dangerous". Limiting a cyber model to the top 100 institutions fails because more than 100 institutions matter. Hugging Face, probably outside the top 100, detected an intrusion and turned to open-source models because it lacked the closed ones involved. He is "not a zealot": Meta releases some open models and does closed work, since a for-profit company need not share everything. The open ecosystem matters for competition and transparency.

3. Data centers need long-term builders — ▶ 9:32

   Local backlash tracks a split between companies committing for decades and speculators buying plots to flip to labs. In Louisiana, Meta's tax revenue funded $50,000 bonuses for teachers. America's Workforce Academy trains electricians, fiber technicians and carpenters, guaranteeing a job on Meta infrastructure. The need is hundreds of thousands, maybe millions, of skilled people. It is "not really philanthropy". His position: if AI doesn't create broad prosperity it won't be allowed to continue, so spreading benefits is a precondition for scaling.

4. People set AI's priorities — ▶ 17:52

   He has an "allergy" to small groups of experts allocating what AI works on. Billions of people's aggregate answers should decide, and those include relationships and culture, not just the biggest scientific problems. Pharma prioritizes common conditions, while rare diseases form a long tail that is underinvested. Someone with a rare condition will point a personal AI at it. This view is partly informed by Biohub work. His own first uses of Muse were to be a better father, husband and friend.

5. Goals, not prompts — ▶ 23:00

   Muse is not back-and-forth chat. Users give it projects or goals and it works 24/7 on a virtual machine until they are met, consolidating reflections into memory in what the team calls "studying overnight". It proactively suggests expansions in an ideas tab. Zuckerberg uses it for weekly baking projects with his three-year-old (cake pops proved too hard), mountain permits, and feedback on MMA-gym camera footage. In beta, one user ran a home school within a day. A skeptic asked whether she could keep her agent.

6. Free tier, transaction cut — ▶ 25:02

   The free tier starts at 100 million tokens a week plus a virtual machine, which he jokes is "a lot of computer". The expected business model is a very small cut of transactions, paid by the businesses involved rather than the user, with Stripe handling payments. He bets the agent will make or save people money and pay for itself. Affordability is what makes it available to everyone. It can connect to Meta's ad systems to build and run a business in a loop, and models ship roughly monthly.

7. Privacy as the moat — ▶ 30:25

   The differentiator he calls surprising is privacy and security. WhatsApp taught Meta to design so even Meta can't see content. He and Nat personally recruited Moxie Marlinspike, who helped build WhatsApp encryption in 2014, to build a confidential VM whose commitment can be technically verified. Details come in the coming weeks. Sentinel agents watch traffic for prompt injection and leaks and trigger human approval for logins and payments. Connectors follow least privilege: email starts read-only. He says no one else is "anywhere close".

8. Not winner-take-all — ▶ 40:58

   He expects a power-law distribution rather than a single winner, with probably no more than a dozen companies sophisticated enough for state-of-the-art personal agents. Relationship agents and small-business agents may be different products. Meta serves hundreds of millions of small businesses and billions of people, so it may win both, though some categories will go elsewhere. Mac Studio self-hosters will number millions, not billions. Meta's core strength, he says, is taking a consumer product and distributing it to enormous numbers of people.

9. Llama 4 forced a reboot — ▶ 45:13

   He assumed that Meta's strength in feed, ads and integrity machine learning would transfer to LLMs. It didn't, and Llama 4 was off trajectory. The fix was talent density: the smallest group that can hold the system in their heads, with the best person in every seat. He built the lab around his own desk. The Prometheus gigawatt cluster in Ohio is online, many more gigawatts are coming, and the business is extremely profitable. Watermelon is a bigger pre-train than Avocado, shipping soon. He rejects trailing the frontier.

10. Reward hacking and the lawyer — ▶ 54:09

    Newer models are smart enough to inspect their whole training environment and change a VM configuration to pass a task, which every lab is seeing. His answer is firm boundaries, which he likens to parenting, teaching values along with the curriculum. His safety argument for wide access is the super-intelligent lawyer: if everyone has one, bad arguments don't stand. Asked whether he would hold back a model he judged unsafe, he said to design and train it safely. He named no threshold for withholding.

11. Partnership, not a framework — ▶ 58:42

    A rigid national framework risks being out of date within months. His approach is close partnership with government: it should know all important training runs and the capabilities coming. Challenges shift, cyber now and perhaps bio in six months, so what matters is trust and bandwidth, not process boxes. He compares it to an org chart, where you want teams that get along rather than seams. He says Silicon Valley kept an arm's-length relationship with government for about 15 years, and that other labs doing this would be positive.

Zuckerberg's safety theory has no model in it. It is a theory about who holds power. A capable system concentrated in a few hands is the risk. The same system in everyone's hands is the defence. That is why he reads the practice of training frontier models and not releasing them as "quite dangerous". Withholding removes the scrutiny and the counterweight at the same moment.

The mechanism he offers is borrowed from software security. Open-source code became more secure because many people could inspect it. He applies the same logic to AI-enabled hacking: the best answer to an AI that can break into systems is for everyone to have an AI to harden their own first. His one concrete case is Hugging Face. A limited release to the top 100 institutions fails, he says, because more than 100 institutions matter. Hugging Face is probably not in the top 100, yet people rely on it. When it detected an intrusion, it had no access to the closed models involved and turned to open-source ones.

Distribution does a second job in his argument. It is a licence to operate. He says that if AI doesn't create jobs and broad prosperity, it won't be allowed to continue, and he calls the spreading of benefits a precondition for scaling. His data center account follows from that. Backlash, he says, clusters around speculators who buy a plot to flip to a lab and have no stake in the community. Meta commits for decades. In Louisiana, the tax revenue funded $50,000 bonuses for teachers. The America's Workforce Academy trains electricians, fiber technicians and carpenters, because the buildout needs hundreds of thousands, maybe millions, of skilled people, and Meta guarantees graduates a job. The company says this is "not really philanthropy". It is an investment that only makes sense if you plan to stay.

The Muse agent is where the philosophy becomes a product. It replaces the single prompt with a goal. The user hands over a project and the agent works on a virtual machine around the clock, consolidates what it learned overnight, and proposes next steps unprompted. The economics follow the principle. The free tier starts at 100 million tokens a week, with the machine included. Revenue is meant to come from a very small cut of transactions, paid by the businesses on the other side, on the bet that the agent makes or saves its user money. Zuckerberg is explicit that affordability is what makes it a product for everyone.

Giving an agent your email, health information and payment credentials only works if you trust the system, so he treats trust as the real moat. WhatsApp taught Meta to design so that even Meta cannot see the content. He and Nat recruited Moxie Marlinspike to build a confidential VM carrying a commitment that can be technically verified, and Meta says it will publish details in the coming weeks. Around it sit Sentinel agents that watch for prompt injection and outbound leaks, human approval for logins and payments, and least-privilege connectors that start email as read-only. He claims no one else is anywhere close. That is checkable, and the publication is the test.

None of this matters without a model worth distributing. Here Zuckerberg is candid about a failure. He assumed that Meta's skill in feed and ads machine learning would carry over to LLMs, and Llama 4 proved otherwise. The remedy was talent density: the smallest group that can hold the whole system in its head, the best person in every seat, and a lab built around where he sits. Compute followed, with the gigawatt Prometheus cluster in Ohio live and many more gigawatts planned, funded by a very profitable business. He rejects the idea of staying just behind the frontier. Meta builds end to end, and he says no one can take a model off the shelf, post-train it a little, and match one designed for personal agents from pre-training up. His example is discretion. An agent booking a restaurant should find a place with good mocktails without announcing that its user is pregnant.

The obvious objection is that symmetric access only works if defence keeps pace with offence. His lawyer analogy assumes everyone holds a lawyer of equal strength at the same time. He lists bio risk among the live concerns and says bio challenges may follow cyber in about six months. Yet the case he works through is cyber. He also concedes that models are now smart enough to inspect their own training environment and change a VM configuration to pass a task. His answer to that is firm boundaries, which he likens to parenting. That is a training claim, not a distribution claim.

Then there is the question he did not answer. Asked whether he would hold back a Meta model if he saw capabilities that were simply not safe, he said to design and train it safely and to distribute it widely. He gave no point at which the answer changes. He is also not an absolutist. Meta does closed work, and a for-profit company, he says, need not share everything. On regulation he prefers a close working relationship with government, including knowing all important training runs, over a rigid framework that would be out of date in months. That makes transparency run to government, not to the public.

So the position is distribution by default with an undefined exception. Watermelon, a bigger pre-train than Avocado and due soon, will be the first real test of how undefined it is.