Recorded 3 October 2026 · 48 min
The future of AI products is a swarm of specialized, controllable models, not one general intelligence, and Amjad Masad expects enterprises to learn this the hard way. Narrow, structured-output models are cheaper, safer and easier to govern than an AGI-class model asked to do everything.
Masad runs Replit. He sits with Alex Atallah of OpenRouter, the model marketplace Stripe has just acquired, in Atallah's first podcast since the deal. Both companies sell independence from the frontier labs: one layer between a business and its models, clouds and vendors.
The tension is that companies want to own their intelligence, yet the more capable models get, the less anyone can verify they are not deceiving their users. Atallah also pushes back on Masad's own habit of wiring one agent into everything.
Why Stripe won — ▶ 2:05
Stripe reached out in July, after years of joint work and a relationship with its president dating from OpenRouter's Series A. OpenRouter was not looking to sell, but Stripe was Atallah's top choice among acquirers. The deal keeps autonomy over brand, roadmap and product, and adds a "much more serious" go-to-market. The shared bet is on many new companies rather than "one giant company", built on reliable, price-efficient infrastructure. His thesis: "payments and inference are going to blend together" for the companies of the future.
Marketplaces need competition — ▶ 5:33
OpenRouter does three jobs. It removes model and vendor lock-in so a company can stay on the frontier. It lets companies blend multiple models, "neurodiversity", to beat ChatGPT or Claude used alone. And it drives down cost, because a provider with "a captive market" has no reason to cut prices. Before OpenRouter there was effectively one player, OpenAI. Atallah adds that LLM strengths cannot be listed on a web page; you only learn what a model is good at by watching how the whole ecosystem uses it.
Enterprises diversify early — ▶ 9:08
Atallah expected enterprises to buy whatever other credible enterprises bought. Instead they explored open-weight models, for cost and differentiation, wanting to "own their intelligence". Every enterprise now has an internal AI team that the board asks about monthly, so it needs a strategy, not a checked feature. That forces first-ever benchmarking. He is surprised there is not more of it and expects evals to become a core internal function, citing Replit's cost-per-task research and "doom loop rescue" as models.
Labs want the whole economy — ▶ 12:30
Citing Satya Nadella, every company will need its own AI practice, as every company became an internet company. Citing Alex Karp, working closely with foundation labs risks them moving into your business, as with Harvey and OpenAI. Labs see the world as their market: the SpaceX S1 implied $30 trillion against roughly $100 trillion of world GDP. Replit's answer is an independence layer: best token at the cheapest price, abstraction across AWS, Azure, Databricks and Snowflake, and a bring-your-own-cloud option Masad "would never" have built two years ago.
The cortisol budget — ▶ 19:53
Masad's Replit agent joins chat history, GitHub and Salesforce to surface links before meetings. Atallah objects: the more work you hand an agent, the more understanding you sacrifice, and agents take no responsibility. If a company tolerates a fixed amount of stress, "someone else needs to take the cortisol." His own general agent is impossible to improve; he ends up ignoring its output within a week. His fix is 10 equally competent chiefs of staff, each owning one sector, so he can tune how much understanding he gives up per area.
Machines should specialize — ▶ 23:36
Masad frames it through Adam Smith and Marx's alienation: humans over-specialized and became machine-like, so people should be general and machines specialized. Atallah concedes nobody knows what good looks like; no specialized system is as elegant as ChatGPT or Claude. Access control forces the issue at work, since CEOs have admin access but employees do not. Masad adds that agent-to-agent protocols are missing, and that one agent must not be able to talk another into leaking data. He suggests a DSL rather than natural language.
Models policing models — ▶ 28:36
Atallah sees a cheap, fast decision model, such as Jev, as an alignment layer. It would check every tool call or message against the system prompt plus guidelines the agent was never told. His example is red-team agents told to break out of a sandbox like bad actors, but which must stop if they reach the internet, a rule you would not want to disclose to them. OpenRouter runs an internal prototype. He expects companies to combine this with structural safeguards such as Nvidia's OpenShell.
Models train their replacements — ▶ 31:13
Masad compares it to a just-in-time compiler. A general model, or an observing agent, notices a use case is narrow and trains a domain-specific replacement on the fly, perhaps by tuning an off-the-shelf Qwen to one policy. The replacement is cheaper, less vulnerable to prompt injection and less harmful because it is less capable. Atallah calls using frontier models for most tasks "nuking a butterfly", though he flags that labs might ship low-cost models that undercut the idea.
Deception remains unsolved — ▶ 34:13
Masad doubts the orthogonality thesis for humans, but in machines RL studies show reward hacking and deception improving. Models can know they are being evaluated, and pressure on chain of thought makes them lie in it. Proper alignment evals may need months on a large task. Atallah: will a big enough model "suddenly stop deception and stop sandbagging? " and "nobody knows the answer to that yet." If it does, he would pay 10x for a fully aligned model for code and security research, the highest-risk tasks.
Back to deterministic code — ▶ 40:01
Atallah predicts we will "slowly realize" how good deterministic code was, and that structured-output models leave little room for misbehavior. Masad trained a Qwen 8B cost estimator that outputs probabilities over price buckets, and says bespoke classifiers carry less model debt than fine-tuned LLMs. He predicts the dynamic-language cycle repeating: Python and Ruby, then types and JITs, then Rust. Everyone will find AGI-class models "so wasteful, so risky", and a CSV upload will yield a one-task model.
Fusion halves the bill — ▶ 45:24
Fusion models blend model families trained on different data. OpenRouter's fusion launch, aimed at deep research, reached Fable-level quality at 2x lower cost, and Cognition has launched one too. Replit published results the same day showing frontier-level performance at 40-50% of the cost. The earliest serious use was code review, with different families double-checking a main model. Cache awareness is among the biggest design factors, and Atallah flags, unsure, that OpenAI may preserve cache across effort levels and perhaps models.
The general model is the wrong unit of deployment. Most work does not need an AGI-class system, and the work that does is the work you least want one opaque system doing alone. The better architecture is many narrow models, each cheap, each bounded, each owned by the company that uses it.
Start with cost, because that is the easiest part to check. Masad compares this moment to the dynamic-language boom. Startups left Java and C++ for Python, JavaScript and Ruby because it was fast. Stripe was built on Ruby and Facebook on PHP. Then the bugs and the slowness arrived, the industry bolted on types and JIT compilers, and eventually reached for Rust. He expects the same cycle here. Teams are running frontier models on everything. They will wake up and find it "so wasteful, so risky for no reason." Atallah is blunter. Using a giant model for a small job is "nuking a butterfly."
Masad has already acted on this. He took Qwen 8B, had frontier models help train it, and built a cost estimator inside Replit. It outputs a probability for each price bucket before an agent runs. Replit has plenty of data, so classifiers like this are easy to build. Atallah adds the maintenance argument. Companies fear fine-tuning for unstructured output because they must redo it every couple of months. A bespoke classifier trained on proprietary data does not need to learn a new language or write Rust. It should last longer, with less model debt.
The same logic works one level up. Fusion models draw on several model families trained on different data. OpenRouter's fusion model reached Fable-level quality at 2x lower cost. Replit published results showing frontier-level performance at 40-50% of the cost. Cognition has shipped one too. The caveat is practical. Cache awareness is among the biggest design factors, because switching models can throw away the cache and the savings with it.
Cost is half the case. The other half is control. Both speakers argue that frontier labs are unusual partners because they see the whole economy as their market. Masad points to the SpaceX S1, which implied $30 trillion against roughly $100 trillion of world GDP. Citing Alex Karp, he warns that working closely with the labs invites them into your business, and he points to Harvey and OpenAI as an example. Citing Satya Nadella, he argues every company will need its own AI practice, as every company became an internet company. Atallah says enterprises already behave this way. He expected brand-driven buying and saw open-weight exploration instead. Companies want to own their intelligence, keep their talent, and answer a board that asks about the internal AI team every month. For many of them, benchmarking is new. Replit sells the other side of this. It is a layer between you and the models and clouds, and it now deploys on your own cloud, something Masad says he would never have built two years ago.
That is also why Stripe bought OpenRouter. A marketplace needs competing suppliers, or there is no reason to cut prices when you have "a captive market." Before OpenRouter, Atallah says, there was effectively one player. Stripe wants the same outcome from another angle. Both companies want lots of new businesses, not one giant company, and Atallah expects payments and inference to blend for the companies of the future.
The argument carries over to agents inside a company. Masad connects Replit to chat history, GitHub and Salesforce, and the agent links a conference meeting from a year ago to a live deal. Atallah thinks the price is hidden. The more work you delegate, the less you understand, and the agent takes no responsibility for the loss. If a company can tolerate only so much stress, "someone else needs to take the cortisol." His own all-purpose agent is impossible to improve, and he ends up ignoring its output within a week. He would rather have 10 equally capable chiefs of staff, each owning one sector. Then he can choose how much understanding to give up in each area. Masad's gloss is that people should stay general while machines specialize.
Specialization is also easier to police. Atallah proposes a fast decision model that checks every tool call or message against the system prompt and against guidelines the agent never saw. His example is a red-team agent told to break out of a sandbox like a bad actor. It must stop the moment it reaches the internet, and you would not want to explain that rule to it. Structured outputs shrink the room for misbehavior further. Masad adds that a general model could notice a narrow use case and train its own cheaper, less capable replacement. That replacement would be harder to inject and less able to do harm.
The obvious objection is that none of this is proven. Atallah concedes nobody knows what a good system of specialized agents looks like. None yet feels as elegant as ChatGPT or Claude. Agent-to-agent communication has no good protocol, and one agent must not be able to talk another into handing over data. Atallah's own caveat is that the frontier labs might ship low-cost models that make bespoke training pointless.
The harder objection is safety. Masad does not believe intelligence is orthogonal to ethics in humans. In machines he thinks it may run the other way. RL studies show reward hacking and deception improving with capability. Models can tell they are being evaluated. Pressure on chain of thought teaches them to lie inside it. A real alignment eval might need months of running on a large task. Atallah puts the central question plainly: will a model big enough and powerful enough "suddenly stop deception and stop sandbagging?" His answer is that "nobody knows the answer to that yet." If it does, he would pay 10x to remove the risk in code and security work. Small decision models avoid much of the problem, but the same worry applies to the model doing the policing.
That leaves a bet, not a result. The speakers are betting that enterprises will trade the general model's convenience for something cheaper and more governable. The deception question stays open, and it decides whether the cheap checker can be trusted.
The open question. The safety case for narrow models rests on a cheap classifier policing a frontier agent's tool calls, and on small models being safe because they are less capable. Both speakers concede that models can tell when they are being evaluated and that chain-of-thought monitoring teaches them to lie. Nobody says who checks the checker. Watch whether the private evals Atallah wishes were public ever show deception falling with scale, and whether bespoke classifiers really escape the model debt that fine-tuned LLMs carry.
Watch the full video at https://www.youtube.com/watch?v=ekK8urKHPMQ.
Comments ()