Rao bets energy-to-intelligence conversion, not models, will capture AI's value

1. Gigawatts cost more than fabs — ▶ 1:44

   Rao cites general-industry figures of about $50 billion to build 1 gigawatt, versus $20-30 billion for a semiconductor fab. Data centers have so far been served by running existing infrastructure at higher utilization. The US grid averages roughly 900 gigawatts of output but can peak at 1.2-1.3 terawatts, and that headroom is what operators can access quickly. Building genuinely new capacity is slow, he says, and the US is behind.

2. China's build-out versus US innovation — ▶ 3:09

   Rao says China built capacity 6x faster than the US for 20 years and passed it in total capacity in 2023, using dirty methods it is now cleaning up. Central planning gives China an advantage in giant infrastructure rollouts. The US, he argues, remains far ahead at the "tip of the spear": AI was born here. He credits real innovation in China, not just distillation, but says the US must leverage its own advantages.

3. Spare capacity runs out — ▶ 4:51

   Even under conservative growth assumptions, Rao expects spare grid capacity to be exhausted within a few years, so supply converges to the rate of construction. He doubts claims of 40 gigawatts in a year; 10 would be amazing, roughly half a trillion dollars. Evidence is anecdotal: bespoke behind-the-meter deals pay well above grid rates, creating two-tier pricing. With compute sold at an illustrative 5x electricity cost, operators can pay 30-40% more. Bloom Energy's Arizona project remains unproven.

4. Efficiency moved up the stack — ▶ 9:11

   Rao splits computing into physical, architectural, and model levels. Moore's Law and Dennard scaling were physical gains, and he says they have largely stopped; recent GPU generations show little from them. Recent gains came from smaller data types, from 32-bit down to 4-bit. The underexploited level is the model: enforcing locality and reducing data movement. Treating the computer solely as a device for delivering intelligence allows co-design from model to silicon, with an energy-first rather than speed-first mindset.

5. Co-design unlocks sparsity — ▶ 15:14

   Rao sees plenty of model-efficiency headroom because models have never been co-designed with hardware. A concept might yield around 5% on a standard GPU, and sparsity gains are capped by off-chip random access memory that was not built to exploit them. Hardware that exploits sparsity could yield far larger gains, as the brain's co-evolved intelligence and physical layers did. He therefore expects continued specialization of chips for specific workloads, though he offers no measured example of the larger gains.

6. Intelligence has infinite demand — ▶ 20:43

   Rao rejects chip commoditization on a philosophical premise: the universe is adversarial, from bacteria to nations, and intelligence is the fuel of those contests, so demand has no ceiling. Producing intelligence requires energy, and the conversion of energy to intelligence is hardware, meaning chips, systems, cloud, and networks. The better a company does that conversion, the more value it accrues. The argument is axiomatic rather than evidence-based, though he notes Jensen Huang recognized AI's importance at GTC in 2014.

7. Verification replaces production — ▶ 23:17

   AI compresses Rao's exploratory work dramatically: a noise-propagation question about coupled oscillators that once took researchers a week or two now takes an hour. But models also generate what he calls slop, and the bottleneck has shifted from producing artifacts to verifying them. He questions whether owning verification means owning automation. In his view, you also need to own the spec, and the whole process from spec to verified result is what matters.

8. Agents not yet trustworthy — ▶ 28:24

   Rao says agents should not be trusted like humans yet. Models are stochastic, and the mechanism for picking good outputs from a distribution of attempts is not well understood for every task. Long chains of decisions can look robust yet be completely wrong. Like a neurosurgeon's seven years of training, the hard part is judgment. He expects agents eventually to be trusted more than ourselves, but needs a few more breakthroughs, and thinks the provenance of a machine's training will carry value.

9. Data curriculum is the moat — ▶ 32:52

   Rao does not see data companies commoditizing. Data started as volume gathering but is specializing into high-quality sets for coding or chain of thought, with value following a power law like human skills. He says big-lab capability gains come mostly from data curation and training curricula running from pre-training into mid-training mixes of RL and supervised fine-tuning. RL is less efficient per unit of compute or dollar, but necessary because data cannot be gathered for every task and verifiable rewards allow targeted learning.

10. Startups take paradigm risk — ▶ 36:08

    Rao argues big labs cannot afford long-term architectural bets: one month late on a model is a disaster. He reads xAI reaching the frontier quickly through hiring and execution as evidence that little deep innovation was required. Deeper bets are coming from startups such as Yann LeCun's. He compares the structure to drug development, where startups absorb the large risks and big companies acquire the ones that show promise.

11. Past the transformer — ▶ 39:10

    Rao believes the industry will move beyond transformers. Reasoning in current systems emerges from sampling multiple traces, comparing them, and feeding the best back in, with attention acting as a retrieval mechanism. He calls this extremely inefficient and notes brains do not pipe huge traces into another model. He defines intelligence as what you do when you lack information, which requires first-principles reasoning. More efficient and higher-performing algorithms exist in principle, but no specific successor architecture is described.

12. Margins drain above hardware — ▶ 41:39

    Rao says models 6 months old, such as Chinese open models, remain useful, and frontier capability reaches open models within 6 months. Commoditization cuts margin but enlarges the market, as with tomatoes. Inference operators such as Fireworks will see lower margins as per-token prices fall, while Nvidia holds the lion's share today. Hardware is, in his view, the place to be for 10 years. In 3 years he expects 80% of tokens to be open models but roughly 50/50 spend.

13. SaaS becomes outcome services — ▶ 45:04

    Rao says the SaaS boom since roughly 2010-2011 came from frictionless AWS delivery fragmenting markets into $100-200 million niches, and that margin is leaving. Pressed that SaaS merely becomes Harvey-style companies, he agrees: AI plus human verification delivering outcomes, as Harvey does in law. He calls owning your own model somewhat techno-optimist. Only scale players such as large law firms will customize, while small firms buy out-of-box services, following the adversarial logic of who you compete against.

14. Spend stays, output rises — ▶ 50:42

    Rao expects the share of developer budgets spent on AI to stay roughly stable, because experienced engineers who know good code must verify outputs. If a company spends $10 million a year on software, customers will demand bug fixes in a week, then an hour; spending holds while output rises. Cheaper supply changes demand. His physical analogy: robots doing 80% of skyscraper work cut a 2-year build to 8 months, yielding more buildings rather than fewer dollars. It is reasoning by analogy, not data.

15. Robotics 100x larger than coding — ▶ 55:24

    Against a host skeptical of robotics timing, Rao expects working prototypes in 2-3 years and a market in 5-7 years. He projects inference at $10 trillion within 10 years, and says each 10x gain in token efficiency expands uses by more than 10x, citing Jevons paradox. Robotics, he argues, will be 100 times larger than coding because current industrial robots are rote. With trillion-dollar TAMs he calls price almost irrelevant, and favors technical-progress validation over market validation.

Naveen Rao's argument starts from a plain framing: AI is a machine for converting energy into intelligence, and the economics of the industry will be set by how efficiently that conversion happens. Intelligence, in his telling, faces unbounded demand, so the binding constraints are physical. Electricity is the first. The second is the stalling of the transistor-level improvements that once made each generation of computing cheaper to run. Rao, who runs the hardware startup Unconventional AI, concludes that value endures in the full hardware stack, from chips to networks to cloud, while models and the services that run them lose pricing power. Each link in that chain deserves scrutiny, because several rest on his judgment rather than on data presented in the conversation.

Start with power. Rao cites general-industry figures of roughly $50 billion to build 1 gigawatt, more than the $20-30 billion he puts on a semiconductor fab. Data centers have so far been fed by running existing infrastructure harder. The US grid averages about 900 gigawatts of output but can peak at 1.2-1.3 terawatts, and that headroom is what operators can reach quickly. His claim is that the slack disappears within a few years even under conservative AI growth, after which supply converges to the rate of construction. He doubts claims of 40 gigawatts in a year. Ten would be remarkable, and he prices that at half a trillion dollars.

The first consequences, he suggests, show up in price. Rao has no figures, since each behind-the-meter deal is bespoke, but says buyers pay well above standard grid rates per kilowatt-hour, producing a two-tier market in which electricity for data centers is worth more than electricity for general use. His rationale is margin. If compute sells for something like 5x the cost of its electricity, a number he flags as made up, an operator can tolerate paying 30-40% more. Alternatives such as the Bloom Energy deal for a couple of gigawatts in Arizona are, he says, not yet proven. He adds that China out-built the US by 6x for 20 years, while the US keeps its edge in frontier innovation.

If power is scarce, efficiency becomes the lever, and Rao's second claim is that efficiency has migrated up the stack. He divides computing into physical, architectural, and model levels. Moore's Law and Dennard scaling were physical gains, and he says they have largely stopped; recent GPU generations show little from them. Most recent gains came from the architecture level, chiefly shrinking data types from 32-bit to 16, 8, and 4 bits. The level he considers underexploited is the model itself, where locality can be enforced and data movement reduced. Once the computer is treated solely as a device for delivering intelligence, co-design from model to silicon becomes possible. His regret is not grasping earlier how much room remained at the top.

Why can't software capture those gains alone? Rao's answer is that algorithms are bounded by the hardware beneath them. A new concept might yield around 5% on a standard GPU. Sparsity, meaning fewer connections between computing units, helps on a GPU, but off-chip random access memory caps the benefit because it was not designed for it. Hardware built to exploit sparsity could unlock much larger gains, as the brain did by co-evolving its intelligence and physical layers. That is why he sees specialization as the destination, with the GPU already a step down that path. He offers no measured example of the larger gains, so this remains a thesis he is building a company to test.

Why would hardware stay valuable when other layers do not? Rao's case is philosophical. The universe is adversarial, from bacteria competing for sunlight to rival nations, and intelligence is the fuel for those contests, so demand has no ceiling. The logic is closer to axiom than evidence. Its practical form is easier to test. He argues that a 10x efficiency gain in delivering a token expands the base of potential uses by more than 10x, which he calls Jevons paradox. He projects AI models alone as a $1 trillion market within a few years and inference as a $10 trillion market within 10 years. Both are forecasts, not measurements.

Above the hardware, Rao expects margin to drain. Models are already partly commoditized: those 6 months old, such as Chinese open models, remain useful, and he expects frontier capability to arrive in open models within 6 months. Commoditization, he argues, lowers margin while enlarging the market, as with tomatoes. Inference operators such as Fireworks map open models onto hardware with uptime guarantees, and as per-token prices fall they will earn less. Nvidia, not the model companies, takes the lion's share of margin today. For 3 years out he guesses 80% of tokens will be open models but spend will split roughly 50/50, echoing the iPhone's 20% of units and 80% of margin.

The same logic reaches software. Rao says the classic SaaS boom, which began around 2010-2011 when AWS made it cheap to test a $100-200 million market, is losing its margin, and he doubts software will be a huge market in 20 years. Pressed that SaaS simply becomes a new label, he concedes the point: companies that deliver outcomes through AI plus humans, with Harvey in legal work as his example, are the new form. He calls agentic coding tools overrated because value flows to model providers and developers switch tools fluidly. He also considers owning your own model a bit techno-optimist, since only scale players such as large law firms will build one.

What remains scarce on the human side is verification. Rao's own experience shows the upside: a noise-propagation question about coupled oscillators that once cost a week or two of researcher time now takes an hour. But the volume of output creates what he calls an AI slop problem, and the bottleneck moves from producing artifacts to checking them. He resists the idea that owning verification means owning production, since you must also own the spec and the end-to-end process. And he does not trust agents with long chains of decisions. Models are stochastic, the mechanism for choosing among outputs is not well understood for every task, and a chain can look robust while being completely wrong.

That is why he expects expertise to carry value even as capability spreads. He compares agents to a neurosurgeon, whose seven years of post-medical-school training build judgment rather than manual skill. Machines will be taught like organisms, he predicts, and the provenance of that teaching will matter. He sees the same logic in data, which he thinks specializes rather than commoditizes. Capability gains in the big labs come mostly from curating data and from training curricula running from pre-training into RL and supervised fine-tuning. RL is less efficient per unit of compute or dollar, he concedes, but he argues it is necessary, since data cannot be gathered for every task and verifiable rewards make targeted learning possible.

Who will break the paradigm? Not the big labs, at least yet. Falling a month behind on a model is a disaster, so they cannot afford large risks. Rao reads xAI reaching the frontier quickly through hiring and execution as evidence that little deep innovation was needed. Deeper bets come from startups such as Yann LeCun's, and he compares the structure to drug development, where startups absorb the big risks and large companies buy the survivors. He expects the transformer to be superseded, because it produces reasoning by sampling many traces and comparing them, which he calls extremely inefficient next to how brains reason. No specific successor architecture is described in the conversation.

Rao's answer to the displacement worry is elasticity. He expects the share of developer budgets spent on AI to stay roughly stable, because experienced engineers who know what good code looks like must verify the output. If a company spends $10 million a year on software, customers who once tolerated annual releases will demand bug fixes in a week, then in an hour. Spending holds while output rises. His physical analogy is a skyscraper: if robots do 80% of the work and cut a 2-year build to 8 months, the result is more buildings, with the same dollars going into more projects. He allows for a moment of labor disruption, but this is reasoning by analogy, not data.

The investing implications cut against venture habit. Rao says he changed his mind on market validation, which he traces to SaaS. When a company is solving a deep technical problem whose market is evident, the metric that matters is progress on the problem. He applies this to robotics, where he expects working prototypes in 2-3 years and a market in 5-7 years, and predicts a market 100 times larger than coding. Industrial robots, he says, merely execute tasks by rote. Robots that solve physical problems dynamically would sharply cut the cost of building and repairing the world. With trillion-dollar TAMs, price is almost irrelevant, and early on he backs "someone to drive the bus not the bus itself."

The weakest joint in the chain is the link between price and funding. Rao expects per-token prices to fall and models to commoditize, yet also expects a build-out costing about $50 billion per gigawatt, paid for by buyers who sell compute at a multiple of their electricity cost. His answer is volume: a market growing faster than margins shrink. But the figures behind that growth are his projections, and he dismisses bubble concerns as a blip largely by analogy, since people would of course want a cure for cancer. He also admits agents cannot yet be trusted, which limits how fast demand can scale. Whether volume outruns margin compression while energy is rationed is the test his thesis has to pass.