> ## Content Index
> Fetch the complete content index at: https://nnutshell.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Nvidia paid $20 billion because GPUs can't do fast inference
- URL: https://nnutshell.com/nvidia-paid-20-billion-because-gpus-cant-do-fast-inference/
- Published: 2026-10-23T18:12:15.000Z
- Updated: 2026-10-24T03:07:05.000Z
- Description: Andrew Feldman, CEO of Cerebras, talks to Matt Turk about why wafer-scale chips beat GPUs at inference, the three shortages that constrain GPU makers, and a decade of nobody buying.
- Author: Admin
- Tags: Andrew Feldman, Cerebras, Nvidia, OpenAI, TSMC, Semiconductors, Inference, AI Infrastructure, Data Centers, #rec-2026-07-26, #Import 2026-10-25 09:54

*Recorded 26 July 2026 · 72 min*

Fast inference is a memory-movement problem, not a compute problem. Generating each word means moving a model's weights from memory to compute, about 100 HD movies' worth for a 70 billion parameter model, and GPUs do it over slow HBM. Cerebras keeps the weights in SRAM on a dinner-plate-sized wafer and claims the move is about 2,500 times faster.

Andrew Feldman, co-founder and CEO of Cerebras, talks with Matt Turk on the Mad Podcast just after a record semiconductor IPO and an OpenAI deal worth more than $20 billion. The tension is between a decade in which nobody wanted his chip and a present in which demand outruns supply everywhere. He argues the shortages hitting rivals do not touch him and that CUDA no longer protects Nvidia. The harder questions, about lab-financed demand and his own move into renting data center capacity, get confident but thinner answers.

1. **Speed becomes the product** — [▶ 2:06](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=126s&ref=nnutshell.com)  
AI was a parlor trick until around the middle of 2025, when models got smart enough to use. Once tokens are productive, fast tokens are worth more, and the metric becomes tokens per second per user, first token to last. Agentic flows multiply the problem because waiting is amplified across multi-cycle turns. Feldman's analogy is Netflix: when the internet got fast it did not get better at mailing DVDs, it became a movie studio. Speed lets users stay longer, return more often and "work on harder problems". The market for dial-up, he says, is zero.
2. **Groq price as admission** — [▶ 8:10](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=490s&ref=nnutshell.com)  
Nvidia's most durable moat, Feldman says, was the perception that the GPU could do everything. Buying Groq for $20 billion, and doing it fast, told the market that GPU architecture could not do fast inference and that the segment was large and growing. Cerebras sales were more than 10 times Groq's, so Nvidia paid $20 billion for the number two player. He frames every chip as an ASIC in some sense: the TPU cannot do graphics, and each design is better at some things and much worse at others.
3. **OpenAI reads the curve** — [▶ 9:15](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=555s&ref=nnutshell.com)  
The OpenAI contract is north of $20 billion, which Feldman calls probably one of the largest deals in Silicon Valley history. His read on OpenAI: it looks at an exponential adoption curve and is not afraid of what it says. Others had to strike really bad deals for capacity, while OpenAI locked in memory and compute early, with Cerebras and others. The usual order is inverted: often people build and hope demand comes, but here the whole industry is chasing usage that already exists. He also calls a multi-silicon market healthy, citing x86's two-player decades, lost to ARM when phones arrived.
4. **China as industrial adversary** — [▶ 12:10](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=730s&ref=nnutshell.com)  
Feldman calls China an industrial adversary rather than a chip rival. It is behind in chips but has invested heavily in power and the grid, a real US weakness, and it produces very good open source models that trail GPT, Anthropic and Gemini. He sees possible Chinese leadership in power, which data centers need. Cerebras does not sell there, for regulatory reasons and some geopolitical ones. On local AI, he says phones and laptops are battery constrained, so the big work goes to the data center, which is where Cerebras focuses.
5. **Not a bubble, he argues** — [▶ 17:24](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=1044s&ref=nnutshell.com)  
Asked whether lab-financed demand is fragile, Feldman says Cerebras has dozens of other customers trying to place very big orders. Bubbles, in his account, are supply getting ahead of demand: 1990s fiber was laid years early and took 6 to 8 years to fill. Now everyone is trying to catch up, only a small part of the world uses AI near its potential, and he says the industry is "overwhelmed" with demand for compute and memory. On market swings, public markets vote in the short term and weigh in the long term. He ignores the daily moves.
6. **Three shortages, none his** — [▶ 19:07](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=1147s&ref=nnutshell.com)  
Every GPU and ASIC except Cerebras uses DRAM, specifically HBM, made by three companies: SK Hynix, Samsung and Micron. It is sold out, and only new factories add supply. TSMC's CoWoS process, a silicon motherboard used by Nvidia and AMD, is sold out too. Capacity at 3 nanometers is congested, and most of the GPU world is there. Cerebras uses none of the three: its architecture avoids HBM and CoWoS, and its chips are on 5 nanometers. The chip is 58 times the size of a GPU with 2,500 to 3,000 times the memory bandwidth.
7. **Agents drive CPU demand** — [▶ 23:21](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=1401s&ref=nnutshell.com)  
Agentic AI initiates action: visiting a website, pulling data from storage, ordering a burrito. CPUs execute all of it. Feldman's metaphor is the accelerator as brain and CPUs as body, so each increment of agentic work generates more CPU calls. That is pushing CPU demand "through the roof", and CPUs hit the same memory shortages as accelerators. It is a second-order effect of agents that most chip talk about GPUs and ASICs misses, and it connects the shortage story to how AI is actually being used.
8. **Ten years of bad timing** — [▶ 25:51](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=1551s&ref=nnutshell.com)  
"The way you have perfect timing is to have horrible timing for 10 years." Cerebras started in 2016, four years after ImageNet, when everything was vague and nascent. The design rule was never to embed the latest model in circuitry, which Feldman calls a historical structural mistake. Its architecture predates transformers yet is fastest at them, and it was fastest at diffusion before diffusion had an architecture. The bet is on the underlying calculation, sparse linear algebra. From SeaMicro, sold to AMD, he carried two questions: can this be made faster, and is there enough of it?
9. **Be 50 times better** — [▶ 30:26](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=1826s&ref=nnutshell.com)  
Feldman told VCs up front that you never beat Nvidia by being a little better: they buy everything for less, apply pricing pressure and bundle. The target was 10 to 50 times better. HBM stores a lot but is slow, while SRAM is unbelievably fast but stores little. In graphics, work dominates data movement. In inference it reverses, because all the weights move for one calculation. The fix is a part the size of a dinner plate stuffed with SRAM. The prior biggest chip was 840 sq mm, while this is 46,000, cut from one 300 mm wafer.
10. **The unknown hard part** — [▶ 33:56](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=2036s&ref=nnutshell.com)  
No vendors existed for a part that looks like nothing else ever made, so power delivery, cooling and packaging had to be invented. For 18 months the company spent $8 million a month and could not build the chip. What everyone feared was easy. What nobody knew to fear, affixing a wafer to a motherboard and powering and cooling it, was the hard part. A failure analysis on every failure got them through, and they announced success in August 2019\. Redundancy is designed in: about a million identical tiles, failed ones shut down and spares used. Water cooling keeps temperatures down.
11. **Decode is the bottleneck** — [▶ 41:37](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=2497s&ref=nnutshell.com)  
Inference has two steps. Prefill processes the prompt and can be parallelized. Decode generates the answer one word at a time, and it cannot be parallelized. Each word requires moving the model's weights from memory to compute. For a 70 billion parameter model that is about 100 HD movies' worth of data, moved again for every word, for perhaps 1,000 words. That exact step is where HBM is slow. Because Cerebras holds the weights in SRAM, it moves them about 2,500 times faster than a GPU. Faster inference also speeds RL, which runs inference inside training.
12. **Reasoning widens the gap** — [▶ 48:30](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=2910s&ref=nnutshell.com)  
Reasoning works like drafting in eighth grade: break the problem into parts, solve, review, improve, answer. It takes more compute at inference, and on slow hardware that is crippling. Every leading model, domestic or Chinese, has moved to it. Feldman calls that a huge advantage for Cerebras, because GPU slowness stacks up while its speed advantage widens. Verification and guardrails cost more passes: re-ask the question differently, score the answer. His preferred pattern is a big model writing and a small model checking, like a journalism data checker verifying each claim independently.
13. **Hardware shifts to cloud** — [▶ 54:28](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=3268s&ref=nnutshell.com)  
Cerebras sells hardware for customers' data centers or rents capacity from its own. The mix was 50/50 last year, about 75/25 hardware last quarter, maybe 50/50 this year, and he expects 30/70 hardware to cloud as the OpenAI deal unfolds. That deal is 750 megawatts: 250 in 2026, 250 in 2027 and more in 2028, on multi-year leases, delivered as a full cloud through an API. Data center availability is a limiting constraint for everybody. He is building in the Nordics for clean, cheap power and cooling, and keeping data centers on the same continent as customers because of fiber latency.
14. **CUDA is not a moat** — [▶ 1:01:05](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=3665s&ref=nnutshell.com)  
Feldman rejects CUDA as a moat. Two years ago every state-of-the-art model was trained in a CUDA flow. Now Gemini trains on TPUs, Claude on Trainium, and only OpenAI still uses CUDA. He says Nvidia lost 70% share of training for leading models in a year or two. In inference there is no moat at all: it takes "eight keystrokes" to move traffic from a GPU API to Cerebras. He concedes CUDA was enormously important to the industry's creation and the moat has shrunk rather than vanished. His own moat is architectural: rivals cannot simply spend their way to fast.
15. **Nobody cared in 2020** — [▶ 1:06:53](https://www.youtube.com/watch?v=UEOSUSz--Ig&t=4013s&ref=nnutshell.com)  
Cerebras delivered a chip no one in computing history had built in 2020, and "Nobody cared." The first generation sold 20 or 30 units, the second 300 to 500, and the third tens of thousands. AI was a hobby then, and nobody cares whether a hobby is fast. Speed matters once AI is in production and used daily. He says first chips rarely win: Google's TPU was good only on the fourth try and is now on its eighth, and AWS's chip was good on the second or third. TSMC agreed to Cerebras's 2017 proposal in the meeting, and the first chip took 2 years.

Cerebras was not built to beat Nvidia at its own game. It was built so that the game stopped applying. Andrew Feldman told his investors at the start that nobody beats a great incumbent by being a little better. The incumbent buys everything for less, takes pricing pressure and bundles. The only route is something so hard in engineering that the result is 10, 15, 20, 30 or 50 times better. In his words, "ordinary and obvious paths are all closed." Everything the company did afterward follows from accepting that.

The hard thing was memory. There are two kinds. DRAM and its HBM variant hold a lot and are slow. SRAM is unbelievably fast and holds little per unit area. Every GPU and every other accelerator maker went with HBM. Cerebras took SRAM and removed its weakness by making a part vastly larger than anything in history. The biggest chip before it measured 840 square millimeters. Theirs measures 46,000\. It is cut from a single 300 mm wafer, is 58 times the size of a GPU, and has 2,500 to 3,000 times the memory bandwidth. For AI, Feldman argues, bigger chips give answers in less time. For a laptop or a phone they make no sense. For AI work, he says, big chips are the best way to go.

The price was paid in years nobody sees. A chip like this has no vendors, because it looks like nothing else ever made. Power, cooling and packaging all had to be invented. For 18 months the company spent $8 million a month and could not build the part. Feldman compares it to the first team up Everest. The team that failed warns about a hard stretch halfway up. The team that summits says that stretch was not the hard part, because nobody had got far enough to know what to fear. Here the hard part was packaging: how to affix a wafer to a motherboard, deliver power to it and cool it. The team ran a failure analysis on every failure until it worked, and announced in August 2019 that it had. Size brings its own risk, and the answers were designed in. About a million identical tiles let the system shut down a failed one and use a spare. Water cooling keeps the parts colder, and temperature is the failure mode in electronics.

Then came the desert. The company delivered in 2020 and nobody cared. The first generation sold 20 or 30 units. The second sold 300 to 500\. The third sold tens of thousands. The reason is that AI was a hobby then. "Who cares if your hobby is really fast?" Speed matters in production, when people use a system every day. His line on timing: "The way you have perfect timing is to have horrible timing for 10 years." What kept the design alive was a refusal to embed any model in the circuitry. The architecture was set before transformers existed and is now the fastest at them. It was fastest at diffusion before diffusion had an architecture. The bet is on the underlying calculation, sparse linear algebra, so whatever model makers invent can be run faster.

The market arrived when models became useful, around the middle of 2025\. Once tokens are productive, faster tokens are worth more, and the metric becomes tokens per second per user. Two changes since then widened the gap. First, every leading model, American or Chinese, moved to reasoning. Reasoning means several drafts, a review and an improvement, and that spends more compute at inference. A slow chip turns that into something crippling. Feldman calls this a huge advantage for Cerebras. Verification and guardrails add to it, since each check is another pass. Second, agents. They loop, so every wait is amplified, and they call on CPUs to act: visit a site, pull from storage, order a burrito. Feldman describes the accelerator as the brain and the CPUs as the body. More agentic work means more CPUs, which now face the same memory shortage.

The same architectural choice sidesteps the shortages that constrain everyone else. HBM is made by three companies, SK Hynix, Samsung and Micron, and is sold out. TSMC's CoWoS packaging, which Nvidia and AMD use as a silicon motherboard, is sold out. Capacity at 3 nanometers is congested. Cerebras uses none of them. It has no HBM, no CoWoS, and its chips are at 5 nanometers. Feldman frames this as a consequence of architecture rather than foresight, but it is why he can describe an industry constrained everywhere except in his own supply chain. The exception is data center space, which he calls a limiting factor for everybody.

The competitive claim is that Nvidia has already conceded the point. Its most durable moat was the belief that the GPU could do everything. Then it paid $20 billion for Groq, the number two player in fast inference, and moved fast to do it. Feldman says Cerebras sales were more than 10 times Groq's. The second claim is about CUDA. Two years ago every state-of-the-art model was trained in a CUDA flow. Now Gemini trains on TPUs and Claude on Trainium. Only OpenAI still trains with CUDA. He puts the share lost at 70%. In inference, he says, there is no moat at all: eight keystrokes move traffic from a GPU API to Cerebras. His own moat is architectural, because rivals cannot simply spend their way to fast. In training the pitch is simplicity. GPUs are small, so large calculations must be split across many of them, which only the best labs do well. A wafer can run data parallel, which is much simpler. He targets the next tier, not the largest labs. Nor does he predict one winner. x86 had two players for 20 years, and when the phone workload arrived both got zero share and ARM took it.

The business has changed shape to follow demand. Cerebras sells hardware for customers' data centers or rents capacity from its own. Revenue was split 50/50 last year, about 75/25 toward hardware last quarter, and maybe 50/50 this year. With the OpenAI deal he expects 30/70, hardware to cloud. That deal is north of $20 billion for 750 megawatts: 250 in 2026, 250 in 2027 and more in 2028, delivered as a full cloud through an API. Feldman credits OpenAI with looking at an exponential usage curve and not being afraid of what it says, while others struck bad deals for capacity. He is building in the Nordics for clean, cheap power and cooling, and keeping data centers on the same continent as customers, since light in fiber travels at about two-thirds of its speed. A separate AWS arrangement pairs Trainium for prefill with Cerebras for decode.

The obvious objection is that all of this rests on labs financed by venture capital, hedge funds and sovereign money. Asked directly about circular deals, Feldman gave a historical answer rather than a customer one. Bubbles, he says, are when supply gets ahead of demand. In the 1990s fiber was laid years before it was needed, and it took 6 to 8 years to fill. Now everyone is trying to build faster for demand that already exists, and only a small share of the world uses AI anywhere near its potential. That is a fair distinction. It does not answer the question asked. He said dozens of other customers are trying to place very big orders. He did not name them or give their share. On a market wobble that erased $1.5 trillion in a day, his reply was that public markets vote in the short run and weigh in the long run.

The cloud pivot sharpens the problem. If most revenue is to be rented capacity, Cerebras takes on the data center scarcity he calls a limit for everybody. His next meeting, he says, is with a data center provider. The CUDA story is also told largely through frontier labs and training, the market where he says Cerebras does not serve the largest labs.

Feldman's own forecast is that the model you use today will be the worst you ever use. If that holds, tokens per user keep climbing and the speed premium compounds. What stays unresolved is the financing. The exponential curve that OpenAI is unafraid of is the same curve that falls if the money behind it slows. Feldman says the demand is already here. A 750 megawatt order from a single customer is the thing to watch.

**The open question.** Feldman answered the circular-financing question with history: bubbles happen when supply outruns demand, and today's problem is the reverse. That may be true of usage and still miss the question of who pays for it. He did not quantify the lab-funded share of demand. The other customers were only "dozens" placing big orders, with no names and no share. With the OpenAI deal pushing the mix toward 70% cloud, Cerebras also takes on the data center and power scarcity he calls a limit for everybody. Watch how much of the 750 megawatts is live in 2026, and who else signs a multi-year lease.

Watch the full video at [https://www.youtube.com/watch?v=UEOSUSz--Ig](https://www.youtube.com/watch?v=UEOSUSz--Ig&ref=nnutshell.com).