Modal stitches 25 clouds into one pool, escaping three-year GPU blocks

1. 50x in two years — ▶ 0:59

   Modal sells a compute and storage platform to customers including Decagon, Suno, Runway and Cognition. The workloads are mostly inference and sandboxes, with some training, fine-tuning and batch jobs. Revenue, users and usage are up at least 50x since the last time Bernhardsson appeared on the podcast two years ago, though he does not remember the exact starting point. The pitch is a software SDK: customers deploy and scale in production while Modal manages all the capacity underneath. The company now has about 200 employees, roughly 150 of them engineers.

2. Execution, not orchestration — ▶ 4:53

   Modal began as a workflow orchestration idea, building on Luigi, the open source framework he wrote at Spotify. He realized the bigger problem was executing the compute, so he went deep and built a file system and container runtime. Stable Diffusion in fall 2022 became the killer app when the team was 4-5 people. Revenue went from $100 a month in November 2022 to $2,000 in December and $30,000 in January 2023. One customer, probably Mage in New York, was 80-90% of revenue in Q1 2023.

3. Old tools don't fit GPUs — ▶ 10:30

   Every new era outgrows its predecessor's tools, as the cloud forced Kubernetes and Docker into existence. Those were built for steady CPU backends running at low utilization, where cost barely matters. GPUs flip that: expensive chips, utilization pressure and huge models loaded over the network. So Modal built its own file system, CPU and GPU snapshotting, autoscaler and scheduler, because you cannot wrap the old stack and get the new behavior. Going multi-region early and treating capacity as one global pool was, in hindsight, a "fateful" decision.

4. From spot to 25 clouds — ▶ 13:21

   Until the end of last year most capacity was spot or one-year reservations from AWS, GCP, Azure and Oracle, aggregated across hundreds of regions. That broke as Modal grew and the market tightened. Now it works with almost 25 clouds, often on 3+ year deals with meaningful money up front. It built a procurement desk of four people calling neoclouds and a 10-person team managing bare metal, bad drives and failing GPUs. Modal owns nothing. He thinks software companies shouldn't hold hard assets, but "everything's on the table" for 2027 capacity.

5. Scarcity isn't cornered — ▶ 18:06

   He disputes that capacity must be bought now. Nvidia ships several hundred thousand Blackwells a month, anyone paying the top price gets GPUs, and he likens the panic to "the great toilet paper shortage". Hardware prices are like gravity and eventually fall; the scarcity rent goes mostly to Nvidia and neoclouds. Reservations leave Modal effectively long GPU prices, so rising prices don't worry him. Collapsing prices do, for anyone locked in at the top. A veteran of 1999, he cites the Hunt brothers' failed silver corner and ramps on a plan matched to demand growth.

6. Janky-warehouse capacity — ▶ 21:19

   Anthropic and OpenAI are buying up everything, but Bernhardsson speculates they prefer tried-and-true vendors, partly for large training blocks, and that Nvidia wants allocation spread toward neoclouds. That leaves riskier small clouds for Modal: 32 nodes in Armenia, 128 in Indonesia. Vendors get a spec sheet covering IOPS, CPU scores and bandwidth, plus one-node acceptance testing. Commitments of tens or hundreds of millions of dollars still involve "leap of faith". He keeps an internal "modal max" score, like SemiAnalysis's ClusterMax, that weighs reliability and support responsiveness.

7. 2027, non-Nvidia, maybe — ▶ 26:13

   He has predicted the year of non-Nvidia accelerators for 5-10 years and been wrong each time, but says 2027 might finally be it. AMD is scoring well on some workloads, TPUs work widely inside Google and Trainium's software team is strong. Fixing the software layer is the unlock. Some run Nvidia for prefill and Cerebras for decode, mixing compute-bound and memory-bound stages, though deployment gets complex. He adds that TSMC and ASML may be a deeper bottleneck than Nvidia, and that "monopolies don't always last".

8. How neoclouds presell — ▶ 28:25

   Neoclouds scale fast through a financing construct. They get a lab such as Anthropic or OpenAI to sign a roughly 5-year chip contract, take that to Wall Street to raise the money, buy the hardware and deliver. They effectively presell chips they haven't bought, and one has hundreds of billions in pipeline despite barely any capacity. Modal's PLG model can't lock long contracts the same way, so scaling is financially harder. He is skeptical that compute becomes a Henry Hub or Brent-style commodity, given differences in networks, locations, disks and reliability.

9. Sandboxes nearly killed — ▶ 32:10

   Functions, launched in 2021, found product-market fit in 2023 and are now about two-thirds of revenue, mostly inference. Sandboxes launched in 2023 after he and his co-founder reasoned over lunch that LLMs output code and need somewhere to run it. A prototype shipped in a day or two. Revenue was insignificant into summer 2023 and the team discussed shutting it down. Then LLMs began generating full apps, and growth came from vibe-coding companies like Lovable, Ramp's background agents and neolabs running reinforcement learning. Sandboxes are now about a third of revenue.

10. Ephemeral beats lift-and-shift — ▶ 43:29

    Most sandbox use today is lift and shift: a dev box in the cloud with an agent driving it. He expects long-running agents to behave like durable execution workflows, spinning up a sandbox in ~100 milliseconds, running a few commands, snapshotting and shutting down. Faster wake and sleep makes compute more efficient and enables forking, so one agent can branch into hundreds. CI is the example: run every test in its own sandbox, across a thousand of them. Today each shard repeats installs and setup, which takes minutes. Snapshot once and clone instead.

11. Hugging Face incident — ▶ 47:30

    An app deployed on Modal was hacked and used as a springboard by agents that then exploited Hugging Face. He insists Modal itself wasn't hacked, just as an app compromised on AWS doesn't mean AWS was. His lesson is more restrictive security defaults, as AWS made S3 buckets harder to expose publicly. Modal now puts agents in sandboxes and tells them to escape, started a few months ago, and has found nothing so far. Pentests may stretch from every six months to yearly. He doesn't think big security vendors are cooked, since they hold enterprise relationships and give buyers someone to blame.

12. Agents can't unlearn syntax — ▶ 54:49

    Nearly everything that makes developer experience good also makes agent experience good: docs, descriptive errors, fast feedback loops. Agents also push infrastructure toward imperative SDKs and away from YAML. A side effect is that training data makes deprecation hard, since agents keep writing old Modal syntax. Modal has become more conservative about removing features and must update every example online. It runs internal benchmarks asking agents to deploy Stable Diffusion or rank sandbox vendors, to see whether they recommend Modal. The answer is clearer content, not spam repos, which he says he doesn't love.

13. FDEs as product pipeline — ▶ 57:48

    Modal grew forward deployed engineers from 4 at the start of the year to about 24. They tune LLM performance with customers, including draft models and serving parameters, catch overprovisioned CPU and memory, and help migrate code during deals. The rule is that Modal must not become a services business. Every FDE task should prompt "how do we turn this into product?" It has hired no product managers yet. He argues infra product management needs rare strategic thinking, or you end up with 35 similar-looking features.

14. Duration transformation — ▶ 1:18:31

    Almost all GPU capacity is sold as a three-year rectangle starting 3-6 months out, but most demand isn't flat. It scales through the day, grows and bursts. Modal can give a customer 100 GPUs immediately, no salesperson, because aggregate demand across thousands of customers is smooth. He calls this duration transformation. He reduces Modal's value to two things: developer experience and flexible capacity, sliced finely thanks to fast scaling and GPU snapshotting. The median sandbox lasts about 8 seconds and may get shorter.

15. Why labs aren't commodities — ▶ 1:21:20

    Inference providers should exist separately from model builders because the best open model keeps changing. A lab holding its own capacity would see demand spike at launch and drop when a rival ships. Modal shifts 50% of the GPUs for one model to another automatically. Asked if that makes LLMs commodities, he says no: Taylor Swift releases an album, tours, then records again, and keeps her brand and margin. Labs stay viable if margins repay training costs. He thinks intelligence isn't plateauing, but expects humans to compete in domains with diffuse targets and long feedback loops.

Modal buys time in three-year blocks and sells it in seconds. The median sandbox on its platform lasts about 8 seconds. The GPUs underneath come from almost 25 clouds, usually on terms of three years or more with meaningful money paid up front. Everything Erik Bernhardsson has built is a machine for earning the spread between those two durations.

The market gives him no help on the buy side. Almost all GPU capacity is sold as a three-year rectangle that starts 3-6 months out. Real demand is not a rectangle. It swells and shrinks across a day, grows over months and arrives in bursts. Bernhardsson's claim is that the mismatch washes out in aggregate. With many thousands of customers, summed demand is smooth. So Modal can hand a customer 100 GPUs immediately, with no salesperson, and absorb the awkward shape of everyone's usage itself. He breaks the value proposition into two parts: developer experience and this flexible capacity.

The same logic explains why inference providers should exist apart from model builders. The best open model keeps changing. When a new Chinese lab ships a leading model, demand for it jumps and demand for the previous leader falls. A lab holding its own capacity would ride that curve up and down. Modal just moves 50% of the GPUs serving one model to another, and nobody has to tell the system to do it. Aggregation turns a lab's volatility into a smooth line.

Slicing capacity this finely requires a stack that the old tools did not offer. Kubernetes and Docker were built for steady CPU backends running at low utilization, where cost barely matters. GPUs flip every assumption. The chips are expensive, utilization matters, and enormous models must be loaded over the network. So Modal wrote its own file system, container runtime, CPU and GPU snapshotting, autoscaler and scheduler. Bernhardsson's view is that you cannot wrap the old stack and get the new behavior. Treating capacity as one global multi-region pool from early on was one of the most consequential choices.

The proof arrived with Stable Diffusion in fall 2022. Image generation is an autoscaling problem: containers have to scale up and down fast. Revenue went from $100 a month in November 2022 to $2,000 in December and $30,000 in January 2023. One customer, probably Mage in New York, made up 80-90% of Q1 revenue. Diversification followed, but the episode shows what the stack was really good at.

The obvious objection is that all this smoothing sits on top of a duration bet. Modal must hold supply for years to resell it by the second. Early on, it could dodge the problem by using spot and one-year reservations from the hyperscalers and aggregating across hundreds of regions. That stopped working as Modal grew and the market tightened. It now has a four-person procurement desk calling neoclouds and a 10-person team managing bare metal, bad drives and failing GPUs. It rents everything. Bernhardsson thinks software companies should not hold hard assets, and he is wary of the financing complexity of buying. Equity is dilutive, and debt wants locked-in long-term revenue to service it. But he says everything is on the table for 2027.

The neoclouds show what the alternative looks like. They sign a lab to a roughly 5-year chip contract, raise money against it on Wall Street, buy the hardware and deliver. They effectively presell chips they do not yet own, which is how one of them has hundreds of billions in pipeline with barely any capacity. Modal cannot do this. Its product-led model does not lock customers into long contracts, so scaling that fast is financially harder. The practical consequence is that the best use of any new capital is capacity prepayments. Salaries, he says, are a rounding error against the GPUs.

Where does Modal find supply that the labs have not already taken? In the unglamorous places. Bernhardsson speculates that Anthropic and OpenAI prefer tried-and-true vendors, partly because they need very large training blocks, and that Nvidia would rather spread allocation toward neoclouds. That leaves smaller, riskier clouds for Modal: 32 nodes in Armenia, 128 in Indonesia. There is a spec sheet and a one-node acceptance test, but commitments of tens or hundreds of millions of dollars still involve a leap of faith. He keeps an internal score of which vendors are reliable and responsive, a private cousin of SemiAnalysis's ClusterMax.

His view of the market is more contrarian than his behavior. He does not accept that capacity is cornered. Nvidia ships several hundred thousand Blackwells a month, and anyone willing to pay the top price gets GPUs. Treating this as a famine is, in his words, like "the great toilet paper shortage". Hardware prices behave like gravity and eventually fall. Because Modal's reservations are locked at set rates, it is effectively long GPU prices, so rising prices help it. What worries him is the reverse. A price collapse would hurt anyone who bought supply at the top, and the risks are correlated. Prices fall, utilization falls and the market for reselling capacity vanishes together. He worked through 1999 and the financial crisis, and he points to the Hunt brothers' failed attempt to corner silver. Hence an aggressive plan that tracks demand growth and not a corner.

On the demand side, the surprise was sandboxes. Modal launched them in 2023 after he and his co-founder reasoned that LLMs output code and need somewhere safe to run it. A prototype shipped in a day or two. Revenue was insignificant, and the team discussed shutting the product down. Then models got good enough to generate full apps. Growth came from vibe-coding companies like Lovable, from Ramp's background agents and from neolabs running reinforcement learning. Sandboxes are now about a third of revenue, with functions making up two-thirds. Bernhardsson admits he would never have bet on revenue this large.

The sandbox vision then pushes the duration thesis further. Today's usage is mostly lift and shift: a dev box in the cloud with an agent driving it. He expects long-running agents to act like durable workflows. Each one wakes a sandbox in about 100 milliseconds, runs a few commands, snapshots and sleeps. Continuous integration shows the payoff. Instead of sharding a test suite and repeating minutes of installs per shard, snapshot the setup once, clone it and run every test in its own sandbox across a thousand of them. The blocker is overhead, so cutting the cost of starting and stopping is the whole game. Shorter sandboxes widen the gap between what Modal buys and what it sells, and make aggregation more valuable.

He is candid about what he does not know. Asked what product comes next, he says that claiming to know exactly what to build would be "delusional". He is bullish on small and mid-size training, batch jobs and low-latency voice, but frames Modal as a maker of general building blocks. The forward deployed engineers feed that approach. The team grew from 4 to about 24 this year, tuning LLM serving, catching overprovisioned customers and migrating code. Each task is meant to prompt the question of how to turn it into a product so that no FDE is needed.

The hardest counter-argument is about the layer above. If the best model keeps changing and Modal moves GPUs between them overnight, are the models commodities? Bernhardsson says no, and asks whether Taylor Swift is a commodity. Labs keep their brand and margin as long as new releases repay training costs. He does not think intelligence is plateauing, though he expects humans to stay competitive where targets are diffuse and feedback loops are long. His analogy is a hedge rather than an answer. He does not explain how a lab differentiates when its customers can switch overnight.

He is similarly hedged on hardware. He has called the year of non-Nvidia accelerators for 5-10 years and been wrong each time. He now says 2027 might finally be it, and also that TSMC and ASML may be the deeper bottleneck.

The security episode shows the cost of the agent bet. Agents that exploited Hugging Face used an app deployed on Modal as a springboard. He says Modal itself was not breached, and that it now sets tighter defaults and runs agents trying to escape its own sandboxes. Nothing has turned up yet.

What stays unresolved is whether smooth aggregate demand survives the conditions that would stress the supply side. Modal's spread works while customers keep growing and prices hold. Bernhardsson says it is long GPU prices, and in the same breath names the scenario where prices, utilization and resale demand all drop at once. Modal then holds multi-year commitments with no easy exit.