Recorded 8 October 2026 · 56 min
The cloud now has a second kind of customer, and it wants different things from people. Matt Garman argues that AWS's core building blocks mostly hold up. But agents care about P999 tail latency, disposable databases, time-boxed permissions and sandboxes, and AWS is adding those on top. Garman interned at AWS in 2005, and the host introduces him as the first general manager of EC2. They talk about what the shift means for startups, the people who built AWS's early business.
The tension is supply. AWS is a $169 billion business growing 37%, with capex around $220 billion this year. Yet every layer from power to memory to TSMC capacity is a constraint, and every startup wants more GPUs than it gets. Garman defends reserving capacity for startups instead of selling it all to frontier labs. He rejects the bubble framing. He also concedes that most enterprise agents are still simple and human-supervised.
Startups as lifeblood — ▶ 2:58
Garman's 2005 internship project concluded that startups were AWS's most interesting customers. He estimates 30-40% of AWS revenue now comes from companies that were startups at some point in AWS's lifetime. They also push the roadmap: banks will want in 5 years what startups want today. Startups now begin with $200 million of funding and billion-dollar valuations. They still ask about architecture, security and IAM for when they have more than three employees. That is why they pick AWS over a neocloud.
Agents care about tails — ▶ 6:39
Agents block on P999 S3 latency, where people rarely notice. That tail sensitivity is why "agentic workflows tend to perform better on AWS than anywhere else." AWS is mostly tuning existing services for both audiences, such as starting a database in 3 seconds. It also has a few agent-specific products, including AgentCore, Bedrock and AWS Context, which is in beta. Context builds a layer across Aurora, S3 and other stores so agents can find data that humans would access differently.
Accounts in 30 seconds — ▶ 10:29
Most customers tell Kiro, Claude or Codex to build on AWS with their credentials. But a user with no cloud who just says "deploy" often lands with a partner that has an easier layer. AWS's answer is a new account flow, still rolling out. It needs no credit card, accepts a Gmail sign-up, and sets VPC and IAM defaults behind the scenes. Users are in a full AWS account in under 30 seconds. There is no later migration, so scaling up never forces a choice.
Disposable databases, new primitives — ▶ 12:58
Agents want a database that does a little work and disappears, so full production durability is overengineered for them. Garman refuses to ship a non-durable tier, because nobody knows at creation whether a database lives a day or grows into production. The aim is cheap to discard but able to grow up. New building blocks are needed too: compute sandboxes, gateways, and time-boxed, fine-grained agent permissions instead of a person's full access. Firecracker microVMs, built about 10 years ago, underpin many sandbox startups.
Rationing GPUs on purpose — ▶ 17:09
AWS could sell every accelerator to the big frontier labs and call it a day. It chooses not to, because a healthier ecosystem and tomorrow's big customers matter. It eventually says yes to about 60% of requests, sometimes later, in another region or in a different configuration. Capex is about $220 billion for 2026, and AWS is buying 2 million NVIDIA GPUs over a couple of years. Garman says: "who knows if that's enough." Constraints include power, data centers, capital, memory, chips and construction labor.
Why it isn't a bubble — ▶ 21:30
Customer concentration at AWS is single-digit percentages at the highest. Neoclouds, he says, sometimes show 30-60% with one or two customers. Most AWS usage is core compute, storage and inference embedded in applications, and customers report positive ROI "almost to a person." He grants that not every billion-dollar startup will make it, as in venture generally. The internet bust left Google and Amazon standing. He does concede that GPU ramp-up has cost AWS much of the elasticity it used to have.
Bring your own power — ▶ 25:06
Fifteen years ago AWS asked utilities for tens of megawatts and got them. Now it pays for solar and nuclear projects, both grid-connected and behind the meter, and plans power and transmission 20 years out. Server planning has moved from multi-quarter to multi-year. His rule from The Goal: "there's never one constraint there's always just the latest constraint." That might be power, memory, TSMC capacity, HBM or connectors. Location matters too, since power in Indonesia does not help Germany. AWS tracks supply four to five tiers down.
Own silicon pays off — ▶ 33:42
Graviton began as ARM cores on an offload card from an acquired team. It has been 20% cheaper with 20% better performance for 5-6 years. Over 90% of the top 100 customers use it, and some halved their server counts. Trainium 3 is sold out until roughly the end of next year. Most Bedrock traffic runs on it, and Anthropic and OpenAI have deals to build on it. Despite the training name, Garman calls it "maybe the best inference chip on the market right now." Trainium 4 is announced.
Enterprise agents stuck — ▶ 40:35
Most enterprise agents are simple and keep a person in the loop. Two things hold them back. First, firms copy Bob's steps 1-5 instead of letting an agent parallelize and try 50 approaches. Second, they do not trust agents not to delete a production database. Evals, labeled data and drift measurement are unsolved, "maybe appropriately" slowing adoption. AWS's forward-deployed engineers work for about 45 days, teach the customer to build evals, then leave. Customers do not want to be beholden to an outside workforce for years.
Bedrock and open weights — ▶ 45:59
Enterprise data is the most valuable asset, so Bedrock keeps data inside the customer's VPC. Model providers never see prompts. Garman says that is why OpenAI workloads are migrating to Bedrock and Anthropic is growing there. AWS took heat three years ago for being slow, and he says it chose to build durable foundations. He is also ramping support for open-weights models. Customers want to post-train or distill them on proprietary data for a better, cheaper model. Most do this in SageMaker today, and he says AWS must make it easier.
Security at machine speed — ▶ 49:44
CEOs mostly ask how to trust agents, which means permissions, sandboxing, guardrails and whether a human stays in the loop. Garman agrees that powerful models can become attack surfaces, but sees opportunity as well. The Continuum service uses such models to scan a customer's environment for vulnerabilities. It prioritizes them using context on permissions and compensating controls. He says customers will need "security at machine speed, not at human speed," and calls Continuum incredibly popular.
An agent does not use the cloud the way a developer does, and AWS is rebuilding parts of itself accordingly. Most of the core building blocks survive the shift. What changes is what gets measured. A person rarely cares about P999 latency on S3. An agent does, and it stalls when the tail is slow. Matt Garman says this is why agentic workflows tend to perform better on AWS than anywhere else. The engine was tuned for tails long before anyone was building agents.
The harder changes come where old design assumptions break. Agents want to create a database, do a little work and throw it away. Production-grade durability is overengineered for that. But Garman refuses to ship a non-durable tier, because nobody knows at creation whether a database will live for a day or grow into the production system. The target is a database that is cheap to discard and can still grow up. Permissions follow the same logic. An agent should not inherit a person's access. It needs time-boxed, fine-grained rights, inside a sandbox. So AWS is adding genuinely new primitives: compute sandboxes, gateways and agent permissions. The sandbox layer leans on Firecracker microVMs, which AWS built about 10 years ago and which most sandbox startups now use.
Onboarding gets the same treatment. Most customers simply tell Kiro, Claude or Codex to build on AWS and hand over credentials. A user with no cloud who says "deploy" often ends up with a partner that has an easier layer on top. AWS's response is a new account flow. Sign up with a Gmail address, no credit card, and VPC and IAM defaults are handled behind the scenes. The user is in a full AWS account in under 30 seconds. There is no later migration.
The audience for all this is startups. Garman's 2005 internship project concluded they were the customer to chase. He now estimates 30-40% of AWS revenue comes from companies that were startups at some point in AWS's lifetime. They also set the roadmap: banks will want in 5 years what startups want today. Startups now begin with $200 million of funding and billion-dollar valuations. They still ask about IAM and security, which is why they choose AWS over a neocloud.
That makes supply the real story. AWS's capex is about $220 billion for 2026, more than Garman thinks any company has spent in a year. It is buying 2 million NVIDIA GPUs over a couple of years. It could sell every accelerator to the frontier labs and chooses not to. Startups get reserved capacity, and AWS eventually says yes to about 60% of requests, sometimes later, in another region or in a different configuration. Garman calls it a good problem to have, "but it's a problem nonetheless."
The limits on supply keep moving. Fifteen years ago AWS asked a utility for tens of megawatts and got them. Now it pays for solar and nuclear projects, grid-connected and behind the meter, and plans power and transmission 20 years out. Server planning has stretched from a few quarters to multiple years. Garman takes his rule from The Goal: "there's never one constraint there's always just the latest constraint." This month it is power. Next it may be memory, TSMC capacity, HBM or connectors. Where capacity sits matters too, because power in Indonesia does not help Germany. AWS tracks supply four to five tiers down. The lesson came from the disk shortage after the Thailand floods.
Is the spending a bubble? Garman's answer is structural. AWS's biggest customers are single-digit percentages of capacity at most. Neoclouds, he says, sometimes show 30-60% tied to one or two customers. Most AWS usage is core compute, storage and inference embedded in production applications. Customers tell him they see positive ROI "almost to a person." He grants that not every billion-dollar startup will survive. That has always been how venture works, and the internet bust still left Google and Amazon standing.
Custom silicon is the other lever. Graviton has been 20% cheaper with 20% better performance for 5-6 years. Over 90% of the top 100 customers use it, and some halved their server counts. Trainium 3 is sold out until roughly the end of next year. Most Bedrock traffic runs on it, and Anthropic and OpenAI have deals to build on it. Despite its training name, Garman calls it "maybe the best inference chip on the market right now."
Now the harder question. The demand side is less mature than the capex implies. Garman says most enterprise agents are simple and keep a person in the loop. Two things hold customers back. One is imagination. Firms copy Bob's steps 1 to 5 instead of letting an agent parallelize and try 50 approaches. The other is trust. Nobody wants an agent that accidentally deletes a production database, and he calls that nervousness "maybe appropriately" slowing adoption. Evals, labeled data and drift measurement are problems he doubts anyone has fully solved. AWS's answer is forward-deployed engineers who work for about 45 days, teach the customer to build evals, then leave. Customers, he says, do not want to depend on an outside workforce for years.
The data question follows the same logic. Bedrock keeps data inside the customer's VPC, and model providers never see prompts. That is the pitch against sending everything to a frontier lab. Garman is also ramping support for open-weights models that customers post-train or distill on proprietary data. Most do that in SageMaker today, and he says AWS must make it easier. On security he sees both a risk and a product. Powerful models can attack a customer's environment. The Continuum service uses them to find and prioritize vulnerabilities instead.
Here the bubble defence gets thin. It rests on ROI from today's simple deployments and on inference tucked inside applications. It does not rest on the autonomous agent swarms that most enterprises have not yet trusted with production systems. Garman asks himself how anyone is meant to estimate 2028 demand, then says AWS is buying anyway. He is probably right that the cloud business survives any correction. Whether trust arrives before the buildout is the part nobody on this call can answer.
The open question. Garman's bubble defence rests on today's evidence: low customer concentration, positive ROI, and inference embedded in production applications. Yet the capex is sized to demand he admits is hard to estimate for 2028, and he says most enterprise agents are still simple, human-supervised and held back by trust and evals. The unexamined step is how much of the $220 billion depends on autonomous agent swarms that most customers have not yet dared to deploy.
Watch the full video at https://www.youtube.com/watch?v=rn_afJaPldg.
Comments ()