AI cut a 6-month feature to 10 days, but operations resists

1. Observability as reliability proxy — ▶ 7:25

   In 2023, engineering leaders spent 5-20% of infrastructure spend on observability, but what they wanted was reliability. Microservices multiplied the data collected, so cost grew faster than output and the marginal reliability per dollar flattened. Grepr's bet is to extract signal upstream in the data stream, cut downstream cost, and feed those signals to whatever AI comes next. Plug it in, pass data through, and telemetry spend drops 90%. Troubleshooting and fixing is the second layer being built now.

2. Good engineers pull away — ▶ 11:26

   The best engineers fit AI into their workflow so that "the AI starts training them" in how to prompt it. As the team grew, the gap between okay and really good engineers multiplied 10x. Good engineers become better ones, and weak engineers do not suddenly become good. The reason: the more complexity your brain can hold, the more you can decompose and delegate. Naous caveats that his team is small, untested across 100 people, and that models cannot think through his distributed algorithms, such as the query translation layer, unaided.

3. Multi-agent workflow details — ▶ 18:25

   The stack is Windows with WSL, Claude Code and Opus 5.5. Naous recently shipped 20-PR batches in two weeks. Review has shifted toward tests and golden expectations. His trick is to have Claude spin up command-line sub-agents, which keep persistent sessions, rather than built-in ones, with an implementer and a tech-lead reviewer iterating on a PR doc. He reviews the design doc first, then lets work parallelize. He runs 3-4 sessions at once, since more loses context, and a human still reviews everything as one stack.

4. Six months became ten days — ▶ 23:18

   The latest project was auto-promotion, or shredding, in a data lake. Every field in a semi-structured log event becomes its own column in Parquet and Iceberg. It means working out metadata commits and coordinating across writers. It took about 10 days. Pre-AI, he estimates about 6 months for one person. That is a 10-20x improvement, and Casado pins down that this is production code for new products, not demos or mockups.

5. Where models run out — ▶ 27:57

   Casado says no model handles his distributed Gaussian splat renderer well, because behavior depends on the running system. Naous finds models work when performance benchmarks and constraints are defined. On query optimization the model initially declared the code fine until he pointed out repeated conjuncts. Grounded in a benchmark dataset of real queries, it then generated and tested hypotheses. How much memory to allocate, though, it has no idea about. That needs production experiments, and models are getting good at recognizing that and running one.

6. Start from change, not alert — ▶ 30:30

   Many AI SRE tools start from a lagging indicator like a metric spike and backtrack to root cause. Naous says that is why they don't work. Grepr inverts it. It clusters at scale to find what is different, and captures early-warning messages such as connections exhausted or memory exceeded. The premise is that most operational issues follow a change. Running internally and at a couple of customers, it has done well. Twice the flagged signal was ignored, and those turned into production incidents.

7. Detector feeds the reasoner — ▶ 35:08

   The generative model is not asked to predict. A separate predictive model scans a customer's petabyte of logs per day and lifts every new pattern to the AI. The AI then reads the code to find where a message comes from, checks whether it happened before, and runs hypotheses. Naous calls AI doing a human's query optimization job "very close". Running a distributed cluster in place of a human with a PhD is "not yet".

8. Overprovisioning versus the heavy tail — ▶ 38:09

   Casado argues code is an in-distribution manifold, while operations is heavy-tailed, exception-based and data-poor. Every added bit doubles the state space. Systems history shows smart algorithms for virtual memory, congestion control and adaptive routing losing to overprovisioning, and chaos engineering never worked. Naous replies that he has all the data, and the failures may be heavy-tailed but the debugging steps are a manageable set. The AI takes the human's role of collapsing the state space through a workflow. Casado concedes the point but says ops startups are having a tough go.

9. Security and irreducibility — ▶ 42:20

   Security looks harder to Casado, since an adversary deliberately hunts for rare states. Naous applies the same pattern: behavioral shifts in entities and cardinalities go to an AI, which generates candidate causal chains and checks them against observed data. Casado raises computational irreducibility, where you must run the system to know, and suggests models may solve only what needs no simulation. Naous counters that an AI can plan its own execution to gather information, perhaps simulating a scientist. Casado: "I don't think the universe gives up its secrets that easily."

10. Constrained domains versus nature — ▶ 53:34

    Casado expects AI to take off "like a wildfire" where the solution distribution matches the training distribution, as in code and images. Natural-world domains will be slower, with harder economics. Self-driving took 20 years and $170 billion to reach worse unit economics than Uber. Equilibrium will be economic, and AI is a universal function approximator. One lever is constraining the environment. Naous agrees, pointing to the automated warehouse as proof that we sometimes change the world to fit what AI can do.

11. Customers buy savings, not AI — ▶ 57:05

    When Grepr pitched AI-driven reliability, customers interrupted: "you said you can save 90%" of their Datadog bill, and that is what they want to buy now. Code sells itself because engineering productivity is one of the largest line items, and the value is obvious. Ops value is harder to price, since the benefit is an incident that didn't happen. Naous says operations is still pretty early.

Writing software and running software are different problems, and AI has so far solved only the first. The gap between them is the gap between a bounded task and an unbounded one. Everything else in this conversation follows from that.

The evidence on the solved side is concrete. Jad Naous recently built auto-promotion for a data lake, which turns every field in a semi-structured log event into its own column. That means working out metadata commits and coordinating across multiple writers. It took about 10 days. Before AI, he puts it at roughly 6 months for one person. That is a 10-20x gain on production code, not on demos.

The gain is not spread evenly. Naous says the best engineers work AI into their routine so that "the AI starts training them" in how to get the most from it. As his team grew, the gap between okay and really good engineers multiplied 10x. Weak engineers did not become good. Strong ones got stronger, because the more complexity you can hold in your head, the more work you can break up and hand off. He caveats this: his team is small, and the claim has not been tested across 100 people.

Code is the easy case because of what the models learned from. Martin Casado's framing is blunt. Models are distribution in, distribution out. Train on all the code ever written and you get a manifold that turns out to be manageable. Operations is different. Every bit added to a program doubles the possible states, and you pass the number of atoms in the universe quickly. Casado's history lesson is that engineers have always wanted smart algorithms for virtual memory, congestion control and routing. Every time, they ended up overprovisioning instead. Chaos engineering tried to tame the state space with product, and it never went anywhere. Casado's instinct is that operations is heavy-tailed, exception-driven and starved of data.

Naous does not dispute the difficulty. His own experience shows where the models fail. Asked how much memory a system should get, a model has no idea. That answer has to come from experiments in production. Even on query optimization, the model first declared its generated code fine. It only found improvements once he pointed at repeated terms and grounded the work in benchmarks and a dataset of real queries. The behavior he values is that the models now recognize what they cannot answer and go run an experiment.

His answer to Casado is architectural. Don't ask a language model to predict. Many AI SRE tools start from a lagging indicator, like a metric spike, and try to backtrack to root cause through a tangled system. He says that is why they fail. Grepr works the other way around. A separate predictive model scans a customer's petabyte of logs per day, finds patterns never seen before, and lifts them to the AI. The AI then reads the code behind the message, checks history, and runs hypotheses. The premise is that most operational trouble follows some change, so find the change first.

This is where Naous turns the heavy tail into an advantage. The failures may be endless, but the steps for debugging them are a limited set. A human used to walk through those steps to shrink a vast state space into something tractable. That was the missing piece in chaos engineering. The AI can take that role. Casado accepted this reframing, though he kept his view that operations startups have a tough road compared with coding ones.

The obvious objection is that none of this is proven at the level of running a cluster. Naous agrees. He calls query optimization "very close" and running a distributed cluster "not yet." His best evidence is partial. Grepr runs internally and at a couple of customers. In a couple of cases the flagged signal was ignored, and those turned into production incidents. That validates detection. It says little about whether the AI can reliably fix what it finds.

Security makes the objection sharper. Operations at least has no one trying to break it on purpose. Security has an adversary who deliberately searches for rare states. Naous applies the same pattern: behavioral shifts in entities and cardinalities become signals, and the AI proposes causal chains and tests them against observed data. Casado raises computational irreducibility. Some things you can only learn by running them, so models may solve only what needs no simulation. Naous replies that an AI can plan its own execution to gather what it needs, perhaps even simulate a scientist. Casado's closing position is "I don't think the universe gives up its secrets that easily."

The wider pattern is economic. Casado expects AI to spread fastest where the solution space matches the training set. In the natural world it will be slower, and economics will decide. His example is self-driving. After the DARPA Grand Challenge it took 20 years and $170 billion to reach worse unit economics than Uber. One lever is to constrain the environment. Naous makes the same point from robotics, where the automated warehouse worked because the whole thing was automated. The world gets changed to suit the tool.

The market shows the same gap. Code sells itself because engineering productivity is a top line item. Operations does not. Naous's customers hear a pitch about AI-driven reliability and ask to go back to the part about saving 90% of their Datadog bill. The value of an avoided incident is hard to price.

So the question stays open. Either operations has a manageable core of reasoning steps, as Naous believes, or it keeps the long tail of states that Casado fears. Naous has not shown how often his agent is right. Casado has not shown the tail is unreachable. Operations, by Naous's own account, is still pretty early.