ElevenLabs expects its audio research lead to shrink, builds moat elsewhere

1. Polish dubbing origin — ▶ 1:06

   Founded in 2022, before ChatGPT, when crypto and the metaverse absorbed everyone's attention, which left room to focus on AI. The trigger was Polish film, where one voice still narrates every character and the emotion and intonation vanish, as it did in his childhood. The ambition: original voice, emotion and intonation carried across languages, with the Hitchhiker's Guide Babelfish as a north star. In 2022 they knew content would need unlocking across modalities, but not how fast static would turn interactive. A model that does this well shipped a week before recording.

2. Research didn't exist yet — ▶ 5:53

   Dubbing needs transcription, translation and speech regeneration, and in 2022 all three were weak. Everything sounded robotic, nothing crossed the uncanny valley, and the best open-source repo was unstable and slow, though Nvidia had decent speech-to-text work. Creators added a second obstacle: they wanted dubbing eventually, but first needed to fix recorded lines in post, hear a script narrated before recording, or replace their own voice-over. So the team parked the language shift and built the speech model itself, the first text-to-speech model to cross human-like quality.

3. Honda-style lab structure — ▶ 12:45

   ElevenLabs describes itself as research plus product deployment, echoing a Honda story the host tells: a research lab for engines, with manufacturing applying the output. The structure is a focused research lab, research engineers who carry work into product, many small teams of usually fewer than 10 people with autonomy, then go-to-market and deployment worldwide. Uses span marketing, such as Ramp's Super Bowl ad, call-centre agents for Deutsche Telekom, inbound sales qualification, and a Polish government agent that follows up healthcare appointments and asks patients how they feel.

4. The audio bottleneck test — ▶ 14:48

   Research is solely audio, because the remaining problems are architecture and the work mixes science with art; voices are subjective. The test for any product: is there a big bottleneck in audio? Two years ago lip dubbing and avatars failed it. Video quality was the constraint, and the best audio could not rescue the experience, so the effort was paused. Open-source video models now justify revisiting it, but the company will not train text-to-video models from scratch. It targets existing assets where audio is the missing layer, as in its new ads engine.

5. Moat beyond research — ▶ 18:47

   The audio focus is a defence against larger labs, and the research edge may not stay as large. So product and ecosystem carry weight: brand, trust and a marketplace where people create a voice, ElevenLabs authenticates it, and they earn passively as it is used. It lists 20,000 voices. The company fills gaps itself by recruiting people for missing accents or demographics. The voice behind George in the Reader app is a voice actor who has worked with the company a long time and is paid every time someone listens.

6. Platform, not coding — ▶ 22:23

   The shape is model, then platform, then application, with communication as the unifying theme: research, one platform, then applications like 11 Reader. Coding and knowledge work are explicitly out, "not our strength, not our domain", despite being a vicious battle. He sees probably three platforms handling these interactions and wants to lead for communication: one place to set up integrations, knowledge, brand and assets, then deploy across the entire customer journey, from marketing and sales to support and operations.

7. Telekom land-and-expand — ▶ 24:08

   Deutsche Telekom started with marketing, because two years ago agents were neither reliable nor quick: audio for a podcast and app, then ads. Support followed, with voice agents tied to Telekom's knowledge for product, refund and billing questions, and lately an agent inside the network that joins calls, books appointments and translates in real time. The rule is to prove impact, not concepts, before scaling. Integrations with CRM systems come next, then testing, which he calls the hardest part of voice agents: simulated calls, small scale, then full deployment, with monitoring afterwards.

8. Engineers who feed product — ▶ 29:17

   At Palantir his first month meant flying to Aberdeen to work beside a North Sea customer, a shock after BlackRock, where outgoing email was reviewed. ElevenLabs puts forward deployed engineers in the product team, not go-to-market, so lessons flow back into the product. Skip that step and the work is "just services". With it, a customer's domain expertise stays theirs while the product experience built on it becomes abstracted and reusable for the next company. The host calls this a second form of R&D, and he agrees.

9. AI-flattened organisation — ▶ 33:41

   No titles, few layers and best idea wins, all borrowed from Palantir, where he sat three or four steps from the top. ElevenLabs runs about 5 layers at most and hopes for fewer. Most managers have close to 10 direct reports. That only works if AI summarises what teams are doing, so leaders hear from the engineer or the person with the client, not a manager's digest, and shift from reactive to proactive. It rests on a deliberate choice to keep docs and data open to everyone in the company.

10. Engineers in every team — ▶ 37:53

    Small teams let AI adoption spread bottom-up rather than by top-down mandate. Engineers sit inside talent, ops and legal teams to automate work and lift others' use of AI, and he expects the biggest and smallest companies to do likewise. Legal and revenue operations stay central. A voice AI SDR on the website lets visitors talk instead of filling in a drop-down form, and they leave far more about their problems and use cases. It is built centrally and refined by each local team.

11. Where revenue comes from — ▶ 43:30

    There are three core product lines plus three developing bets, and each core line holds 20-30 different things. The biggest revenue lines are agents and creative, and the number of conversational-agent use cases is "skyrocketing". Fintech adopts fastest despite slightly different regulation, with Revolut named. Healthcare and telecoms come second, and retail and e-commerce only started moving this year. He lists those four as the biggest verticals.

12. Giving voices back — ▶ 49:46

    Voice carries recognition and emotion, and over 10,000 people who lost theirs to ALS, cancer and other causes have had it restored. A musician is touring the UK with his old band using an AI voice. A congresswoman addressed Congress with one. Tim Green, a former NFL player and author with ALS, hosts a podcast in an ElevenLabs voice and won an Emmy this year; hundreds to thousands asked how to get theirs back. The company partners with 800 organisations. Safeguards are acknowledged as the second part of the story, without detail.

13. Imperfection reads as human — ▶ 54:13

    A perfect voice agent with no filler did not sound human. Adding the ums made people respond that it was "so human" and good to speak with. Voice carries pauses, intonation and imperfection that text lacks, which is why it is a hard research problem: every voice sounds different to every listener. Benchmarking text-to-speech is extremely hard because each model ships different voices, making them uncomparable. The host adds that audiences admire authenticity far more than perfection.

14. No sale, three or four offers — ▶ 56:49

    Three or four concrete acquisition offers, the last in June last year, all declined. "AI is changing the world. We can build at the frontier of that change." The host argues that at 31 he would be selling his best idea with decades ahead. Staniszewski agrees it is likely his best idea and best timing, and says he does not know how the world looks in 5 to 10 years, naming universal high income and societal adoption as open questions. The best move for the world and for the company is to keep building. His co-founder is also all in.

15. Voice as the interface — ▶ 1:04:01

    As intelligence keeps improving, the next bottleneck is how people access and collaborate with it. Decades of keyboards, screens and code made humans learn the machine's language; voice flips that. Within 12 months he expects systems that combine IQ with EQ: they understand how you feel, adjust, pause, think and re-enter the conversation. The host likens it to the telegraph versus the telephone. The Babelfish ambition stays, delivered into existing devices rather than a device of its own, and he hopes the phone goes back in the back pocket.

A pure research lead in audio is a wasting asset, and Mati Staniszewski plans as if he knows it. ElevenLabs builds frontier text-to-speech, speech-to-text and conversation orchestration models, and he believes nobody else has comparable technology. Yet he concedes that over the long term the advantage on pure research may not be as big as it is today. Asked whether the relentless audio focus is a defence against the larger labs, he says yes without hesitation. So the company runs as a conversion exercise. It is turning a temporary model lead into things that outlast model leads: a marketplace, deep deployments inside customers, and a claim on how people will talk to machines.

The lead itself came from necessity, not strategy. The founders began with dubbing, triggered by Polish cinema, where one voice still narrates every character and the intonation disappears. Dubbing has three steps: transcription, translation and regenerating speech in the target language. In 2022 each was weak. Everything sounded robotic, nothing had crossed the uncanny valley, and the best open-source speech repo was unstable and slow to generate. Creators made the idea harder to sell. They wanted dubbing one day, but first they needed to fix a badly recorded line in post, hear a script narrated before filming, or hand the voice-over to an AI. So the team set the language shift aside and built speech generation, the model it says first crossed human-like quality. A model that does the original dubbing idea well shipped only a week before this conversation. The product idea survived. The order flipped.

The same discipline governs what ElevenLabs refuses to build. The test for any new product is whether audio is the bottleneck. If something else limits the experience, audio is not a superpower and the company stays out. Two years ago that ruled out lip-sync dubbing and avatars, because video quality was the constraint and the best voice in the world would not rescue the result. Open-source video models have since improved enough to revisit the space, but ElevenLabs will not train text-to-video models from scratch. It works where an asset already exists and audio is the missing layer, such as taking an ad international. Coding and knowledge work are "not our strength, not our domain." The structure borrows from Honda: a focused research lab, research engineers who carry work into product, and many small teams, usually under 10 people, with autonomy to chase specific customer problems.

Research alone does not make a business, so the first durable layer is the ecosystem. People create a voice, ElevenLabs authenticates it, and they earn passively whenever it is used. The marketplace holds 20,000 voices across accents, ages, languages and styles, and the company recruits people where it sees gaps. The host listens to research dossiers read aloud by a voice called George in the Reader app. A voice actor stands behind George, and he is paid every time someone listens. Staniszewski puts brand and trust alongside the marketplace as the reason product and ecosystem matter more as the research gap narrows.

Subjectivity is part of why audio is hard to commoditise, in his telling. Every listener has a preference for how a voice should sound, and capturing and delivering those preferences is a tricky challenge. Brands face the same problem when they define how they communicate. He credits Palantir's notion of an artist colony for teaching him that the work needs some understanding of art. As everyone gains the ability to create everything, he says, what separates a good product or deployment is design language and how tasteful it is.

The second layer is deployment, and the evidence is Deutsche Telekom. It started with marketing: audio for a podcast and an app, then ads. It started there because two years ago agents were neither reliable nor quick. Support followed, with voice agents wired into Telekom's knowledge to handle product questions, refunds and billing. The latest step puts an agent inside the network, where a caller can ask it to join, book an appointment or translate in real time. Every step rests on one rule. Prove the impact before asking for scale. Then comes the unglamorous work: connecting CRM systems, making the agent follow the customer's logic, and testing with simulated calls, then small live volumes, then full scale. He calls testing the hardest part of voice agents. Deployment also does not end at launch. Evaluation and refinement continue.

Forward deployed engineers do this work, a habit from Palantir, where his first month meant flying to Aberdeen to sit with a North Sea customer. At BlackRock he had barely met a customer. The twist is where the engineers sit. At ElevenLabs they belong to the product team, not go-to-market, because their second job is carrying what they learn back into the product. Without that step the work is "just services". With it, each customer's domain expertise stays theirs while the product experience built on it becomes reusable by the next company. The host calls this another form of R&D, and Staniszewski agrees. The revenue shape fits. Agents and creative are the biggest lines, fintech adopts fastest, healthcare and telecoms follow, and retail and e-commerce only began moving this year. Even a Polish government agent that follows up healthcare appointments and asks patients how they feel comes from the same pattern.

The third layer is a bet about interfaces. Staniszewski argues that as intelligence improves, the next bottleneck is how people reach it. Decades of keyboards, screens and coding languages made humans learn the machine's terms. Voice flips that. Within 12 months he expects systems that combine IQ with EQ: they will understand how you feel, adjust, pause, think and re-enter the conversation. The host reaches for the telegraph and the telephone, one needing a learned code and the other needing nothing new. The Babelfish ambition from the start still stands, but delivered into existing devices rather than a device of its own.

The evidence for that bet is small and behavioural. A perfectly clean voice agent did not sound human. Adding ums made people react that it was so human. A voice agent on the company website pulls in far more detail about customer problems than a drop-down form does, because people speak more freely than they type. The same logic runs inside the company. Most managers have close to 10 direct reports, with about 5 layers at most, no titles and open internal documents. AI summaries are meant to carry signal up without a manager in between, shifting leadership from reactive to proactive. Engineers sit inside talent, ops and legal teams so AI adoption spreads bottom-up rather than by mandate.

The human case is the most vivid. Over 10,000 people have had voices restored after ALS, cancer and other causes. A musician is touring with his old band using an AI voice. A congresswoman addressed Congress with one. Tim Green, a former NFL player and author with ALS, hosts a podcast in an ElevenLabs voice and won an Emmy this year. Hundreds to thousands wrote to ask how to get theirs back. Staniszewski offers this as the reason voice matters: it carries recognition and emotion that text cannot. He gives one sentence to safeguards and moves on.

Now the obvious objection. Everything above assumes product depth outlasts model quality, and nothing here demonstrates it. Twenty thousand voices is a count, not retention or share. The research lead is itself hard to verify, since text-to-speech benchmarks are extremely hard to run when every model ships different voices. The forward-deployed approach is expensive, and Staniszewski names its failure mode himself: if lessons do not flow back, it is services. He also expects only about three platforms to carry these interactions and does not claim a place for his. And he declines to widen the lens. Asked about Alex Karp's warning that lab technology could be nationalised, he gives no direct answer and turns to European sovereignty.

Then there is the question of selling. He has had 3 or 4 concrete acquisition offers, the last in June last year, and turned them all down. The host argues that at 31 he would be selling probably the best idea he will ever have, with decades ahead. Staniszewski agrees it is likely his best idea and best timing. His reasoning is brief: "AI is changing the world. We can build at the frontier of that change." He also admits he does not know what the world looks like in 5 to 10 years, listing universal high income and societal adoption among the open questions. The wager is that the window to build is now, and that trust, supply and deployment depth can be assembled before the research gap closes. Whether they compound faster than rivals' models catch up is the number that decides it, and nobody in the conversation has it.