Exactly one year ago, ElevenLabs, a voice AI startup, entered the European generative AI scene by launching its text-to-speech technology and announcing its tiny pre-seed round. Since then, it’s been on a wild ride.
Just a few months later, the London and New York-headquartered business landed a $19m Series A and today it’s announcing another $80m funding in a Series B co-led by Andreessen Horowitz, Nat Friedman and Daniel Gross, and joined by Sequoia Capital, among others.
The company, founded by two Polish engineers and childhood friends, is now valued at $1.1bn (a nice hat tip to the company’s name), becoming a unicorn less than two years after launching.
Mati Staniszewski, the startup’s cofounder and CEO, tells Sifted that the speed of the company’s growth has been “much greater than expected”. But his focus remains crystal clear: to continue the internal research on audio AI to be able to stay ahead of the competition in terms of product quality.
“We’re just getting started,” he says.
What does ElevenLabs do?
A year ago, the company started off by launching technology that produces artificial voices and clones existing ones. The AI-powered tool can convert text to speech in these synthetic voices, featuring lifelike emotion and intonation, allowing clients like publishers and content creators to turn written material into audio.
But Staniszewski’s first aim has always been to use AI for dubbing — he grew up in Poland where foreign films are often dubbed with one emotionless voice — and that’s what the company diverted its efforts into last year.
Now its tech also allows for speech-to-speech conversion and dubbing (currently available in 29 languages). Last year the company widened the number of languages it operates in and expanded into multiple sectors, including entertainment, gaming, media and conversational chatbots.
As generative AI boomed through 2023, there are many other startups and big tech companies that have started to generate AI voices. But Staniszewski says that there’s one thing that makes ElevenLabs stand out: its focus.
“One thing which is special for us and that we want to keep is that we focus on audio AI… we do our own internal research, our own technology and we build products around it. Most of the companies on the market do more of one or more of another,” he says.
“This is our advantage over startups, which usually don’t do their research, and over big companies, which do their research, but audio is priority number seven or 20 for them.”
He says that the main competitor for ElevenLabs today is OpenAI — mostly because of the access to talent and the team of engineers it has built.
“They can build great research and potentially other companies can build new solutions on their tech,” Staniszewski says. “Our big advantage is that we’re very focused.”
Next steps
That’s why ElevenLabs plans to spend a vast part of its fresh money on research and building the team.
“We already have the world’s best audio engineering team, but we want to grow it,” Staniszewski says, adding that the company also plans to build specific products for some of the verticals it services, such as publishing and filmmaking, and strengthen its sales team.
Staniszewski hopes that the investment into research will quickly yield useful new features, giving the example of allowing users to control artificial voices’ emotions, which he says could happen by the end of the year.
“Now technology can create audio that can interpret what sort of emotions it should use — you provide it with text or audio [and] the model will interpret how it should be read: with a happy emotion, with pauses,” Staniszewski says.
“What we will see this year or in the upcoming years, is how to control the model. The model will predict that something should be read in a happy tone, but if you want a sad tone, we imagine that you’ll be able to describe how you want audio to be read.”
The other thing the company is working on, in the slightly longer term, is real-time audio translation into all existing languages, delivered with the same voices and emotions — this, Staniszewski predicts, might be available in the next three to five years.
Focusing on safety
Staniszewski appreciates the risks posed by technology like the one he’s working on — both in terms of disruption of traditional job markets and when it comes to creating dangerous and fraudulent online content.
That’s why, he says, a part of ElevenLab’s efforts and money will be directed at ensuring the safety of its technology. Staniszewski says that the technology allows for full traceability — it’s easy to track down who has created a given piece of content — and accountability: only people with verified, paid accounts will be able to do high-quality voice cloning. The company is also launching a tool that will allow anyone on the internet to check if a voice has been created by ElevenLabs technology (and soon also by other models).
Staniszewski also says that generative AI’s impact on the job market is not only negative: it also allows the creation of new use cases for voice usage. For example, he says, now that press articles can be converted to audio, ElevenLabs wants the voice acting industry to take advantage of the new opportunities — that’s why it’s also created a voice marketplace.
“We’ll allow people who have great voices, such as voice actors, to be able to create their voice and monetise it,” he says. “They’ll have control over it, earn on it and make it available for those new use cases.”
Who has invested?
- Andreessen Horowitz, a US VC
- Nat Friedman, an entrepreneur and investor, ex-CEO of GitHub
- Daniel Gross, a startup founder and investor, a former machine learning director at Apple
- Sequoia Capital, a US VC
- Smash Capital, a US VC
- SV Angel, a US VC
- BroadLight Capital, a US VC
- Credo Ventures, a Czech VC
