ElevenLabs is a startup company focusing on AI voice cloning and text-to-speech API. Recently, it announced the launch of a new feature that allows users to build complete conversational AI agents.

Users can now customize various variables of conversation agents according to their own needs on ElevenLabs' developer platform, such as voice tone and reply length.

ElevenLabs mainly provided different voices and text-to-speech services in the past. Sam Sklar, the company's growth director, said in an interview with TechCrunch that many customers have already been using the platform to create conversational AI agents. But integrating knowledge bases and handling customer interruptions are the biggest challenges. Therefore, ElevenLabs decided to build a complete conversation bot pipeline to simplify this process.

Users can start building conversation agents by logging into their ElevenLabs accounts and choosing templates or creating new projects. They can select the main language of the agent, the first message, and system prompts to determine the personality of the agent.

In addition, developers also need to choose large language models (such as Gemini, GPT, or Claude), the temperature of responses (which determines creativity), and token usage limits.

Users can also add knowledge bases according to their own needs, such as files, URLs, or text blocks, to enhance the capabilities of conversation bots. At the same time, they can integrate their own custom large language models with this bot. ElevenLabs' SDK is compatible with Python, JavaScript, React, and Swift. In addition, the company also provides a WebSocket API for further customization.

The company also allows users to define data collection standards, such as the name and email of customers who have conversations with the agent, and use natural language to define evaluation criteria for the success of calls.

ElevenLabs is leveraging its existing text-to-speech pipeline and is also developing speech-to-text functions for its new conversational artificial intelligence product. Currently, the company does not provide a speech-to-text API separately, but it may be launched in the future, thus competing with speech-to-text APIs of companies such as Google, Microsoft, and Amazon, as well as professional APIs such as OpenAI's Whisper, AssemblyAI, Deepgram, Speechmatics, and Gladia.

The company plans to raise a new round of funds at a valuation of more than $3 billion and is competing with other voice AI startups such as Vapi and Retell, which are also building conversation agents. More importantly, ElevenLabs will compete with OpenAI's real-time conversation API. However, ElevenLabs believes that its customization capabilities and flexibility in switching models will give it an advantage in the competition.