Xiaohongshu Open Source Fully Continuous Autoregressive TTS Model | _2024111120230_ | Supports Zero-Shot Audio Cloning

According to Beating's monitoring, Xiaohongshu's hi lab has open-sourced its 2 billion-parameter end-to-end autoregressive text-to-speech (TTS) model dots.tts, and released the complete inference and fine-tuning code under the Apache 2.0 license. The publicly released weights include a base pre-trained version, a self-correcting alignment (SCA) fine-tuned version, and a low-latency inference distillation version. Unlike traditional TTS architectures that rely on discrete audio codec tokens (such as VALL-E, CosyVoice, ChatTTS, etc.), dots.tts implements a fully continuous, end-to-end autoregressive stream matching architecture, completely eliminating the use of any discrete tokens throughout the pipeline. dots.tts Continuous features extracted from AudioVAE at a sampling rate of 48 kHz are combined with a semantic encoder, a backbone language model (initialized from Qwen2.5-1.5B-Base, directly processing BPE text without requiring Pinyin input), and an autoregressive stream-matching acoustic head to predict continuous latent variables, which are then reconstructed into audio by a generator. Because continuous features are directly predicted, dots.tts the audio quality loss caused by discrete quantization is avoided, preserving pronunciation details, timbre similarity, and emotional expressiveness. dots.tts Pre-trained on approximately 1.5 million hours of speech data. In the Seed-TTS-Eval benchmark, dots.tts achieved word error rates (WER) of 0.94% / 1.30% / 6.60% on Chinese, English, and Chinese difficult test sets, respectively, and similarity scores (SIM) of 81.0 / 77.1 / 79.5, both reaching the open-source state-of-the-art (SOTA) level. In the MiniMax Multilingual benchmark test across 24 languages, the average speaker similarity reached 83.9. Xiaohongshu has provided a Gradio experience space on Hugging Face for users to test zero-shot voice cloning online.