ChatTTS

A generative speech model for daily dialogue.

](https://github.com/2noise/ChatTTS/blob/main/LICENSE) [

](https://huggingface.co/2Noise/ChatTTS) [ [](https://discord.gg/Ud5Jxgx5yD)

English | **简体中文** | **日本語** | **Русский** | **Español** | **Français** | **한국어**

Introduction

[!Note] This repo contains the algorithm infrastructure and some simple examples.

!Tip] For the extended end-user products, please refer to the index repo [Awesome-ChatTTS maintained by the community. You can find a diagram visualization of the codebase here.

ChatTTS is a text-to-speech model designed specifically for dialogue scenarios such as LLM assistant.

Supported Languages

• [x] English
• [x] Chinese
• [ ] Coming Soon...

Highlights

You can refer to [this video on Bilibili](https://www.bilibili.com/video/BV1zn4y1o7iV) for the detailed description.

1. 1.Conversational TTS: ChatTTS is optimized for dialogue-based tasks, enabling natural and expressive speech synthesis. It supports multiple speakers, facilitating interactive conversations.
2. 2.Fine-grained Control: The model could predict and control fine-grained prosodic features, including laughter, pauses, and interjections.
3. 3.Better Prosody: ChatTTS surpasses most of open-source TTS models in terms of prosody. We provide pretrained models to support further research and development.

Dataset & Model

[!Important] The released model is for academic purposes only.

• The main model is trained with Chinese and English audio data of 100,000+ hours.
• The open-source version on [HuggingFace](https://huggingface.co/2Noise/ChatTTS) is a 40,000 hours pre-trained model without SFT.

Roadmap

• [x] Open-source the 40k-hours-base model and spk_stats file.
• [x] Streaming audio generation.
• [x] Open-source DVAE encoder and zero shot inferring code.
• [ ] Multi-emotion controlling.
• [ ] ChatTTS.cpp (new repo in 2noise org is welcomed)

Licenses

#### The Code

The code is published under AGPLv3+ license.

#### The model

The model is published under CC BY-NC 4.0 license. It is intended for educational and research use, and should not be used for any commercial or illegal purposes. The authors do not guarantee the accuracy, completeness, or reliability of the information. The information and data used in this repo, are for academic and research purposes only. The data obtained from publicly available sources, and the authors do not claim any ownership or copyright over the data.

Disclaimer