GPT-SoVITSTranslation site

GPT-SoVITS is an open source that combines GPT (Generative Pretraining Model) and SoVITS (Singing Voice Conversion via Variational Information Bottleneck Technology).sound cloningTool that is mainly used for sound (mainly singing) conversion tasks.

Main features

1. High quality conversionGPT-SoVITS can realize natural and smooth sound conversion with the powerful generating ability of GPT, which makes the converted songs more realistic.
2. cross-language support: The tool supports voice reasoning in multiple languages, including English, Japanese, Korean, Cantonese and Chinese, breaking down language barriers and enabling voice cloning technology to cross borders and serve a wider range of people.
3. Zero-sample text-to-speech (TTS): Users only need to provide a 5-second voice sample to immediately experience the text-to-speech conversion feature.
4. Integration aids: GPT-SoVITS integrates auxiliary tools such as voice accompaniment separation, automatic training set segmentation, Chinese automatic speech recognition (ASR), and text annotation, which further enhance the system's functionality and usability, making it easy for even beginners to create training datasets and GPT/SoVITS models.
5. End-to-end training: Models can be trained directly from input to output without complex intermediate processing steps, dramatically reducing the training time of sound models.

application scenario

1. Entertainment: GPT-SoVITS can be used to create celebrity voice packs or mimic the voices of celebrities, providing fans with a richer entertainment experience.
2. Education: This tool can help students practice pronunciation or create audiobooks to improve learning.
3. Customer Service Area: GPT-SoVITS can be used to generate personalized voice responses to enhance the customer experience.

system requirements

1. hardware requirement::

   • GPU: CUDA-enabled NVIDIA graphics card with at least 6GB of video memory, NVIDIA GTX 1660 or higher recommended.
   • CPU: Multi-core CPUs, such as Intel Core i5 or higher, speed up data processing and model inference.
   • Memory: at least 16GB RAM, 32GB recommended for large-scale data and training tasks.
   • Storage space: at least 50GB of available hard disk space, SSDs will significantly increase speed.

2. Software Requirements::

   • Python: Python version 3.8 or 3.9 is recommended.
   • CUDA and cuDNN: If using NVIDIA GPUs, you need to install the appropriate version of CUDA and cuDNN.
   • PyTorch: GPT-SoVITS relies on PyTorch as a deep learning framework.
   • Other dependent libraries: including numpy, scipy, librosa and other audio processing libraries.

Usage

1. Installation of GPT-SoVITS::

   • Users can download the GPT-SoVITS project file from GitHub and extract it to a non-Chinese path.
   • Locate and double-click to run the go-webui.bat file to launch the GPT-SoVITS web interface.

2. Prepare Audio::

   • The user will need to prepare an audio for the cutting exercise, which should be as clear and pure as possible, avoiding background sounds, murmurs, or other characters' voices.
   • Audio length is recommended to be limited to about 1 minute for better training results.

3. audio processing::

   • In the web interface of GPT-SoVITS, users can perform audio processing operations, including vocal separation, denoising, and cutting.
   • These operations help extract high-quality speech samples and lay a solid foundation for subsequent training and inference.

4. Training and Reasoning::

   • After completing the audio processing, the user needs to fill in information such as the model name, the ASR file address and the address of the cut audio folder.
   • Turn on one-button triplet operation (i.e., automate the steps of training set formatting, fine-tuning training, and TTS inference).
   • Waiting for the training process to complete, the generated model can be used for speech synthesis.

5. speech synthesis::

   • After training, the user can select the generated GPT and SoVITS models and upload a good quality slice of audio as a reference sound.
   • Fill in the text content to be synthesized and click the Synthesize Speech button.
   • After a few moments, users can play or download the generated voice files online.