vits japros webui

I will be focusing on developing Style-Bert-VITS2 so I will not update this anymore: https://github.com/litagin02/Style-Bert-VITS2

Bert-VITS2 reference article: https://zenn.dev/litagin/articles/b1ddc1da5ea2b3

VITS-JaPros-WebUI

This is a WebUI for Windows that allows you to learn Japanese VITS models and allows you to synthesize speech with accents. If you only have a speech synthesis, you can use it even without a graphics card.

? Speech synthesis demo

Speech synthesis	study

JaPros?

• There is a framework called ESPnet that allows for a unified handling of various voice processing tasks in machine learning.
• VITS can be used as a way to learn TTS on ESPnet.
• When learning TTS, ESPnet allows you to specify a method ( g2p ) for converting learning text (Japanese sentences) to phoneme sequences (g2p), and one of them is pyopenjtalk_prosody , which has an accent symbol added.

In this situation, I'm taking a model trained with g2p in Japanese using pyopenjtalk_prosody and reading it for convenience (a proposal from Bing-chan).

pyopenjtalk_prosody also handles symbols such as accents, so you can use them to control accents (ハ➚シハ➘シ).

Accent symbol details

symbol	role	example
[	The accent rises from here (image of ➚)	Hello →コ[ンニチワ
]	The accent falls from here (image of ➘)	Kyoto →キョ]オト
(Half-width space)	The cut in accent poem (somehow a single piece of cake)	ソ[レワ ム[ズカシ]イ
、	Pose (taking a breath). Use it when you want to make a short pose.	ハ]イ、ソ[オ オ[モイマ]ス
?	I'll add it to the end of the question.	キ[ミワ ダ]レ?

what is this?

This is something that allows you to train, load and speech synthesis of VITS JaPros models in a local Windows environment.

Learning

• With automatic transcription from audio files using faster-whisper
• The learning itself is modified to run on Windows, allowing VITS JaPros to be learned with minimal operations.

About speech synthesis

• A somewhat intuitive accent control (probably) with katakana and symbols
• Simple speech speed, pitch and intonation adjustment function (from pyworld)
• It also works on the CPU (can be started up separately during learning and checked)
• Even if it is not a model created using this, if it is a model that is pyopenjtalk_prosody in VITS with ESPnet, it should work if it is included with config.yaml

How to use

install

I have confirmed it works on RTX 4070 on Windows 11 with Python 3.10.

1. First, clone this repository.

git clone https://github.com/litagin02/vits-japros-webui.git

2. Double-click setup.bat inside and wait a moment. When Setup complete. appears, you are done.

How to use

• Learning: Double click webui_train.bat
• Speech Synthesis: View below to place the pth file and then double-click webui_infer.bat
• Update: Double click update.bat

For more information and if you don't need a WebUI, please click here.

Place a model for speech synthesis

For models, create a subdirectory in the weights directory and place the {数字}epoch.pth file inside. If you are using an external model (only compatible with models created with pyopenjtalk_prosody in VITS with ESPnet), please also include config.yaml when studying.

 weights
├── model1
│    └── 100epoch.pth
|── model2
│    ├── 50epoch.pth
│    └── config.yaml
...

credit

• ESPnet: This repository uses the original ESPnet Python module to run on Windows (there are only modifications to os.uname and symbolic link creation locations).