euanka

Fully deploy your own mockup voice conversation role locally

introduce

What you need most at the moment is your STAR, thank you!

What you need most at the moment is your STAR, thank you!

What you need most at the moment is your STAR, thank you!

This is an AI dialogue service system that is about to start. Of course, they are currently under conception and improvement. If someone likes it, they will continue to update...

Function

• Use k2 to achieve streaming speech recognition ASR
• Use Rasa as NLU semantic word segmentation dialogue to speed up the acquisition of external source data, and quickly give real external source data to the big model. Unlike agent Agents such as AutoChain and LangChain, the input model is repeatedly observed and implemented, there is currently: Weather
• Use ChatGLM2-6B CPP version as mockup reasoning and as role reasoning
• Use VITS as TTS to provide paramount voice
• Dialogue protocol distribution service, implemented using Go
• Vue implements simple front-end access
• Raspberry Pi, Snowboy Go and C++ dynamic library realize voice keyword wake-up

Service Architecture

Use euanka-web front-end effects

Click on the picture to open the video

Customized wake-up interaction using Raspberry Pi 4 microphone array

Click on the picture to open the video

environment

The currently built versions are all x86 architectures, i5 8gen 4 cores 32G, which can run slowly

Build

Local build

Build

 go build src/main.go

run

./main

docker build

 docker build -t registry.cn-shenzhen.aliyuncs.com/yuanfangqiao/euanka:0.1 .

Support one-click operation, docker-compose

Go to the docker-compose folder and start with one click

 docker-compose -f euanka-all-compose.yaml up -d

Check the startup status

 docker-compose -f euanka-all-compose.yaml ps

As follows, all startups can be accessed

             Name                           Command               State                    Ports
------------------------------------------------------------------------------------------------------------------
docker-compose_action-server_1   ./entrypoint.sh start --ac ...   Up      0.0.0.0:5055->5055/tcp,:::5055->5055/tcp
docker-compose_asr_1             ./build/bin/sherpa-onnx-on ...   Up      0.0.0.0:6006->6006/tcp,:::6006->6006/tcp
docker-compose_euanka-web_1      /docker-entrypoint.sh /bin ...   Up      0.0.0.0:80->80/tcp,:::80->80/tcp
docker-compose_euanka_1          /bin/demo                        Up      0.0.0.0:8080->8080/tcp,:::8080->8080/tcp
docker-compose_llm_1             python -u websokcet_server ...   Up      0.0.0.0:7600->7600/tcp,:::7600->7600/tcp
docker-compose_rasa_1            rasa run                         Up      0.0.0.0:5005->5005/tcp,:::5005->5005/tcp
docker-compose_tts_1             python3 -u app.py                Up      0.0.0.0:7860->7860/tcp,:::7860->7860/tcp

The euanka-web front-end has been deployed, and you can access the web page directly, such as: http://192.168.1.16. You need to refer to the euanka-web project to solve recording problems. The access effect is as follows: the above web effect.

Depend on projects

All the following project support is required, and all docker builds and runs are implemented.

The following project I redeveloped must be supported, and the operation has been run interactively according to a certain request protocol format.

The project can be used as a learning reference.

• Web conversation Vue implementation

• Raspberry Pi voice terminal Snowboy wake-up GO implementation

• Streaming recognition ASR K2

• Natural Language Processing NLP Rasa

• Big Model LLM chatglm.cpp

• Text to Speech TTS Vits

Acknowledge

Thanks to the support of the following open source projects!

• sherpa-onnx

• rasa

• spacy

• spacy-models

• chatglm.cpp

• VITS-Umamusume-voice-synthesizer