EMOTIBOT

Emotibot: Face Tracking and Emotion-Responsive Robot

MEMBER:

WOOSUNG YUN, INGUU CHOI, Minhyuk Lee, DAEWAN KIM, Minho KIM, Kim Kyung -hyun, Kyonoeng Go, Kim Si -on (SION Kim), Lee Hyun -seung (hyunseung Lee)

(Click on the image above to see the demo link.)

Emotibot is a robot with a face of dynamic cells. Estimates the user's face position, allowing you to see your eyes using dynamic cells. In addition, if the user says "high emotici", it has the ability to ask an emotional question, which is input to the GPT API and answers according to the user's emotional status, as the counselor responds directly.

Technology stack

---

Emotibot uses the following technology stack:

• STT (Speech-to-Text): Used to recognize the user's voice input.
• TTS (Text-to-Speech): used for the voice output of the robot.
• GPT-3.5 Turbo: It is used to identify and talk about the user's emotional status.
• OpenCV: Use OpenCV's Cascade Classifier for Face Recognition.
• ROS (Noetic): Used for communication between nodes of robots.
• Dynamixel: used to implement the robot's eye movement.
• Arduino: The robot is used to find out the status of the NEO LED.

function

---

The function of emotibot is as follows.

• Face location estimation function: OPENCV CASCADE CLASSIFIER estimates the user's face position and uses the estimated location to move the robot's eyes using dynamic cells.
• Emotional Status Recognition: Recognize the emotional state through the user's voice input and talk with the user as if talking to the counselor using the GPT-3.5 Turbo.
• Voice output function: TTS API uses users to talk with users or provide a variety of notifications.

Hardware Architecture

---

Workflow & nodegraph

---

Install

---

 cd ~ /catkin_ws/src
git clone https://github.com/ggh-png/EMOTIROBOT.git
git clone https://github.com/ROBOTIS-GIT/DynamixelSDK.git
cd ~ /catkin_ws && catkin_make

pip install pyaudio
pip install google-cloud-speech
pip install google-cloud-texttospeech
pip install playsound
pip install six
pip install openai

Launch

---

roslaunch EMOTI_core EMOTI_core.launch