Facial caricature profile GIF

Face Caricature Profile Image and GIF Animation Creation

Contents

1. Background and purpose
2. Organizer/Team Won
3. Project period
4. Project introduction
   4.1 Project Course
   4.2 Model description
5. INFERENCE
6. Result
7. Presentation

1. Background and purpose

• It may be burdensome to upload your proof on public platforms such as Github profile.
• Create a caricature that reflects the characteristics of the user based on the actual face picture and converts it to a GIF animation.

2. Organizer/Team member

• Organizer/Organizer: Kookmin University AI Big Data Convergence Management Society X: AI

• A total of 5 people [Kwon Min -ji, Kim Seo -ryeong, Kim Jin -ha. Kwang -yeol Yoo (Team Leader), Cho Hyun -sik]

3. Project period

• 2024.07 ~ 2024.08 (2 months)

4. Introduction of project

4.1. Project process

1. Use SAM to create masks.
2. In the generated mask, only the mask of the part you want to apply in the style conversion is entered into the Styletransfer.
3. Diffstyler enters user's face photos, style images and mask to create a caricature that reflects the style.
4. The Image2video model, Liveportrait, enters the caricature image and a driving video to output the final result that converts it to GIF.

4.2. Model used

Sam

Diffstyler

Image2video

5. INFERENCE

1. git clone

 git clone https://github.com/yugwangyeol/2024_XAI_ADV_CV1_Toyproject.git

2. Please download the necessary packages.

 cd 2024_XAI_ADV_CV1_Toyproject
pip install git+https://github.com/facebookresearch/segment-anything.git
pip install gradio

# DiffStyler
cd DiffStyler
pip install -r requirements.txt
pip install "jax[cuda12_pip]==0.4.23" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

# LivePortrait
cd ../LivePortrait
pip install -r requirements.txt
cd ../

3. Run the code below to run Gradio.

 python main.py

• In the Colab environment, you can run Gradio_main.ipynb file and run the code in turn.

6. Result

Original	Mask	Style Image	Diffstyler

Driving Video	LiveProtrait

Gradio

Presentation

X: AI ADV CV1 Team TOY PROJECT Intermediate Announcement.PDF

X: AI ADV CV1 Team Toy Project Final Announcement.PDF

Presentation