Unduh Simple Lora - Unduh Kode Sumber Simple Lora

Lora sederhana

Ini adalah pembeda dari WebUI Automatic1111, yang lebih ramah bagi pengembang, atau pelatihan idola virtual.

Kabar baik

Sisi Google Colab beroperasi:

pameran

Tunjukkan Efek Lora dari Pelatihan dengan sejumlah kecil Dilraba Dilraba, Reba Eropa-Ras Mixed-Race

lingkungan

 pip install -r requirements.txt
git lfs install

Model pra-terlatih

 # blip 模型
wget https : // storage . googleapis . com / sfr - vision - language - research / BLIP / models / model_base_caption_capfilt_large . pth - P . / pretrained_models

# bert-base-uncased
cd pretrained_models
git clone https : // huggingface . co / bert - base - uncased

# diffusion base model
# 我选用的是chilloutmix_NiPrunedFp32Fix
git clone https : // huggingface . co / naonovn / chilloutmix_NiPrunedFp32Fix
# safetenosor模型转换
cd ..
python process / convert_original_stable_diffusion_to_difdusers . py 
    - - checkpoint_path . / pretrained_models / chilloutmix_NiPrunedFp32Fix / chilloutmix_NiPrunedFp32Fix . safetensors 
    - - dump_path . / pretrained_models / chilloutmixNiPruned_Tw1O - - from_safetensors

Persiapan data

Data Huggingface [Opsi]
Data pokemon sebagai contoh

 # 下载数据
mkdir -p dataset
cd dataset
git clone https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions/

Data Pengguna [Opsi]
Pelatihan Lora untuk gambar tunggal

 # 图片文本获取
python process / run_caption . py - - img_base . / dataset / custom

# 将a woman 替换成<dlrb>
python process / change_txt . py - - img_base . / dataset / custom - - ori_txt 'a woman' - - new_txt "<dlrb>"

kereta

Parameter penyesuaian self.custom = benar untuk menggunakan data pengguna benar, false menggunakan data huggingfaec

 - - train_text_encoder # 开启text_encoder lora训练
- - dist # 关闭DDP多机多卡训练模式
- - batch_size 1 # 设置batch_size大小

# 训练脚本
python  train . py  - - batch_size 1 - - dist - - train_text_encoder

pemikiran

 python inference . py 
    - - mode 'lora' 
    - - lora_path checkpoint / Lora / 000 - 00000600. pth 
    - - prompt  "<dlrb>,solo, long hair, black hair, choker, breasts, earrings, blue eyes, jewelry, lipstick, makeup, dark, bare shoulders, mountain, night, upper body, dress, large breasts, ((masterpiece))" 
    - - outpath results / 1. png 
    - - num_images_per_prompt 2

Semakin sedikit gambar pelatihan, semakin kecil jumlah iterasi dari model yang dipilih seharusnya. Misalnya, jika Anda memilih sekitar 1000 untuk satu pelatihan gambar, dan jika Anda memilih sekitar 2500 untuk pelatihan 10 gambar, Anda dapat memilih sekitar 10 pelatihan gambar.

controlNet

Menambahkan konversi controlnet, lihat di sini

Unduh model asli v1-5-pruned.ckpt, control_sd15_openpose.pth ke pretrained_models
Konversikan model dasar Anda sendiri ke Formulir ControlNet

 python process/tool_transfer_control.py 
--path_input pretrained_models/chilloutmix_NiPrunedFp32Fix/chilloutmix_NiPrunedFp32Fix.safetensors 
--path_output pretrained_models/chilloutmix_control.pth

Controlnet ke bentuk diffuser

 python process / convert_controlnet_to_diffusers . py 
- - checkpoint_path  pretrained_models / chilloutmix_control . pth 
- - original_config_file model / third / cldm_v15 . yaml 
- - dump_path  pretrained_models / chilloutmix_control - - device cuda

Unduh OpenPose Model Body_Pose_Model.pth, hand_pose_model.pth ke pretrained_models/openpose
pemikiran

 python inference . py 
    - - mode 'control' 
    - - lora_path checkpoint / Lora / 000 - 00000600. pth 
    - - control_path pretrained_models / chilloutmix_control 
    - - pose_img assets / pose . png 
    - - prompt  "<dlrb>,solo, long hair, black hair, choker, breasts, earrings, blue eyes, jewelry, lipstick, makeup, dark, bare shoulders, mountain, night, upper body, dress, large breasts, ((masterpiece))" 
    - - outpath results / 1. png 
    - - num_images_per_prompt 2

Inpaiting

Unduh modelnya

 cd pretrained_models
git clone https : // huggingface . co / runwayml / stable - diffusion - inpainting
# 下载parsing模型
wget https : // github . com / LeslieZhoa / LVT / releases / download / v0 . 0 / face_parsing . pt - P pretrained_models

pemikiran

 python inference . py 
    - - mode 'inpait' 
    - - inpait_path pretrained_models / stable - diffusion - inpainting 
    - - mask_area all 
    - - ref_img assets / ref . png 
    - - prompt  "green hair,short hair,curly hair, green hair,beach,seaside" 
    - - outpath results / 1. png 
    - - num_images_per_prompt 2

Adaptor T2I

Inpaiting lebih halus

Unduh model adaptor

 wget https : // huggingface . co / TencentARC / T2I - Adapter / resolve / main / models / t2iadapter_seg_sd14v1 . pth - P pretrained_models

pemikiran

 python inference . py 
    - - mode 't2iinpait' 
    - - ref_img assets / t2i - input . png 
    - - mask assets / t2i - mask . png 
    - - adapter_mask assets / t2i - adapter . png 
    - - prompt  "green hair,curly hair, green hair,beach,seaside" 
    - - outpath results / 1. png 
    - - num_images_per_prompt 2

Gaya Insruct-Pix2pix

Download model

 cd pretrained_models
git clone https : // huggingface . co / timbrooks / instruct - pix2pix

pemikiran

 python inference . py 
    - - mode 'instruct' 
    - - ref_img assets / t2i - input . png 
    - - prompt  "turn her face to comic style" 
    - - neg_prompt None 
    - - image_guidance_scale 1 
    - - outpath results / 1. png 
    - - num_images_per_prompt 1

Foto masih bergerak

Model ini terutama berasal dari facevid2vid, yang menambahkan definisi 512 HD

 wget https://github.com/LeslieZhoa/Simple-Lora/releases/download/v0.0/script.zip
unzip script.zip && rm -rf script.zip 
python script/run.py  --input assets/6.png
ffmpeg  -r 25 -f image2 -i results/%06d.png  -vcodec libx264   11.mp4

11.mp4

merujuk ke

https://github.com/huggingface/diffusers
https://github.com/automatic1111/stable-diffusion-webui
https://github.com/salesforce/blip
https://github.com/haofanwang/lora-for-diffusers
https://github.com/lllyasviel/controlnet
https://github.com/haofanwang/controlnet-for-diffusers
https://github.com/haofanwang/t2i- adapter-for-diffusers
https://github.com/tencentarc/t2i- adapter
https://github.com/himario/diffusers-t2i-adapter
https://github.com/zhanglonghao1992/one-shot_free-view_neural_talking_head_synthesis