LECO

LECO ✏️

Adaptasi rendah untuk menghapus konsep dari model difusi.

Repositori Asli: Menghapus Konsep dari Model Difusi

dan halaman proyek: https://erasing.baulab.info/

(Tidak hanya untuk menghapus konsep, tetapi juga menekankan atau menukarnya dengan merancang petunjuk dan berat Lora. Lihat ConceptMod untuk lebih jelasnya)

Pengaturan

conda create -n leco python=3.10
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118
pip install xformers
pip install -r requirements.txt

Kereta

Anda setidaknya membutuhkan VRAM 8GB.

python ./train_lora.py --config_file " ./examples/config.yaml "

config.yaml :

 prompts_file : " ./prompts.yaml "

pretrained_model :
  name_or_path : " stabilityai/stable-diffusion-2-1 " # you can also use .ckpt or .safetensors models
  v2 : true # true if model is v2.x
  v_pred : true # true if model uses v-prediction

network :
  type : " lierla " # or "c3lier"
  rank : 4
  alpha : 1.0

train :
  precision : " bfloat16 "
  noise_scheduler : " ddim " # or "ddpm", "lms", "euler_a"
  iterations : 500
  lr : 1e-4
  optimizer : " AdamW "
  lr_scheduler : " constant "

save :
  name : " van_gogh "
  path : " ./output "
  per_steps : 200
  precision : " bfloat16 "

logging :
  use_wandb : false
  verbose : false

other :
  use_xformers : true

prompts.yaml :

- target : " van gogh " # what word for erasing the positive concept from
  positive : " van gogh " # concept to erase
  unconditional : " " # word to take the difference from the positive concept
  neutral : " " # starting point for conditioning the target
  action : " erase " # erase or enhance
  guidance_scale : 1.0
  resolution : 512
  dynamic_resolution : false
  batch_size : 2

Lihat Contoh Konfigurasi untuk detail lebih lanjut.

Catatan: Anda dapat menggunakan float16 tetapi tidak stabil dan tidak disarankan. Silakan gunakan bfloat16 atau float32.

Bobot pretrained

Anda dapat menggunakan bobot pretrained pada WebUI Automatic1111.

? HuggingFace: https://huggingface.co/p1atdev/leco

SDV1.5

• Van Gogh Style (dilatih untuk menghapus konsep "gaya van gogh" pada sdv1.5)

Hasil oil painting of van gogh by himself :

Pengaturan Generasi

 oil painting of van gogh by himself
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 3870472781, Size: 512x512, Model hash: cc6cb27103, Model: v1-5-pruned-emaonly, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: van_gogh_4_last(db68853d039b), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0

Hasil painting of scenery by monet :

Pengaturan Generasi

 painting of scenery by monet
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 1284787312, Size: 512x512, Model hash: cc6cb27103, Model: v1-5-pruned-emaonly, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: van_gogh_4_last(db68853d039b), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0

SDV2.1-768

• Mona Lisa (dilatih untuk menghapus konsep "Mona Lisa" di SDV2.1-768)

Hasil mona lisa with jewelry :

Pengaturan Generasi

 mona lisa with jewelry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 3630495347, Size: 512x512, Model hash: 832eb50c0c, Model: v2-1_768-ema-pruned, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: mona_lisa2_last(393beb35c4b1), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0

Hasil photo of a cute cat :

Pengaturan Generasi

 photo of a cute cat
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 900866192, Size: 512x512, Model hash: 832eb50c0c, Model: v2-1_768-ema-pruned, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: mona_lisa2_last(393beb35c4b1), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0

WD1.5 Beta3

• Telinga kucing (dilatih untuk menggantikan "1girl" dengan "1Girl, telinga kucing" di WD1.5 beta3)

Telinga kucing akan terpasang secara paksa saat menggunakan dengan berat 1.0 ~ 3.0.

Jika -1.0 ~ -3.0, telinga kucing tidak akan pernah muncul.

Pengaturan Pelatihan: Lihat Konfigurasi.

Pengaturan Generasi

 masterpiece, best quality, exceptional, best aesthetic, anime, 1girl, school uniform, upper body, smile
Negative prompt : worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 4103955758, Size: 512x512, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "0, 2, 3, 4", Version: v1.3.0

• Unreal (dilatih untuk menghapus "realistis", "kehidupan nyata", "Instagram" di wd1.5 beta3)

Pengaturan Pelatihan: Lihat Konfigurasi.

Dengan "kehidupan nyata, Instagram":

Pengaturan Generasi

 real life, instagram, masterpiece, best quality, exceptional, best aesthetic, 1girl, cat ears, blue hair, school uniform, upper body
Negative prompt : worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 757542759, Size: 768x768, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: unreal_6_many_prompts_200steps(fff5917285da), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0

Tanpa "kehidupan nyata, Instagram":

Pengaturan Generasi

 masterpiece, best quality, exceptional, best aesthetic,, 1girl, aqua eyes, baseball cap, blonde hair, closed mouth, earrings, green background, hat, hoop earrings, jewelry, looking at viewer, shirt, short hair, simple background, solo, upper body, yellow shirt,
Negative prompt : worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 2867636749, Size: 768x768, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: unreal_6_many_prompts_200steps(fff5917285da), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0 

Referensi

Saya sangat terinspirasi oleh dan pekerjaan saya bergantung pada upaya luar biasa dari proyek -proyek berikut. Saya ingin mengucapkan terima kasih yang mendalam kepada proyek -proyek ini dan pengembang mereka:

• https://github.com/rohitgandikota/erasing: Menghapus konsep dari model difusi

  • https://huggingface.co/spaces/baulab/erasing-consepts-in-diffusion: demo ESD

• https://github.com/cloneofsimo/lora: adaptasi rendah untuk fine-tuning difusi teks-ke-image cepat

• https://github.com/kohya-ss/sd-scripts: pelatihan, generasi dan skrip utilitas untuk difusi yang stabil

• https://github.com/ntc-ai/conceptmod: Modifikasi konsep dari model difusi menggunakan DSL