從擴散模型中擦除概念的低級適應性。
原始存儲庫:從擴散模型中刪除概念
和項目頁面:https://erasing.baulab.info/
(不僅用於擦除概念,而且還通過設計提示和洛拉的重量來強調或交換它們。有關更多詳細信息,請參見ConceptMod)
conda create -n leco python=3.10
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118
pip install xformers
pip install -r requirements.txt您至少需要8GB VRAM。
python ./train_lora.py --config_file " ./examples/config.yaml " config.yaml :
prompts_file : " ./prompts.yaml "
pretrained_model :
name_or_path : " stabilityai/stable-diffusion-2-1 " # you can also use .ckpt or .safetensors models
v2 : true # true if model is v2.x
v_pred : true # true if model uses v-prediction
network :
type : " lierla " # or "c3lier"
rank : 4
alpha : 1.0
train :
precision : " bfloat16 "
noise_scheduler : " ddim " # or "ddpm", "lms", "euler_a"
iterations : 500
lr : 1e-4
optimizer : " AdamW "
lr_scheduler : " constant "
save :
name : " van_gogh "
path : " ./output "
per_steps : 200
precision : " bfloat16 "
logging :
use_wandb : false
verbose : false
other :
use_xformers : true prompts.yaml :yaml:
- target : " van gogh " # what word for erasing the positive concept from
positive : " van gogh " # concept to erase
unconditional : " " # word to take the difference from the positive concept
neutral : " " # starting point for conditioning the target
action : " erase " # erase or enhance
guidance_scale : 1.0
resolution : 512
dynamic_resolution : false
batch_size : 2有關更多詳細信息,請參見示例配置。
注意:您可以使用float16,但不穩定且不建議使用。請使用Bfloat16或Float32。
您可以在Automatic1111的WebUI上使用預告片的權重。
? huggingface:https://huggingface.co/p1atdev/leco
oil painting of van gogh by himself結果:

oil painting of van gogh by himself
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 3870472781, Size: 512x512, Model hash: cc6cb27103, Model: v1-5-pruned-emaonly, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: van_gogh_4_last(db68853d039b), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0 painting of scenery by monet結果:

painting of scenery by monet
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 1284787312, Size: 512x512, Model hash: cc6cb27103, Model: v1-5-pruned-emaonly, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: van_gogh_4_last(db68853d039b), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0 mona lisa with jewelry的結果:

mona lisa with jewelry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 3630495347, Size: 512x512, Model hash: 832eb50c0c, Model: v2-1_768-ema-pruned, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: mona_lisa2_last(393beb35c4b1), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0 photo of a cute cat的結果:

photo of a cute cat
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 900866192, Size: 512x512, Model hash: 832eb50c0c, Model: v2-1_768-ema-pruned, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: mona_lisa2_last(393beb35c4b1), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0使用1.0〜3.0重量時,貓耳將被強行連接。
如果-1.0〜 -3.0,貓耳將永遠不會出現。
培訓設置:請參閱配置。

masterpiece, best quality, exceptional, best aesthetic, anime, 1girl, school uniform, upper body, smile
Negative prompt : worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 4103955758, Size: 512x512, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "0, 2, 3, 4", Version: v1.3.0培訓設置:請參閱配置。
用“現實生活,Instagram”:

real life, instagram, masterpiece, best quality, exceptional, best aesthetic, 1girl, cat ears, blue hair, school uniform, upper body
Negative prompt : worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 757542759, Size: 768x768, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: unreal_6_many_prompts_200steps(fff5917285da), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0沒有“現實生活,Instagram”:

masterpiece, best quality, exceptional, best aesthetic,, 1girl, aqua eyes, baseball cap, blonde hair, closed mouth, earrings, green background, hat, hoop earrings, jewelry, looking at viewer, shirt, short hair, simple background, solo, upper body, yellow shirt,
Negative prompt : worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 2867636749, Size: 768x768, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: unreal_6_many_prompts_200steps(fff5917285da), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0 我深深地啟發了我的工作,我的工作依賴於以下項目的傑出努力。我想對這些項目及其開發人員表示深切的感謝:
https://github.com/rohitgandikota/erasing:從擴散模型中刪除概念
https://github.com/cloneofsimo/lora:快速的文本對圖像擴散微調的低範圍改編
https://github.com/kohya-ss/sd-scripts:培訓,發電和實用腳本穩定擴散
https://github.com/ntc-ai/conceptmod:使用DSL從擴散模型中修改概念