从扩散模型中擦除概念的低级适应性。
原始存储库:从扩散模型中删除概念
和项目页面:https://erasing.baulab.info/
(不仅用于擦除概念,而且还通过设计提示和洛拉的重量来强调或交换它们。有关更多详细信息,请参见ConceptMod)
conda create -n leco python=3.10
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118
pip install xformers
pip install -r requirements.txt您至少需要8GB VRAM。
python ./train_lora.py --config_file " ./examples/config.yaml " config.yaml :
prompts_file : " ./prompts.yaml "
pretrained_model :
name_or_path : " stabilityai/stable-diffusion-2-1 " # you can also use .ckpt or .safetensors models
v2 : true # true if model is v2.x
v_pred : true # true if model uses v-prediction
network :
type : " lierla " # or "c3lier"
rank : 4
alpha : 1.0
train :
precision : " bfloat16 "
noise_scheduler : " ddim " # or "ddpm", "lms", "euler_a"
iterations : 500
lr : 1e-4
optimizer : " AdamW "
lr_scheduler : " constant "
save :
name : " van_gogh "
path : " ./output "
per_steps : 200
precision : " bfloat16 "
logging :
use_wandb : false
verbose : false
other :
use_xformers : true prompts.yaml :yaml:
- target : " van gogh " # what word for erasing the positive concept from
positive : " van gogh " # concept to erase
unconditional : " " # word to take the difference from the positive concept
neutral : " " # starting point for conditioning the target
action : " erase " # erase or enhance
guidance_scale : 1.0
resolution : 512
dynamic_resolution : false
batch_size : 2有关更多详细信息,请参见示例配置。
注意:您可以使用float16,但不稳定且不建议使用。请使用Bfloat16或Float32。
您可以在Automatic1111的WebUI上使用预告片的权重。
? huggingface:https://huggingface.co/p1atdev/leco
oil painting of van gogh by himself结果:

oil painting of van gogh by himself
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 3870472781, Size: 512x512, Model hash: cc6cb27103, Model: v1-5-pruned-emaonly, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: van_gogh_4_last(db68853d039b), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0 painting of scenery by monet结果:

painting of scenery by monet
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 1284787312, Size: 512x512, Model hash: cc6cb27103, Model: v1-5-pruned-emaonly, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: van_gogh_4_last(db68853d039b), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0 mona lisa with jewelry的结果:

mona lisa with jewelry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 3630495347, Size: 512x512, Model hash: 832eb50c0c, Model: v2-1_768-ema-pruned, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: mona_lisa2_last(393beb35c4b1), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0 photo of a cute cat的结果:

photo of a cute cat
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 900866192, Size: 512x512, Model hash: 832eb50c0c, Model: v2-1_768-ema-pruned, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: mona_lisa2_last(393beb35c4b1), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0使用1.0〜3.0重量时,猫耳将被强行连接。
如果-1.0〜 -3.0,猫耳将永远不会出现。
培训设置:请参阅配置。

masterpiece, best quality, exceptional, best aesthetic, anime, 1girl, school uniform, upper body, smile
Negative prompt : worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 4103955758, Size: 512x512, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "0, 2, 3, 4", Version: v1.3.0培训设置:请参阅配置。
用“现实生活,Instagram”:

real life, instagram, masterpiece, best quality, exceptional, best aesthetic, 1girl, cat ears, blue hair, school uniform, upper body
Negative prompt : worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 757542759, Size: 768x768, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: unreal_6_many_prompts_200steps(fff5917285da), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0没有“现实生活,Instagram”:

masterpiece, best quality, exceptional, best aesthetic,, 1girl, aqua eyes, baseball cap, blonde hair, closed mouth, earrings, green background, hat, hoop earrings, jewelry, looking at viewer, shirt, short hair, simple background, solo, upper body, yellow shirt,
Negative prompt : worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
Steps : 20, Sampler: Euler a, CFG scale: 7, Seed: 2867636749, Size: 768x768, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: unreal_6_many_prompts_200steps(fff5917285da), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: "-1, 0, 1", Version: v1.3.0 我深深地启发了我的工作,我的工作依赖于以下项目的杰出努力。我想对这些项目及其开发人员表示深切的感谢:
https://github.com/rohitgandikota/erasing:从扩散模型中删除概念
https://github.com/cloneofsimo/lora:快速的文本对图像扩散微调的低范围改编
https://github.com/kohya-ss/sd-scripts:培训,发电和实用脚本稳定扩散
https://github.com/ntc-ai/conceptmod:使用DSL从扩散模型中修改概念