SDFT
1.0.0
SDFT是一個自我教育項目，旨在概述主要的穩定擴散微調技術。穩定的擴散實現來自擁抱面擴散器庫。
概述的技術：
所有微調技術均在一個名為“ Dark Fantasy”的手工製造的玩具數據集上進行。使用穩定的擴散XL Base-1.0模型從SteStieAi中收集數據集，以使人聯想到1970年代和1980年代的樣式，從而生成類似深色幻想的圖像。目的是說明如何在此數據集中概述所有技術。
數據集可以在datasets/目錄下找到。
用洛拉微調SDXL：
accelerate launch train_lora_sdxl.py
--pretrained_model_name_or_path=stabilityai/stable-diffusion-xl-base-1.0
--pretrained_vae_model_name_or_path=madebyollin/sdxl-vae-fp16-fix
--allow_tf32
--mixed_precision= " fp16 "
--rank=32
--train_data_dir=datasets/dark_fantasy/
--caption_column= " text "
--dataloader_num_workers=16
--resolution=512
--use_center_crop
--use_random_flip
--train_batch_size=2
--gradient_accumulation_steps=4 --gradient_checkpointing
--max_train_steps=1500
--learning_rate=1e-04
--max_grad_norm=5
--lr_scheduler= " cosine_with_restarts "
--lr_warmup_steps=100
--output_dir=runs/lora_run/
--checkpointing_steps=100
--validation_epochs=10
--num_validation_images=4
--save_images_on_disk
--validation_prompt= " A picture of a misterious figure in cape, back view. "
--logging_dir= " logs "
--seed=1337使用LORA檢查點進行推斷：
accelerate launch run_lora_inference.py
--pretrained_model_name_or_path=stabilityai/stable-diffusion-xl-base-1.0
--pretrained_vae_model_name_or_path=madebyollin/sdxl-vae-fp16-fix
--output_dir=runs/lora_v1/
--lora_checkpoint_path=runs/lora_run/checkpoint-100/
--resolution=1024
--num_images_to_generate=5
--guidance_scale=5.0
--num_inference_steps=40
--prompt= " A picture of a misterious figure in cape, back view. "
--negative_prompt= " logo, watermark, text, blurry "
--seed=1337無洛拉 - 洛拉圖像比較。使用同一潛伏期生成成對的圖像。
"A picture of a heavy red Kenworth truck riding in the night across the abanoned city streets."
"A picture of a wounded orc warrior, climbing in misty mountains, front view, exhausted face, looking at the camera."
"A picture of space rocket launching, Earth on the background, candid photo."
"A picture of a supermassive black hole, devouring the galaxy, cinematic picture"
"A picture of a human woman warrior, black hair, looking at the camera, front view."
用文本反演（Ti）微調SDXL：
accelerate launch train_ti_sdxl.py
--pretrained_model_name_or_path=stabilityai/stable-diffusion-xl-base-1.0
--pretrained_vae_model_name_or_path=madebyollin/sdxl-vae-fp16-fix
--allow_tf32
--mixed_precision= " fp16 "
--train_data_dir=datasets/skull
--learnable_property= " style "
--placeholder_token= " <skull_lamp> "
--initializer_token= " skull "
--num_vectors=8
--resolution=1024
--repeats=1
--train_batch_size=2
--gradient_accumulation_steps=4 --gradient_checkpointing
--max_train_steps=5000
--learning_rate=3e-3
--lr_scheduler= " piecewise_constant "
--lr_warmup_steps=30
--output_dir= " runs/ti_run "
--validation_prompt= " A painting of Eiffel tower in the style of <skull_lamp> "
--num_validation_images=4
--validation_steps=100
--embeddings_save_steps=500
--save_images_on_disk
--use_random_flip
--use_center_crop
--seed=1337 使用訓練有素的TI嵌入進行推斷：
accelerate launch run_ti_inference.py
--pretrained_model_name_or_path=stabilityai/stable-diffusion-xl-base-1.0
--pretrained_vae_model_name_or_path=madebyollin/sdxl-vae-fp16-fix
--output_dir=runs/ti_run
--path_to_embeddings=runs/ti_run/ti-embeddings-final.safetensors
--resolution=1024
--num_images_to_generate=1
--guidance_scale=5.0
--num_inference_steps=50
--placeholder_token= " <skull_lamp> "
--prompt= " A <skull_lamp>, made of lego "
--negative_prompt= " logo, watermark, text, blurry, bad quality "
--seed=1337沒有Ti -Ti圖像比較。使用同一潛伏期生成成對的圖像。
注意：由於培訓數據集僅包含5個帶有默認字幕的圖像，因此結果並不令人鼓舞，但是引入更多的細粒字幕會使它變得更好。
"A <skull_lamp>, made of lego."
"A painting of Eiffel tower in the style of <skull_lamp>."
"A painting of the great pyramids in the style of <skull_lamp>."
"An oil painting of a skyscraper in the style of <skull_lamp>."
"The painting of a mug in the style of <skull_lamp>."
