稳定的扩散 - 视频
在Colab中自己尝试：
示例- “蓝莓意大利面”和“草莓意大利面”之间的变形
berry_good_spaghetti.2.mp4
安装
pip install stable_diffusion_videos用法
查看示例文件夹，例如脚本?
制作视频
注意：对于Apple M1体系结构，请改用Torch.float32，因为MPS上没有Torch.float16。
from stable_diffusion_videos import StableDiffusionWalkPipeline
import torch
pipeline = StableDiffusionWalkPipeline . from_pretrained (
"CompVis/stable-diffusion-v1-4" ,
torch_dtype = torch . float16 ,
). to ( "cuda" )
video_path = pipeline . walk (
prompts = [ 'a cat' , 'a dog' ],
seeds = [ 42 , 1337 ],
num_interpolation_steps = 3 ,
height = 512 , # use multiples of 64 if > 512. Multiples of 8 if < 512.
width = 512 , # use multiples of 64 if > 512. Multiples of 8 if < 512.
output_dir = 'dreams' , # Where images/videos will be saved
name = 'animals_test' , # Subdirectory of output_dir where images/videos will be saved
guidance_scale = 8.5 , # Higher adheres to prompt more, lower lets model take the wheel
num_inference_steps = 50 , # Number of diffusion steps per image generated. 50 is good default
)制作音乐视频
新的！可以通过提供音频文件的路径来添加音乐。音频将告知插值率，因此视频转移到节拍?
from stable_diffusion_videos import StableDiffusionWalkPipeline
import torch
pipeline = StableDiffusionWalkPipeline . from_pretrained (
"CompVis/stable-diffusion-v1-4" ,
torch_dtype = torch . float16 ,
). to ( "cuda" )
# Seconds in the song.
audio_offsets = [ 146 , 148 ] # [Start, end]
fps = 30 # Use lower values for testing (5 or 10), higher values for better quality (30 or 60)
# Convert seconds to frames
num_interpolation_steps = [( b - a ) * fps for a , b in zip ( audio_offsets , audio_offsets [ 1 :])]
video_path = pipeline . walk (
prompts = [ 'a cat' , 'a dog' ],
seeds = [ 42 , 1337 ],
num_interpolation_steps = num_interpolation_steps ,
audio_filepath = 'audio.mp3' ,
audio_start_sec = audio_offsets [ 0 ],
fps = fps ,
height = 512 , # use multiples of 64 if > 512. Multiples of 8 if < 512.
width = 512 , # use multiples of 64 if > 512. Multiples of 8 if < 512.
output_dir = 'dreams' , # Where images/videos will be saved
guidance_scale = 7.5 , # Higher adheres to prompt more, lower lets model take the wheel
num_inference_steps = 50 , # Number of diffusion steps per image generated. 50 is good default
)使用UI
from stable_diffusion_videos import StableDiffusionWalkPipeline , Interface
import torch
pipeline = StableDiffusionWalkPipeline . from_pretrained (
"CompVis/stable-diffusion-v1-4" ,
torch_dtype = torch . float16 ,
). to ( "cuda" )
interface = Interface ( pipeline )
interface . launch ()学分
这项工作是由@karpathy共享的脚本构建的。将脚本修改为此要旨，然后将其更新/修改为此回购。
贡献
您可以在此处提交任何问题/功能请求
享受?
下载源码
通过命令行克隆项目:
git clone https://github.com/nateraw/stable-diffusion-videos.git