ByteDance

/

Hyper-SD

like 1.34k

Follow

ByteDance 4.56k

Why we have to fuse the LoRA, can we just load lora weights ? I tried but find that it dose not work only loading weights.

pipe.unload_lora_weights()
pipe.load_lora_weights(ckpt_path, cross_attention_kwargs={"scale": 0.125})
# pipe.load_lora_weights(ckpt_path)
seed = 1
image1 = pipe(
    prompt="a photo of a cat",
    num_inference_steps=20,
    guidance_scale=3.5,
    generator=torch.Generator(device='cpu').manual_seed(seed)
).images[0]

pipe.fuse_lora(lora_scale=0.125)
image2 = pipe(
    prompt="a photo of a cat",
    num_inference_steps=20,
    guidance_scale=3.5,
    generator=torch.Generator(device='cpu').manual_seed(seed)
).images[0]

plot_multi_images([image1, image2], ['before fuse', 'after fuse'])