Slow loading of Loras with SDXL / diffuser

[mysticfall]

I'm trying to move my setup to SDXL / diffuser. One thing I've noticed after the migration was how slow Lora loading has become.

I hadn't given much attention to the overhead using multiple Loras incurs when I was on SD 1.5. But now, a single Lora takes about 20 seconds (tested on a Runpod instance with RTX3090/24Gb) every time I generate images, and the overhead seems to be multiplying according to the number of Loras I use.

Is there something wrong with my setup? Or is it just that either the Diffuser backend or Lora extension for SDXL is much slower than their counterparts for SD 1.5?

[vladmandic]

which version of sdnext?
i'm aware of the slow load and we're working on improving the method, that's why right now there are multiple methods available - settings -> diffusers -> lora loading method.

[mysticfall]

I'm using the latest master branch. If it's not something only happening on my end, then I'm fine with waiting a bit before things settle down.

Anyway, I didn't know I there was such an option, so I'll see if it makes any difference. Thanks!

[vladmandic]

sequential apply option is about 10x faster than diffusers default, but we're still ironing out some issues.

[mysticfall]

I'm already using the sequential option. Looks like I'll need to wait a bit before it gets optimised.

[zengjie617789]

how to fix this, I load lora lcm weights cost 27s

[edwardqin-creator]

sequential apply option is about 10x faster than diffusers default, but we're still ironing out some issues.

Could you please tell me more details about the implementation of 'sequential apply'? I'm now studying how to efficiently load LoRA weights to StableDiffusionXLPipeline. Thanks a lot.