Lora/Unet issues with latest version of SDnext

[xenlaRZ]

Hi all,
Hoping I've just missed a setting, however the recent update to SDnext which included the balanced offload mode seems to have upset something on my end when it comes to loading Loras. In essence, the first image in any batch size for the same prompt will work perfectly, however every subsequent run will ignore the Lora and be accompanied with the following error message:

ValueError: Adapter name X already in use in the Unet - please select a new adapter name.

Where 'X' is the name of the Lora being requested in the prompt. The full traceback looks something like this:

ERROR    LoRA: ValueError
╭─────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────╮
│ F:\AI\Local\SDnext\automatic\extensions-builtin\Lora\networks.py:98 in load_diffusers                                   │
│                                                                                                                         │
│    97 │   try:                                                                                                          │
│ ❱  98 │   │   shared.sd_model.load_lora_weights(network_on_disk.filename, adapter_name=name)                            │
│    99 │   except Exception as e:                                                                                        │
│                                                                                                                         │
│ F:\AI\Local\SDnext\automatic\venv\lib\site-packages\diffusers\loaders\lora_pipeline.py:562 in load_lora_weights         │
│                                                                                                                         │
│    561 │   │                                                                                                            │
│ ❱  562 │   │   self.load_lora_into_unet(                                                                                │
│    563 │   │   │   state_dict, network_alphas=network_alphas, unet=self.unet, adapter_name=adapter_name, _pipeline=self │
│                                                                                                                         │
│ F:\AI\Local\SDnext\automatic\venv\lib\site-packages\diffusers\loaders\lora_pipeline.py:732 in load_lora_into_unet       │
│                                                                                                                         │
│    731 │   │   │   logger.info(f"Loading {cls.unet_name}.")                                                             │
│ ❱  732 │   │   │   unet.load_attn_procs(                                                                                │
│    733 │   │   │   │   state_dict, network_alphas=network_alphas, adapter_name=adapter_name, _pipeline=_pipeline        │
│                                                                                                                         │
│ F:\AI\Local\SDnext\automatic\venv\lib\site-packages\huggingface_hub\utils\_validators.py:114 in _inner_fn               │
│                                                                                                                         │
│   113 │   │                                                                                                             │
│ ❱ 114 │   │   return fn(*args, **kwargs)                                                                                │
│   115                                                                                                                   │
│                                                                                                                         │
│ F:\AI\Local\SDnext\automatic\venv\lib\site-packages\diffusers\loaders\unet.py:206 in load_attn_procs                    │
│                                                                                                                         │
│   205 │   │   elif is_lora:                                                                                             │
│ ❱ 206 │   │   │   is_model_cpu_offload, is_sequential_cpu_offload = self._process_lora(                                 │
│   207 │   │   │   │   state_dict=state_dict,                                                                            │
│                                                                                                                         │
│ F:\AI\Local\SDnext\automatic\venv\lib\site-packages\diffusers\loaders\unet.py:303 in _process_lora                      │
│                                                                                                                         │
│   302 │   │   │   if adapter_name in getattr(self, "peft_config", {}):                                                  │
│ ❱ 303 │   │   │   │   raise ValueError(                                                                                 │
│   304 │   │   │   │   │   f"Adapter name {adapter_name} already in use in the Unet - please select a new adapter name." │

For context, I am running with zluda on a 6800xt in windows, using SDXL. Has generally worked fine on previous versions. With the last version of SDnext the first run is noticeably faster, very keen to see if this unet/adapter name issue can be resolved. Not willing to call it a bug just yet in case I've made a mistake in the settings! Appreciate any help that's offered.

[YefgenyK]

Ah, that’s quite an interesting scenario you’ve got there. The notion of adapter names within the Unet framework seems to present a rather subtle complication, doesn’t it? It brings to mind how intricate the interactions between components can become when such updates introduce new dynamics. The issue you’re describing, particularly around the repeated use of an adapter name, might not be immediately obvious as a setting problem, yet it also doesn’t quite scream “bug,” does it?

There’s this curious angle where the process, once initiated, might not be fully resetting itself in terms of how it recognizes the Lora adapter. One might consider whether there’s a hidden mechanism at play that retains elements in memory longer than anticipated, which could easily cause such repetition. Now, in a scenario where you allow the system to assign names with a certain degree of variability, there’s a chance the conflict might simply dissolve, though it’s tricky to assert without delving deeper into the specifics.

Now, as for the environment itself—working within zluda on that particular GPU architecture—you've undoubtedly noticed certain nuances, possibly related to how resources are handled batch after batch. Whether that first image speed-up is a clue to the underlying shift post-update is something worth reflecting on.

It might also be worth considering the rhythm with which you approach the problem: instead of rigidly categorizing this as purely an adapter issue, you might start questioning how the broader pipeline shifts under new load-balancing modes. Revisiting the settings from an earlier perspective, before the update, could reveal something that now appears innocuous but plays a larger role than one might expect.

[vladmandic]

if you're on master, I suggest to switch to dev branch as this should already be fixed. or wait for a new release, should be soon.

And if issue persists, please report it using all required info as issue, not a discussion topic.

[xenlaRZ]

Thank you Vlad,
I can confirm, on the dev branch with identical settings there are no issues. Looking forward to the new master release now!

Thanks to yourself and @YefgenyK for the very fast replies as well. It's greatly appreciated.