Intel ARC GPU (XPU) Support

[TotalDay]

Thanks for Intel ARC GPU support.
Please add Intel ARC GPu in the instruction to install (launch)
Today only Linux user can run WebUI with SD with XPU
WSL have memory leak trouble.
To Run WEBUI SD with intel GPU (XPU) users must install "base Intel one API" and many other packages for CLI support.

and, please make
Auto-adjust parameters when running on XPU (Intel ARC GPU)

[TotalDay]

1. Intel® software packages for general purpose GPU capabilities are provided for several Linux*-based distributions.
   https://dgpu-docs.intel.com/installation-guides/index.html
   https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-jammy-arc.html

2. Install oneAPI Base Toolkit
   https://www.intel.com/content/www/us/en/docs/oneapi/installation-guide-linux/2023-0/apt.html
   or
   https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html
   Need to install components of Intel® oneAPI Base Toolkit:
   Intel® oneAPI DPC++ Compiler
   Intel® oneAPI Math Kernel Library (oneMKL)
   https://intel.github.io/intel-extension-for-pytorch/xpu/latest/tutorials/installation.html

[vladmandic]

those need to go into docs, i don't install any system components that are not accesible as python package.
same for amd rocm - user needs to install it before app detects rocm is available and installs appropriate version of torch.

but yes, i need to spend some serious time rewriting docs.

[TotalDay]

Thanks
Please, add into the launcher auto-adjust parameters when running on XPU (Intel ARC GPU) like for CPU \ Nvidia

[GreatTVBox]

at Ubuntu Jammy (SSD) all work
Torch
Torchvision
Torch-extension
installed automaticcaly

Instruction - all version (with GPU drivers)

------ 
Ubuntu Jammy
------ 
sudo apt-get install -y gpg-agent wget
wget -qO - https://repositories.intel.com/graphics/intel-graphics.key | sudo gpg --dearmor --output /usr/share/keyrings/intel-graphics.gpg
echo 'deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/graphics/ubuntu jammy max' | sudo tee  /etc/apt/sources.list.d/intel.gpu.jammy.list
sudo apt-get update && sudo apt-get install  -y --install-suggests linux-image-5.15.0-57-generic
sudo sed -i "s/GRUB_DEFAULT=.*/GRUB_DEFAULT=\"1> $(echo $(($(awk -F\' '/menuentry / {print $2}' /boot/grub/grub.cfg | grep -no '5.15.0-57' | sed 's/:/\n/g' | head -n 1)-2)))\"/" /etc/default/grub
sudo sed -i "s/GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT=\"$(echo $(awk -F'="' '$1  == "GRUB_CMDLINE_LINUX_DEFAULT" {print $2}' /etc/default/grub | tr -d '"') | sed 's/pci=realloc=off//g') pci=realloc=off\"/" /etc/default/grub
sudo  update-grub
sudo reboot
-----
Ubuntu Jammy 5.15.0-57-generic
-----
sudo apt-get update
sudo apt-get -y install gawk dkms linux-headers-$(uname -r) libc6-dev 
sudo apt-get install -y intel-platform-vsec-dkms intel-platform-cse-dkms
sudo apt-get install -y intel-i915-dkms intel-fw-gpu
sudo apt-get install -y intel-opencl-icd intel-level-zero-gpu level-zero intel-media-va-driver-non-free libmfx1 libmfxgen1 libvpl2 libegl-mesa0 libegl1-mesa libegl1-mesa-dev libgbm1 libgl1-mesa-dev libgl1-mesa-dri libglapi-mesa libgles2-mesa-dev libglx-mesa0 libigdgmm12 libxatracker2 mesa-va-drivers mesa-vdpau-drivers mesa-vulkan-drivers va-driver-all vainfo
sudo apt-get install -y libigc-dev intel-igc-cm libigdfcl-dev libigfxcmrt-dev level-zero-dev
sudo reboot
-----
Ubuntu Jammy 5.15.0-57-generic
-----
stat -c "%G" /dev/dri/render*
groups ${USER}
sudo gpasswd -a ${USER} render
newgrp render
sudo apt-get install hwinfo
hwinfo --display
export DISPLAY=:0.0
vainfo
sudo apt-get install clinfo
clinfo
sudo reboot
-----
Ubuntu Jammy 5.15.0-57-generic
----- 
wget -qO - https://repositories.intel.com/graphics/intel-graphics.key | sudo gpg --dearmor --output /usr/share/keyrings/intel-graphics.gpg
echo 'deb [arch=amd64,i386 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/graphics/ubuntu jammy arc' | sudo tee /etc/apt/sources.list.d/intel.gpu.jammy.list
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
echo 'deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main' | sudo tee /etc/apt/sources.list.d/oneAPI.list
sudo apt update && sudo apt upgrade -y
sudo apt-get install intel-opencl-icd intel-level-zero-gpu level-zero intel-media-va-driver-non-free libmfx1 libgl-dev intel-oneapi-compiler-dpcpp-cpp intel-oneapi-mkl python3-pip
sudo apt install python3.10-venv mc git
python3 -m pip install --user virtualenv
echo 'source /opt/intel/oneapi/setvars.sh' >> ~/.bashrc
git clone https://github.com/vladmandic/automatic
cd automatic
wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/89283df8-c667-47b0-b7e1-c4573e37bd3e/2023.1-linux-hotfix.zip
unzip 2023.1-linux-hotfix.zip
cd 2023.1-linux-hotfix
sudo cp libpi_level_zero.so /opt/intel/oneapi/compiler/2023.1.0/linux/lib/libpi_level_zero.so
sudo chmod +775 /opt/intel/oneapi/compiler/2023.1.0/linux/lib/libpi_level_zero.so
exit
-----
Load new Terminal Emulator
-----
cd automatic
python3 installer,py
python3 launch.py --use-ipex
------
Setup of WebUI
Cross-attention layer optimization
Set - InvokeAI's

[TotalDay]

Tested - Work!
Thanks

[onix331]

This worked flawlessly, using kernel 5.15.0-72-generic on Ubuntu 22.04.2
Thank you lots

[kotx]

--lowvram and --medvram options don't seem to work.

22:43:38-621186 ERROR    loading stable diffusion model: AssertionError                                                                                                                     
╭─────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────────────────────────────────────╮
│ /home/kot/Documents/automatic/webui.py:151 in load_model                                                                                                                                 │
│ ❱ 151 │   │   modules.sd_models.load_model()                                                                                                                                             │
│   152 │   │   modules.sd_models.skip_next_load = True                                                                                                                                    │
│                                                                                                                                                                                          │
│ /home/kot/Documents/automatic/modules/sd_models.py:399 in load_model                                                                                                                     │
│                                                                                                                                                                                          │
│   398 │   if shared.cmd_opts.use_ipex:                                                                                                                                                   │
│ ❱ 399 │   │   sd_model = torch.xpu.optimize(sd_model, dtype=devices.dtype)                                                                                                               │
│   400 │   │   shared.log.info("Applied IPEX Optimize")                                                                                                                                   │
│                                                                                                                                                                                          │
│ /home/kot/Documents/automatic/venv/lib/python3.10/site-packages/intel_extension_for_pytorch/xpu/utils.py:259 in optimize                                                                 │
│ ❱ 259 │   return frontend.optimize(model, dtype, optimizer, level,                                                                                                                       │
│   260 │   │   │   │   │   │   │    inplace, conv_bn_folding, linear_bn_folding,                                                                                                          │
│                                                                                                                                                                                          │
│   579 │   │   │   │   if dtype == torch.half:                                                                                                                                            │
│ ❱ 580 │   │   │   │   │   assert core.onednn_has_fp16_support(), \                                                                                                                       │
│   581 │   │   │   │   │   │   "FP16 weight prepack needs the cpu support avx512_core_fp16, " +                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
AssertionError: FP16 weight prepack needs the cpu support avx512_core_fp16, please set dtype to torch.float or set weights_prepack to False.
22:43:38-737421 ERROR    Stable diffusion model failed to load

[Disty0]

This commit should fix this issue:
8022de7

[silentst0rm]

ControlNet depth does not work

---

Loading model: control_v11p_sd15_depth_fp16 [12c052a1]
Loaded state_dict from [/home/silentstorm/automatic/models/ControlNet/control_v11p_sd15_depth_fp16.safetensors]
Loading config: /home/silentstorm/automatic/models/ControlNet/control_v11p_sd15_depth_fp16.yaml
ControlNet model control_v11p_sd15_depth_fp16 [12c052a1] loaded.
Loading preprocessor: depth
preprocessor resolution = 512
ControlNet preprocessor location: /home/silentstorm/automatic/extensions-builtin/sd-webui-controlnet/annotator/downloads
20:56:59-197382 ERROR Running script process: /home/silentstorm/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py: RuntimeError
╭───────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────────────────────────────────╮
│ /home/silentstorm/automatic/modules/scripts.py:376 in process │
│ │
│ 375 │ │ │ │ args = p.per_script_args.get(script.title(), p.script_args[script.args_f │
│ ❱ 376 │ │ │ │ script.process(p, *args, **kwargs) │
│ 377 │ │ │ │ # log.debug(f'Script process end : {script.title()}') │
│ │
│ /home/silentstorm/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py:1172 in process │
│ │
│ 1171 │ │ │ print(f'preprocessor resolution = {preprocessor_resolution}') │
│ ❱ 1172 │ │ │ detected_map, is_image = preprocessor(input_image, res=preprocessor_resoluti │
│ 1173 │
│ │
│ ... 15 frames hidden ... │
│ │
│ /home/silentstorm/.local/lib/python3.10/site-packages/timm/models/layers/std_conv.py:69 in forward │
│ │
│ 68 │ │ │ x = pad_same(x, self.kernel_size, self.stride, self.dilation) │
│ ❱ 69 │ │ weight = F.batch_norm( │
│ 70 │ │ │ self.weight.reshape(1, self.out_channels, -1), None, None, │
│ │
│ /home/silentstorm/.local/lib/python3.10/site-packages/torch/nn/functional.py:2450 in batch_norm │
│ │
│ 2449 │ │
│ ❱ 2450 │ return torch.batch_norm( │
│ 2451 │ │ input, weight, bias, running_mean, running_var, training, momentum, eps, torch.b │
╰─────────────────────────────────────────────────────────────────────────────────────
RuntimeError: tensor does not have a device

[vladmandic]

interesting - i'll raise that with controlnet author.

[Disty0]

anime_lineart preprocessor doesn't work too. But anime_lineart_denoise works.
It seems like only the preprocessor cause this error.

[Disty0]

Seems like torch.batch_norm and torch.instance_norm doesn't support XPU.
Fixed them in this commit: 102503a

Everything works expect Shuffle and Normal_BAE.
Normal works with Normal_Midas.
Shuffle throws this error when generating an image:

/home/disty/Apps/automatic/extensions-builtin/sd-webui-controlnet/scripts/hook.py:162 in cat                                                               │
│                                                                                                                                                            │
│   161 │   │                                                                                                                                                │
│ ❱ 162 │   │   return torch.cat(tensors, *args, **kwargs)                                                                                                   │
│   163                                                                                                                                                      │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: could not create a primitive descriptor for a concat primitive

[Disty0]

Wrote a guide for Intel ARC GPU's:

https://www.technopat.net/sosyal/konu/using-stable-diffusion-webui-with-intel-arc-gpus.2593077/

[Disty0]

intel-compute-runtime intel-graphics-compiler from the official repos was giving random errors when i used them.
But it seems like official packages catched up to the AUR.

Python 3.11 is not supported by IPEX so you have to use python310.
Also you don't need to uninstall python 3.11. Python310 won't affect the latest one.

[HubKing]

I installed all those packages listed for Arch (from official repository when possible). webui.sh said not edit itself, so I created a new file webui-user.sh next to webui.sh and added the following line

python_cmd="python3.10"

I was not sure about "Apply Intel OneAPI DPC++ Patch:​", so I skipped it, and ran ./webui.sh --use-ipex, but the script crashed. Is this because I did not apply that patch?

Create and activate python venv
Setting OneAPI environment
 
:: initializing oneAPI environment ...
   webui.sh: BASH_VERSION = 5.1.16(1)-release
   args: Using "$@" for setvars.sh arguments: --use-ipex
:: advisor -- latest
:: ccl -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: dnnl -- latest
:: dpcpp-ct -- latest
:: dpl -- latest
:: ipp -- latest
:: ippcp -- latest
:: ipp -- latest
:: mkl -- latest
:: mpi -- latest
:: tbb -- latest
:: vtune -- latest
:: oneAPI environment initialized ::
 
Launching launch.py...
Traceback (most recent call last):
  File "/home/username/diffusion/automatic/launch.py", line 11, in <module>
    import installer
  File "/home/username/diffusion/automatic/installer.py", line 12, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

[Disty0]

"Apply Intel OneAPI DPC++ Patch:​" This fixes some memory leak issues.

What i meant by add python_cmd="python3.10" to the start is this:

python_cmd="python3.10" ./webui.sh --use-ipex
It shloud create a virtual environment with Python 3.10 and python3 will be replaced with python3.10 in that venv so doing this at the first start is enough.

Error you have is probably from Python 3.11.
Remove the venv folder and try again.

[HubKing]

Thanks. Deleting the directory solved that problem. I succeeded to create an image with the default settings other than the sampling method as your page mentioned. However, there were some warnings and failure messages during the start up. Are they ignorable?

Launching ipexrun launch.py...
/home/username/diffusion/automatic/venv/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - INFO - Use TCMalloc memory allocator
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - INFO - OMP_NUM_THREADS=6
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - WARNING - Unable to find the iomp library file libiomp5.so in $CONDA_PREFIX/lib or $VIRTUAL_ENV/lib or /.local/lib/ or /usr/local/lib/ or /usr/local/lib64/ or /usr/lib or /usr/lib64 or /home/username/.local/lib/ so the LD_PRELOAD environment variable will not be set.you can use 'conda install intel-openm' to install intel openMP
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - INFO - LD_PRELOAD=/usr/lib/libtcmalloc.so
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - INFO - numactl -C 0-5 -m 0 /home/username/diffusion/automatic/venv/bin/python3.10 -u launch.py --use-ipex
21:17:57-263837 INFO     Running extension preloading                                                                                                                         
21:17:57-267650 INFO     Starting SD.Next                                                                                                                                     
21:17:57-268282 INFO     Python 3.10.11 on Linux                                                                                                                              
21:17:57-280197 INFO     Version: 9726b4d2 Sun Jun 4 13:21:52 2023 -0400                                                                                                      
21:17:57-622929 INFO     Setting environment tuning                                                                                                                           
21:17:57-623991 INFO     Intel OneAPI Toolkit detected                                                                                                                        
21:17:58-096736 INFO     Torch 1.13.0a0+gitb1dde16                                                                                                                            
/home/username/diffusion/automatic/venv/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
21:17:58-684113 INFO     Torch backend: Intel IPEX 1.13.120+xpu                                                                                                               
21:17:58-788289 INFO     Intel(R) oneAPI DPC++/C++ Compiler 2023.1.0 (2023.1.0.20230320)                                                                                      
                         Target: x86_64-unknown-linux-gnu                                                                                                                     
                         Thread model: posix                                                                                                                                  
                         InstalledDir: /opt/intel/oneapi/compiler/2023.1.0/linux/bin-llvm                                                                                     
                         Configuration file: /opt/intel/oneapi/compiler/2023.1.0/linux/bin-llvm/../bin/icpx.cfg                                                               
21:17:58-791248 INFO     Torch detected GPU: Intel(R) Arc(TM) A770 Graphics VRAM 15474                                                                                        
21:17:58-792248 INFO     Verifying requirements                                                                                                                               
21:17:58-805488 INFO     Installing packages                                                                                                                                  
21:17:58-814754 INFO     No changes detected: Quick launch active                                                                                                             
21:17:58-815647 INFO     Running extension preloading                                                                                                                         
21:17:58-816243 INFO     Server arguments: ['--use-ipex']                                                                                                                     
No module 'xformers'. Proceeding without it.
21:18:01-164791 INFO     Libraries loaded                                                                                                                                     
21:18:01-167465 INFO     Using data path: /home/username/diffusion/automatic                                                                                                       
21:18:01-168866 INFO     Available VAEs: /home/username/diffusion/automatic/models/VAE 0                                                                                           
21:18:01-169673 INFO     Available models: /home/username/diffusion/automatic/models/Stable-diffusion 0  

[Disty0]

These are harmless, ignore them.

warn(f"Failed to load image Python extension: {e}")
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - INFO - Use TCMalloc memory allocator
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - INFO - OMP_NUM_THREADS=6
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - WARNING - Unable to find the iomp library file libiomp5.so in $CONDA_PREFIX/lib or $VIRTUAL_ENV/lib or /.local/lib/ or /usr/local/lib/ or /usr/local/lib64/ or /usr/lib or /usr/lib64 or /home/username/.local/lib/ so the LD_PRELOAD environment variable will not be set.you can use 'conda install intel-openm' to install intel openMP
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - INFO - LD_PRELOAD=/usr/lib/libtcmalloc.so
2023-06-05 21:17:56,999 - intel_extension_for_pytorch.cpu.launch - INFO - numactl -C 0-5 -m 0 /home/username/diffusion/automatic/venv/bin/python3.10 -u launch.py --use-ipex

This is expected since xformers is for Nvidia:
No module 'xformers'. Proceeding without it.

[Nuullll]

Made a docker image that works for SD.Next + Arc: https://github.com/Nuullll/ipex-sd-docker-for-arc-gpu
Would love to hear feedback from the Arc community!

[cyatarow]

Just a question.
This repo seems to support torch 2.0, but with Intel Arc, it uses 1.13. (I use WSL.)
Can torch 2.0 work with Intel Arc?

The System Info tab says:

Torch
1.13.0a0+gitb1dde16 Autocast half

[kotx]

Installation succeeded with running pip install --upgrade torch==2.0.0a0 torchvision==0.15.2 intel_extension_for_pytorch==2.0.110+gitba7f6c1 -f https://developer.intel.com/ipex-whl-stable-xpu --no-deps (replacing torchvision 0.15.2a0 with 0.15.2 and ignoring dep conflicts manually in the venv, but startup fails with this:

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ C:\Users\Kot\Documents\automatic\launch.py:192 in <module>                                                                                                                                                                                               │
│                                                                                                                                                                                                                                                          │
│   191                                                                                                                                                                                                                                                    │
│ ❱ 192     instance = start_server(immediate=True, server=None)                                                                                                                                                                                           │
│   193     while True:                                                                                                                                                                                                                                    │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\launch.py:136 in start_server                                                                                                                                                                                           │
│                                                                                                                                                                                                                                                          │
│   135     installer.log.info(f"Server arguments: {sys.argv[1:]}")                                                                                                                                                                                        │
│ ❱ 136     module_spec.loader.exec_module(server)                                                                                                                                                                                                         │
│   137     if args.test:                                                                                                                                                                                                                                  │
│ in exec_module:883                                                                                                                                                                                                                                       │
│ in _call_with_frames_removed:241                                                                                                                                                                                                                         │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\webui.py:18 in <module>                                                                                                                                                                                                 │
│                                                                                                                                                                                                                                                          │
│    17 errors.log.debug('Loading Torch')                                                                                                                                                                                                                  │
│ ❱  18 import torch # pylint: disable=C0411                                                                                                                                                                                                               │
│    19 try:                                                                                                                                                                                                                                               │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\venv\lib\site-packages\torch\__init__.py:133 in <module>                                                                                                                                                                │
│                                                                                                                                                                                                                                                          │
│    132                 err.strerror += f' Error loading "{dll}" or one of its dependencies.'                                                                                                                                                             │
│ ❱  133                 raise err                                                                                                                                                                                                                         │
│    134                                                                                                                                                                                                                                                   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\Kot\Documents\automatic\venv\lib\site-packages\torch\lib\backend_with_compiler.dll" or one of its dependencies.
Press any key to continue . . .

Running it again yields a different error!

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ C:\Users\Kot\Documents\automatic\launch.py:192 in <module>                                                                                                                                                                                               │
│                                                                                                                                                                                                                                                          │
│   191                                                                                                                                                                                                                                                    │
│ ❱ 192     instance = start_server(immediate=True, server=None)                                                                                                                                                                                           │
│   193     while True:                                                                                                                                                                                                                                    │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\launch.py:136 in start_server                                                                                                                                                                                           │
│                                                                                                                                                                                                                                                          │
│   135     installer.log.info(f"Server arguments: {sys.argv[1:]}")                                                                                                                                                                                        │
│ ❱ 136     module_spec.loader.exec_module(server)                                                                                                                                                                                                         │
│   137     if args.test:                                                                                                                                                                                                                                  │
│ in exec_module:883                                                                                                                                                                                                                                       │
│ in _call_with_frames_removed:241                                                                                                                                                                                                                         │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\webui.py:44 in <module>                                                                                                                                                                                                 │
│                                                                                                                                                                                                                                                          │
│    43 import ldm.modules.encoders.modules # pylint: disable=W0611,C0411,E0401                                                                                                                                                                            │
│ ❱  44 from modules.call_queue import queue_lock, wrap_queued_call, wrap_gradio_gpu_call # pyli                                                                                                                                                           │
│    45 from modules.paths import create_paths                                                                                                                                                                                                             │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\modules\call_queue.py:8 in <module>                                                                                                                                                                                     │
│                                                                                                                                                                                                                                                          │
│     7 from rich import print # pylint: disable=redefined-builtin                                                                                                                                                                                         │
│ ❱   8 from modules import shared, progress, errors                                                                                                                                                                                                       │
│     9                                                                                                                                                                                                                                                    │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\modules\shared.py:16 in <module>                                                                                                                                                                                        │
│                                                                                                                                                                                                                                                          │
│     15 from modules.dml import memory_providers, default_memory_provider, directml_do_hijack                                                                                                                                                             │
│ ❱   16 import modules.interrogate                                                                                                                                                                                                                        │
│     17 import modules.memmon                                                                                                                                                                                                                             │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\modules\interrogate.py:10 in <module>                                                                                                                                                                                   │
│                                                                                                                                                                                                                                                          │
│     9 from torchvision.transforms.functional import InterpolationMode                                                                                                                                                                                    │
│ ❱  10 from modules import devices, paths, shared, lowvram, modelloader, errors                                                                                                                                                                           │
│    11                                                                                                                                                                                                                                                    │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\modules\devices.py:170 in <module>                                                                                                                                                                                      │
│                                                                                                                                                                                                                                                          │
│   169     backend = 'ipex'                                                                                                                                                                                                                               │
│ ❱ 170     from modules.ipex_specific import ipex_init                                                                                                                                                                                                    │
│   171     ipex_init()                                                                                                                                                                                                                                    │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\modules\ipex_specific\__init__.py:4 in <module>                                                                                                                                                                         │
│                                                                                                                                                                                                                                                          │
│     3 import torch                                                                                                                                                                                                                                       │
│ ❱   4 import intel_extension_for_pytorch as ipex                                                                                                                                                                                                         │
│     5 from modules import shared                                                                                                                                                                                                                         │
│                                                                                                                                                                                                                                                          │
│ C:\Users\Kot\Documents\automatic\venv\lib\site-packages\intel_extension_for_pytorch\__init__.py:89 in <module>                                                                                                                                           │
│                                                                                                                                                                                                                                                          │
│    88                 err.strerror += f' Error loading "{dll}" or one of its dependencies.'                                                                                                                                                              │
│ ❱  89                 raise err                                                                                                                                                                                                                          │
│    90                                                                                                                                                                                                                                                    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\Kot\Documents\automatic\venv\lib\site-packages\intel_extension_for_pytorch\bin\intel-ext-pt-gpu.dll" or one of its dependencies.
Press any key to continue . . .

[Nuullll]

@kotx intel/intel-extension-for-pytorch#398

[cyatarow]

How can I run it on Windows natively? Is there a guide somewhere?
And which is faster and more memory efficient, running on Windows natively or via WSL?

[Disty0]

#2023

Linux is faster.

[cyatarow]

How about the VRAM leak issue?

[cyatarow]

e70520e
Wow, OpenVINO is also supported now? I'm curious about the difference from the IPEX implementation.

[Disty0]

Yes, It's basically Intel's TensorRT equivalent.
It's limited just like TensorRT but OpenVINO doesn't need manual model conversions.
OpenVINO handles conversions with torch.compile.
And OpenVINO can run on iGPUs too.

Also don't use OpenVINO with IPEX.
Use OpenVINO as a seperate install.

A770 goes from 8 it/s to 10 it/s with OpenVINO at 512x512.
My R7 5800X3D goes from 6 s/it to 3 s/it with OpenVINO CPU mode.

Export this environment variable to use CPU mode:
OPENVINO_TORCH_BACKEND_DEVICE=CPU

[cyatarow]

So OpenVINO is more powerful and usable than IPEX?
If so, I'll reinstall this repo with OpenVINO.

[Disty0]

Better for lower res, (Up to 768x768 for SD 1.5 and 1536x1536 for SDXL).
If you generate at high res, don't wipe out IPEX.