Version 2.3.6 of flash-attn Is Incompatible With axolotl

[casey-martin]

Please check that this issue hasn't been reported before.

• I searched previous Bug Reports didn't find any similar reports.

Expected Behavior

Installation via the git instructions in a clean conda environment was successful. I should be able to execute scripts using axolotl.cli.inference.

Current behaviour

Executing scripts results in an ImportError.

ImportError: undefined symbol: _ZN3c104cuda9SetDeviceEi

This error has been recently reported by "Oobabooga users".

Steps to reproduce

$ accelerate launch -m axolotl.cli.train ./phi-ft.yml --deepspeed ./deepspeed/zero1.json 
The following values were not passed to `accelerate launch` and had defaults used instead:
        `--num_processes` was set to a value of `1`
        `--num_machines` was set to a value of `1`
        `--mixed_precision` was set to a value of `'no'`
        `--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Traceback (most recent call last):                                            
  File "/home/user/mambaforge/envs/phi/lib/python3.9/runpy.py", line 188, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/user/mambaforge/envs/phi/lib/python3.9/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)                                                      
  File "/home/user/scripts/project/axolotl/axolotl/src/axolotl/cli/__init__.py", line 23, in <module>
    from axolotl.common.cli import TrainerCliArgs, load_model_and_tokenizer
  File "/home/user/scripts/project/axolotl/axolotl/src/axolotl/common/cli.py", line 11, in <module>
    from axolotl.utils.models import load_model, load_tokenizer
  File "/home/user/scripts/project/axolotl/axolotl/src/axolotl/utils/models.py", line 10, in <module>
    from optimum.bettertransformer import BetterTransformer
  File "/home/user/mambaforge/envs/phi/lib/python3.9/site-packages/optimum/bettertransformer/__init__.py", line 14, in <module>
    from .models import BetterTransformerManager
  File "/home/user/mambaforge/envs/phi/lib/python3.9/site-packages/optimum/bettertransformer/models/__init__.py", line 17, in <module>
    from .decoder_models import (                                             
  File "/home/user/mambaforge/envs/phi/lib/python3.9/site-packages/optimum/bettertransformer/models/decoder_models.py", line 18, in <module>
    from transformers.models.bart.modeling_bart import BartAttention
  File "/home/user/mambaforge/envs/phi/lib/python3.9/site-packages/transformers/models/bart/modeling_bart.py", line 52, in <module>
    from flash_attn import flash_attn_func, flash_attn_varlen_func
  File "/home/user/mambaforge/envs/phi/lib/python3.9/site-packages/flash_attn/__init__.py", line 3, in <module>
    from flash_attn.flash_attn_interface import (
  File "/home/user/mambaforge/envs/phi/lib/python3.9/site-packages/flash_attn/flash_attn_interface.py", line 10, in <module>
    import flash_attn_2_cuda as flash_attn_cuda
ImportError: /home/user/mambaforge/envs/phi/lib/python3.9/site-packages/flash_attn_2_cuda.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

Config yaml

base_model: microsoft/phi-1_5
model_type: PhiForCausalLM
tokenizer_type: AutoTokenizer
is_llama_derived_model: false
trust_remote_code: true

load_in_8bit: false
load_in_4bit: false
strict: false

datasets:
  - path: garage-bAInd/Open-Platypus
    type: alpaca

dataset_prepared_path:
val_set_size: 0.05
output_dir: ./phi-sft-out

sequence_len: 2048
sample_packing: true
pad_to_sequence_len:

adapter:
lora_model_dir:
lora_r:
lora_alpha:
lora_dropout:
lora_target_linear:
lora_fan_in_fan_out:

wandb_project:
wandb_entity:
wandb_watch:
wandb_run_id:
wandb_log_model:

gradient_accumulation_steps: 1
micro_batch_size: 1
num_epochs: 4
optimizer: adamw_torch
adam_beta2: 0.95
adam_epsilon: 0.00001
max_grad_norm: 1.0
lr_scheduler: cosine
learning_rate: 0.000003

train_on_inputs: false
group_by_length: true
bf16: true
fp16: false
tf32: true

gradient_checkpointing:
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention:

warmup_steps: 100
eval_steps: 0.05
save_steps:
debug:
deepspeed:
weight_decay: 0.1
fsdp:
fsdp_config:
resize_token_embeddings_to_32x: true
special_tokens:
  bos_token: "<|endoftext|>"
  eos_token: "<|endoftext|>"
  unk_token: "<|endoftext|>"
  pad_token: "<|endoftext|>"

Possible solution

Rolling back to flash-attn==2.3.3 resolves the issue.

Which Operating Systems are you using?

• Linux
• macOS
• Windows

Python Version

3.9

axolotl branch-commit

main/4d6490b

Acknowledgements

• My issue title is concise, descriptive, and in title casing.
• I have searched the existing issues to make sure this bug has not been reported yet.
• I am using the latest version of axolotl.
• I have provided enough information for the maintainers to reproduce and diagnose the issue.

[winglian]

Did you mean flash-attn instead of accelerate?

[casey-martin]

Apologies - you are correct. I'll edit the issue accordingly.

[MB7979]

I experienced this too. Downgrading to 2.3.3 solved it for now.

[npmccallum]

FYI, I hit this in the container (docker.io/winglian/axolotl:main-py3.10-cu118-2.1.0) today. The workaround worked. But this means that the default container image doesn't work out of the box.

[casper-hansen]

I got the same. I think the issue may be that flash attention wheels are built using the nightly build of PyTorch.

Dao-AILab/flash-attention@d4a7c8f#diff-551d1fcf87f78cc3bc18a7b332a4dc5d8773a512062df881c5aba28a6f5c48d7R47

[casper-hansen]

Until this is fixed, it's easy to install a working version:

pip install -e .
pip install flash-attn==2.3.3

[linux-leo]

I am experiencing this issue in a colab pro+ enviroment, despite using flash-attn 2.3.3, for me, flash-attn 2.3.2 worked.

Output of ´pip list´

Package                          Version               Editable project location
-------------------------------- --------------------- -------------------------
absl-py                          1.4.0
accelerate                       0.25.0.dev0
addict                           2.4.0
aiobotocore                      2.5.4
aiofiles                         23.2.1
aiohttp                          3.9.1
aioitertools                     0.11.0
aiosignal                        1.3.1
alabaster                        0.7.13
albumentations                   1.3.1
altair                           4.2.2
anyio                            3.7.1
appdirs                          1.4.4
argon2-cffi                      23.1.0
argon2-cffi-bindings             21.2.0
array-record                     0.5.0
art                              6.1
arviz                            0.15.1
astropy                          5.3.4
astunparse                       1.6.3
async-timeout                    4.0.3
atpublic                         4.0
attrs                            23.1.0
audioread                        3.0.1
auto-gptq                        0.5.1
autograd                         1.6.2
axolotl                          0.3.0                 /content/axolotl/src
Babel                            2.14.0
backcall                         0.2.0
beautifulsoup4                   4.11.2
bert-score                       0.3.13
bidict                           0.22.1
bigframes                        0.17.0
bitsandbytes                     0.41.3.post2
bleach                           6.1.0
blinker                          1.4
blis                             0.7.11
blosc2                           2.0.0
bokeh                            3.3.2
botocore                         1.31.17
bqplot                           0.12.42
branca                           0.7.0
build                            1.0.3
CacheControl                     0.13.1
cachetools                       5.3.2
catalogue                        2.0.10
certifi                          2023.11.17
cffi                             1.16.0
chardet                          5.2.0
charset-normalizer               3.3.2
chex                             0.1.7
click                            8.1.7
click-plugins                    1.1.1
cligj                            0.7.2
cloudpickle                      2.2.1
cmake                            3.27.9
cmdstanpy                        1.2.0
colorama                         0.4.6
colorcet                         3.0.1
coloredlogs                      15.0.1
colorlover                       0.3.0
colour                           0.1.5
community                        1.0.0b1
confection                       0.1.4
cons                             0.4.6
contextlib2                      21.6.0
contourpy                        1.2.0
cryptography                     41.0.7
cufflinks                        0.17.3
cupy-cuda12x                     12.2.0
cvxopt                           1.3.2
cvxpy                            1.3.2
cycler                           0.12.1
cymem                            2.0.8
Cython                           3.0.6
dask                             2023.8.1
datascience                      0.17.6
datasets                         2.16.0
db-dtypes                        1.2.0
dbus-python                      1.2.18
debugpy                          1.6.6
decorator                        4.4.2
deepspeed                        0.12.6
defusedxml                       0.7.1
dill                             0.3.7
diskcache                        5.6.3
distributed                      2023.8.1
distro                           1.7.0
dlib                             19.24.2
dm-tree                          0.1.8
docker-pycreds                   0.4.0
docutils                         0.18.1
dopamine-rl                      4.0.6
duckdb                           0.9.2
earthengine-api                  0.1.384
easydict                         1.11
ecos                             2.0.12
editdistance                     0.6.2
eerepr                           0.0.4
einops                           0.7.0
en-core-web-sm                   3.6.0
entrypoints                      0.4
et-xmlfile                       1.1.0
etils                            1.6.0
etuples                          0.3.9
evaluate                         0.4.0
exceptiongroup                   1.2.0
fastai                           2.7.13
fastapi                          0.105.0
fastcore                         1.5.29
fastdownload                     0.0.7
fastjsonschema                   2.19.0
fastprogress                     1.0.3
fastrlock                        0.8.2
ffmpy                            0.3.1
filelock                         3.13.1
fiona                            1.9.5
fire                             0.5.0
firebase-admin                   5.3.0
flash-attn                       2.3.3
Flask                            2.2.5
flatbuffers                      23.5.26
flax                             0.7.5
folium                           0.14.0
fonttools                        4.46.0
frozendict                       2.3.10
frozenlist                       1.4.1
fschat                           0.2.34
fsspec                           2023.6.0
future                           0.18.3
gast                             0.5.4
gcsfs                            2023.6.0
GDAL                             3.4.3
gdown                            4.6.6
geemap                           0.29.6
gekko                            1.0.6
gensim                           4.3.2
geocoder                         1.38.1
geographiclib                    2.0
geopandas                        0.13.2
geopy                            2.3.0
gin-config                       0.5.0
gitdb                            4.0.11
GitPython                        3.1.40
glob2                            0.7
google                           2.0.3
google-ai-generativelanguage     0.4.0
google-api-core                  2.11.1
google-api-python-client         2.84.0
google-auth                      2.17.3
google-auth-httplib2             0.1.1
google-auth-oauthlib             1.2.0
google-cloud-aiplatform          1.38.1
google-cloud-bigquery            3.12.0
google-cloud-bigquery-connection 1.12.1
google-cloud-bigquery-storage    2.24.0
google-cloud-core                2.3.3
google-cloud-datastore           2.15.2
google-cloud-firestore           2.11.1
google-cloud-functions           1.13.3
google-cloud-iam                 2.13.0
google-cloud-language            2.9.1
google-cloud-resource-manager    1.11.0
google-cloud-storage             2.8.0
google-cloud-translate           3.11.3
google-colab                     1.0.0
google-crc32c                    1.5.0
google-generativeai              0.3.1
google-pasta                     0.2.0
google-resumable-media           2.7.0
googleapis-common-protos         1.62.0
googledrivedownloader            0.4
gradio                           3.50.2
gradio_client                    0.6.1
graphviz                         0.20.1
greenlet                         3.0.2
grpc-google-iam-v1               0.13.0
grpcio                           1.60.0
grpcio-status                    1.48.2
gspread                          3.4.2
gspread-dataframe                3.3.1
gym                              0.25.2
gym-notices                      0.0.8
h11                              0.14.0
h5netcdf                         1.3.0
h5py                             3.9.0
hf_transfer                      0.1.4
hjson                            3.1.0
holidays                         0.38
holoviews                        1.17.1
html5lib                         1.1
httpcore                         1.0.2
httpimport                       1.3.1
httplib2                         0.22.0
httpx                            0.26.0
huggingface-hub                  0.19.4
humanfriendly                    10.0
humanize                         4.7.0
hyperopt                         0.2.7
ibis-framework                   6.2.0
idna                             3.6
imageio                          2.31.6
imageio-ffmpeg                   0.4.9
imagesize                        1.4.1
imbalanced-learn                 0.10.1
imgaug                           0.4.0
importlib-metadata               7.0.0
importlib-resources              6.1.1
imutils                          0.5.4
inflect                          7.0.0
iniconfig                        2.0.0
install                          1.3.5
intel-openmp                     2023.2.3
ipyevents                        2.0.2
ipyfilechooser                   0.6.0
ipykernel                        5.5.6
ipyleaflet                       0.18.1
ipython                          7.34.0
ipython-genutils                 0.2.0
ipython-sql                      0.5.0
ipytree                          0.2.2
ipywidgets                       7.7.1
itsdangerous                     2.1.2
jax                              0.4.23
jaxlib                           0.4.23+cuda12.cudnn89
jeepney                          0.7.1
jieba                            0.42.1
Jinja2                           3.1.2
jmespath                         1.0.1
joblib                           1.3.2
jsonpickle                       3.0.2
jsonschema                       4.19.2
jsonschema-specifications        2023.11.2
jupyter-client                   6.1.12
jupyter-console                  6.1.0
jupyter_core                     5.5.1
jupyter-server                   1.24.0
jupyterlab_pygments              0.3.0
jupyterlab-widgets               3.0.9
kaggle                           1.5.16
kagglehub                        0.1.4
keras                            2.15.0
keyring                          23.5.0
kiwisolver                       1.4.5
langcodes                        3.3.0
launchpadlib                     1.10.16
lazr.restfulclient               0.14.4
lazr.uri                         1.0.6
lazy_loader                      0.3
libclang                         16.0.6
librosa                          0.10.1
lida                             0.0.10
lightgbm                         4.1.0
linkify-it-py                    2.0.2
lit                              17.0.6
llmx                             0.0.15a0
llvmlite                         0.41.1
locket                           1.0.0
logical-unification              0.4.6
lxml                             4.9.3
malloy                           2023.1067
Markdown                         3.5.1
markdown-it-py                   3.0.0
markdown2                        2.4.12
MarkupSafe                       2.1.3
matplotlib                       3.7.1
matplotlib-inline                0.1.6
matplotlib-venn                  0.11.9
mdit-py-plugins                  0.4.0
mdurl                            0.1.2
miniKanren                       1.0.3
missingno                        0.5.2
mistune                          0.8.4
mizani                           0.9.3
mkl                              2023.2.0
ml-dtypes                        0.2.0
mlxtend                          0.22.0
more-itertools                   10.1.0
moviepy                          1.0.3
mpmath                           1.3.0
msgpack                          1.0.7
multidict                        6.0.4
multipledispatch                 1.0.0
multiprocess                     0.70.15
multitasking                     0.0.11
murmurhash                       1.0.10
music21                          9.1.0
natsort                          8.4.0
nbclassic                        1.0.0
nbclient                         0.9.0
nbconvert                        6.5.4
nbformat                         5.9.2
nest-asyncio                     1.5.8
networkx                         3.2.1
nh3                              0.2.15
nibabel                          4.0.2
ninja                            1.11.1.1
nltk                             3.8.1
notebook                         6.5.5
notebook_shim                    0.2.3
numba                            0.58.1
numexpr                          2.8.8
numpy                            1.26.2
nvidia-cublas-cu11               11.10.3.66
nvidia-cuda-cupti-cu11           11.7.101
nvidia-cuda-nvrtc-cu11           11.7.99
nvidia-cuda-runtime-cu11         11.7.99
nvidia-cudnn-cu11                8.5.0.96
nvidia-cufft-cu11                10.9.0.58
nvidia-curand-cu11               10.2.10.91
nvidia-cusolver-cu11             11.4.0.1
nvidia-cusparse-cu11             11.7.4.91
nvidia-nccl-cu11                 2.14.3
nvidia-nvtx-cu11                 11.7.91
oauth2client                     4.1.3
oauthlib                         3.2.2
opencv-contrib-python            4.8.0.76
opencv-python                    4.8.0.76
opencv-python-headless           4.8.1.78
openpyxl                         3.1.2
opt-einsum                       3.3.0
optax                            0.1.7
optimum                          1.13.2
orbax-checkpoint                 0.4.4
orjson                           3.9.10
osqp                             0.6.2.post8
packaging                        23.2
pandas                           1.5.3
pandas-datareader                0.10.0
pandas-gbq                       0.19.2
pandas-stubs                     1.5.3.230304
pandocfilters                    1.5.0
panel                            1.3.4
param                            2.0.1
parso                            0.8.3
parsy                            2.1
partd                            1.4.1
pathlib                          1.0.1
pathy                            0.10.3
patsy                            0.5.4
peewee                           3.17.0
peft                             0.7.2.dev0
pexpect                          4.9.0
pickleshare                      0.7.5
Pillow                           9.4.0
pip                              23.1.2
pip-tools                        6.13.0
platformdirs                     4.1.0
plotly                           5.15.0
plotnine                         0.12.4
pluggy                           1.3.0
polars                           0.17.3
pooch                            1.8.0
portpicker                       1.5.2
prefetch-generator               1.0.3
preshed                          3.0.9
prettytable                      3.9.0
proglog                          0.1.10
progressbar2                     4.2.0
prometheus-client                0.19.0
promise                          2.3
prompt-toolkit                   3.0.43
prophet                          1.1.5
proto-plus                       1.23.0
protobuf                         3.20.3
psutil                           5.9.5
psycopg2                         2.9.9
ptyprocess                       0.7.0
py-cpuinfo                       9.0.0
py4j                             0.10.9.7
pyarrow                          10.0.1
pyarrow-hotfix                   0.6
pyasn1                           0.5.1
pyasn1-modules                   0.3.0
pycocotools                      2.0.7
pycparser                        2.21
pyct                             0.5.0
pydantic                         1.10.13
pydata-google-auth               1.8.2
pydot                            1.4.2
pydot-ng                         2.0.0
pydotplus                        2.0.2
PyDrive                          1.3.1
PyDrive2                         1.6.3
pydub                            0.25.1
pyerfa                           2.0.1.1
pygame                           2.5.2
Pygments                         2.16.1
PyGObject                        3.42.1
PyJWT                            2.3.0
pymc                             5.7.2
pymystem3                        0.2.0
pynvml                           11.5.0
PyOpenGL                         3.1.7
pyOpenSSL                        23.3.0
pyparsing                        3.1.1
pyperclip                        1.8.2
pyproj                           3.6.1
pyproject_hooks                  1.0.0
pyshp                            2.3.1
PySocks                          1.7.1
pytensor                         2.14.2
pytest                           7.4.3
python-apt                       0.0.0
python-box                       7.1.1
python-dateutil                  2.8.2
python-louvain                   0.16
python-multipart                 0.0.6
python-slugify                   8.0.1
python-utils                     3.8.1
pytz                             2023.3.post1
pyviz_comms                      3.0.0
PyWavelets                       1.5.0
PyYAML                           6.0.1
pyzmq                            23.2.1
qdldl                            0.1.7.post0
qudida                           0.0.4
ratelim                          0.1.6
referencing                      0.32.0
regex                            2023.6.3
requests                         2.31.0
requests-oauthlib                1.3.1
requirements-parser              0.5.0
responses                        0.18.0
rich                             13.7.0
rouge                            1.0.1
rouge-score                      0.1.2
rpds-py                          0.15.2
rpy2                             3.4.2
rsa                              4.9
s3fs                             2023.6.0
safetensors                      0.4.1
scikit-image                     0.19.3
scikit-learn                     1.2.2
scipy                            1.11.4
scooby                           0.9.2
scs                              3.2.4.post1
seaborn                          0.12.2
SecretStorage                    3.3.1
semantic-version                 2.10.0
Send2Trash                       1.8.2
sentencepiece                    0.1.99
sentry-sdk                       1.39.1
setproctitle                     1.3.3
setuptools                       67.7.2
shapely                          2.0.2
shortuuid                        1.0.11
six                              1.16.0
sklearn-pandas                   2.2.0
smart-open                       6.4.0
smmap                            5.0.1
sniffio                          1.3.0
snowballstemmer                  2.2.0
sortedcontainers                 2.4.0
soundfile                        0.12.1
soupsieve                        2.5
soxr                             0.3.7
spacy                            3.6.1
spacy-legacy                     3.0.12
spacy-loggers                    1.0.5
Sphinx                           5.0.2
sphinxcontrib-applehelp          1.0.7
sphinxcontrib-devhelp            1.0.5
sphinxcontrib-htmlhelp           2.0.4
sphinxcontrib-jsmath             1.0.1
sphinxcontrib-qthelp             1.0.6
sphinxcontrib-serializinghtml    1.1.9
SQLAlchemy                       2.0.23
sqlglot                          17.16.2
sqlparse                         0.4.4
srsly                            2.4.8
stanio                           0.3.0
starlette                        0.27.0
statsmodels                      0.14.1
svgwrite                         1.4.3
sympy                            1.12
tables                           3.8.0
tabulate                         0.9.0
tbb                              2021.11.0
tblib                            3.0.0
tenacity                         8.2.3
tensorboard                      2.15.1
tensorboard-data-server          0.7.2
tensorflow                       2.15.0
tensorflow-datasets              4.9.4
tensorflow-estimator             2.15.0
tensorflow-gcs-config            2.15.0
tensorflow-hub                   0.15.0
tensorflow-io-gcs-filesystem     0.34.0
tensorflow-metadata              1.14.0
tensorflow-probability           0.22.0
tensorstore                      0.1.45
termcolor                        2.4.0
terminado                        0.18.0
text-unidecode                   1.3
textblob                         0.17.1
tf-slim                          1.1.0
thinc                            8.1.12
threadpoolctl                    3.2.0
tifffile                         2023.12.9
tiktoken                         0.5.2
tinycss2                         1.2.1
tokenizers                       0.15.0
toml                             0.10.2
tomli                            2.0.1
toolz                            0.12.0
torch                            2.0.1
torchaudio                       2.1.0+cu121
torchdata                        0.7.0
torchsummary                     1.5.1
torchtext                        0.16.0
torchvision                      0.16.0+cu121
tornado                          6.3.2
tqdm                             4.66.1
traitlets                        5.7.1
traittypes                       0.2.1
transformers                     4.37.0.dev0
triton                           2.0.0
tweepy                           4.14.0
typer                            0.9.0
types-pytz                       2023.3.1.1
types-setuptools                 69.0.0.0
typing_extensions                4.9.0
tzlocal                          5.2
uc-micro-py                      1.0.2
uritemplate                      4.1.1
urllib3                          1.26.18
uvicorn                          0.25.0
vega-datasets                    0.9.0
wadllib                          1.3.6
wandb                            0.16.1
wasabi                           1.1.2
wavedrom                         2.0.3.post3
wcwidth                          0.2.12
webcolors                        1.13
webencodings                     0.5.1
websocket-client                 1.7.0
websockets                       11.0.3
Werkzeug                         3.0.1
wheel                            0.42.0
widgetsnbextension               3.6.6
wordcloud                        1.9.3
wrapt                            1.14.1
xarray                           2023.7.0
xarray-einstats                  0.6.0
xformers                         0.0.22
xgboost                          2.0.2
xlrd                             2.0.1
xxhash                           3.4.1
xyzservices                      2023.10.1
yarl                             1.9.4
yellowbrick                      1.5
yfinance                         0.2.33
zict                             3.0.0
zipp                             3.17.0

[Sehyo]

I am getting this with flash attention 2.3.3.

[eryk-mazus]

Installing from source worked for me:

git clone https://github.com/Dao-AILab/flash-attention.git
cd flash-attention
pip install . --no-build-isolation

[Syzygianinfern0]

I have this same issue

[nctu6]

Same issue with flash_attn 2.5.9.post1 and 2.3.3.

[winglian]

@nctu6 what version of PyTorch?

[nctu6]

The incompatible problem occurs with flash_attn versions 2.5.9.post1 and 2.3.3 on H100.

axolotl 0.4.0
torch 2.1.1
torchaudio 2.1.1+cu121
torchvision 0.16.1+cu121

    result = forward_call(*args, **kwargs)                                                              
  File "/home/axolotl/src/axolotl/monkeypatch/llama_attn_hijack_flash.py", line 209, in forwa
rd                                                                                                      
    return self.swiglu(x)                                                                               
  File "/home/miniconda3/envs/axolotl_h100/lib/python3.10/site-packages/torch/nn/modules/modu
le.py", line 1518, in _wrapped_call_impl                                                                
    return self._call_impl(*args, **kwargs)                                                             
  File "/home/miniconda3/envs/axolotl_h100/lib/python3.10/site-packages/torch/nn/modules/modu
le.py", line 1568, in _call_impl                                                                        
    result = forward_call(*args, **kwargs)                                                              
  File "/home/miniconda3/envs/axolotl_h100/lib/python3.10/site-packages/xformers/ops/swiglu_o
p.py", line 440, in forward                                                                             
    return swiglu(x, *self._ordered_params(), op=self.op)                                               
  File "/home/miniconda3/envs/axolotl_h100/lib/python3.10/site-packages/xformers/ops/swiglu_o
p.py", line 459, in _ordered_params                                                                     
    w1w2.view([2, w1w2.shape[0] // 2, w1w2.shape[1]]),                                                  
IndexError: tuple index out of range