Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add experimental install guide for ROCm #1550

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Add experimental install guide for ROCm #1550

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9921e4f
Add experimental install guide for ROCm
xzuyn Apr 19, 2024
8e15925
Use the newer version of the bitsandbytes fork
xzuyn Apr 20, 2024
33727dc
Missed the clone location name
xzuyn Apr 21, 2024
a9a77a2
Use stable ROCm PyTorch
xzuyn Jun 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Features:
- [Conda/Pip venv](#condapip-venv)
- [Cloud GPU](#cloud-gpu) - Latitude.sh, JarvisLabs, RunPod
- [Bare Metal Cloud GPU](#bare-metal-cloud-gpu)
- [ROCm (Experimental)](#rocm-experimental)
- [Windows](#windows)
- [Mac](#mac)
- [Google Colab](#google-colab)
Expand Down Expand Up @@ -254,6 +255,45 @@ pip uninstall -y torch_xla[tpu]

</details>

#### ROCm (Experimental)

<details>

<summary>Click to Expand</summary>

**Axolotl is *not* maintained for ROCm use.**

This is an experimental setup guide for RDNA3 on ROCm 6.1 which has been tested by one person Ubuntu 22.04.4 with an RX 7900 XTX. *Your results may vary.*

There is no Flash Attention support except on [MI200 & MI300 GPUs using the ROCm fork](https://github.com/ROCm/flash-attention), so will need to use `sdp_attention: true` if you wish to use sample packing during training.

```bash
# Initial setup for axolotl
pip install packaging ninja
pip install -e '.[deepspeed]'

# Remove PyTorch, bitsandbytes, & xformers so we can replace them with ROCm versions (no xformers though)
pip uninstall -y torch bitsandbytes xformers
pip install torch --index-url https://download.pytorch.org/whl/rocm6.0

# Setup the bitsandbytes fork by arlo-phoenix
git clone -b rocm https://github.com/arlo-phoenix/bitsandbytes-rocm-5.6.git bitsandbytes
cd bitsandbytes

# Replace this with the correct folder name if you aren't on ROCm 6.1
export ROCM_HOME=/opt/rocm-6.1.0

# Replace this with the correct target if you aren't using an RDNA3 GPU
make hip ROCM_TARGET=gfx1100
pip install .

# Completely optional; You can uninstall nvidia related packages to save space
cd ..
pip uninstall -y nvidia-cublas-cu12 nvidia-cuda-cupti-cu12 nvidia-cuda-nvrtc-cu12 nvidia-cuda-runtime-cu12 nvidia-cudnn-cu12 nvidia-cufft-cu12 nvidia-curand-cu12 nvidia-cusolver-cu12 nvidia-cusparse-cu12 nvidia-nccl-cu12 nvidia-nvjitlink-cu12 nvidia-nvtx-cu12
```

</details>

#### Windows
Please use WSL or Docker!

Expand Down
Loading