Test Upgrading PyTorch and torchvision torchaudio with CUDA 11.6

[atalman]

Hello,

We are planning on releasing

pytorch v1.13.0
torchvision v0.14.0
torchaudio v0.13.0

With CUDA 11.6 support, deprecating CUDA 11.3. We want to try CUDA 11.6 wheels on Google Collab and make sure it works.

The wheel installation script:

pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cu116

Other notes
If you require wheels for Python 3.8 / 3.9 or 3.10 you can find the links here: download.pytorch.org/whl/nightly/

We're looking to having it updated in Colab for next release

Thanks very much.

cc'ing @colaboratory-team @craigcitro @malfet

Similar to #2372

[malfet]

Creating a collab to check that it works with 1.12.1, will update this comment with the findings and URL to the notebook
Looks like it simply passes the smoke test. I.e. following cell on GPU machine:

!pip3 uninstall -y torch
!pip3 install --pre torch  --extra-index-url https://download.pytorch.org/whl/cu116
import torch
print(torch.__version__)
print(torch.cuda.device_count(), torch.cuda.get_device_name(0), torch.cuda.get_device_capability(0))
x=torch.rand(3, 3, device="cuda")
print(torch.linalg.svd(torch.mm(x, x.t())))

Outputs following:

1.12.1+cu116
1 Tesla T4 (7, 5)
torch.return_types.linalg_svd(
U=tensor([[ 0.8758, -0.4787, -0.0621],
        [ 0.1730,  0.1912,  0.9662],
        [ 0.4506,  0.8569, -0.2502]], device='cuda:0'),
S=tensor([2.6278e+00, 2.3089e-01, 3.1565e-04], device='cuda:0'),
Vh=tensor([[ 0.8758,  0.1730,  0.4506],
        [-0.4787,  0.1912,  0.8569],
        [-0.0621,  0.9662, -0.2502]], device='cuda:0'))

Link to the notebook

[craigcitro]

That looks promising!

I'm fine saying that we can try the CUDA 11.6 wheels for the next torch release; if something goes sideways, we can fall back to the current version and reconsider. (If there are CUDA 11.6 wheels for the current pytorch release, we could also try those first.)

[robotaiguy]

You do realize that CUDA is the issue that needs to be resolved first, right? You can have all the compatible libraries/frameworks in the world fully supported, but we're sitting here ONLY able to connect to environments with CUDA 11.2 or 11.1.

I upgraded to the Colab Pro + plan because your sales team convinced me with phrases involving terminology such as, "priority access", and I ended up wasting 2 complete days installing, reinstalling, uninstalling, on an environment that you arbitrarily either make me restart anyway or you do it yourself. It's a joke. For proper Pytorch integration as of today, we need either 11.3 or 11.6 on these virtuals, available like YESTERDAY! I'll be going back to Microsoft Azure until this is resolved. At least if I spend a day getting the environment set up, I can actually use it...not have it vanish seconds later.

[robotaiguy]

Actually, I'll give you a free business plan. Provide us with the option of putting the notebook in a "Environment Setup" state, that comes with some persistence, just enough to cover setting up environments that require a reboot. I would pay for that today.