Modifying data_parallel_tutorial.py to enable multiple GPU support

[prithviraj-maurya]

Fixes #2563

Description

Modifying data_parallel_tutorial.py, by removing the specification "cuda:0," you enable the use of multiple GPUs. When you specify the index "0," it restricts the computation to only the GPU at index 0. However, if your system has multiple GPUs available, PyTorch will automatically distribute the computation across all available GPUs, resulting in faster training for deep learning tasks. This flexibility allows you to take full advantage of your GPU resources and potentially reduce training times when working with multiple GPUs.

Checklist

• The issue that is being fixed is referred in the description (see above "Fixes #ISSUE_NUMBER")
• Only one issue is addressed in this pull request
• Labels from the issue that this PR is fixing are added to this pull request
• No unnecessary issues are included into this pull request.

cc @sekyondaMeta @svekars @carljparker @NicolasHug @kit1980 @subramen

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/2652

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit e786bd3 with merge base 77aec05 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[malfet]

Can you please modify the PR title to clearly describe what are you trying to achieve there?

[prithviraj-maurya]

@malfet I have updated the title to reflect the task, do let me know if that looks good?

[malfet]

@prithviraj-maurya thank you for the update. In that case, do you mind modifying this line as well:

tutorials/beginner_source/blitz/data_parallel_tutorial.py

Line 12 in e7563f6

device = torch.device("cuda:0")

[prithviraj-maurya]

@malfet I have updated the description of PR.

[malfet]

When I run original code of the tutorial on multi-gpu system, I see it's already allocating memory on multiple GPU, so I'm not sure where the notion that replacing "cuda:0", with "cuda" would have any effect is coming from

[prithviraj-maurya]

@malfet Ah, I see that now. Do you think this change might not be needed then? The original issue talked about changes required on that code.

[svekars]

We need to close this as we found out that the changes are not needed per @malfet. Will grant half credit for the issue.