Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new NVIDIA CUDA feature #80

Merged
merged 38 commits into from
Aug 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
caa4520
Add new Nvidia feature
jungaretti Aug 4, 2022
2e115a9
Remove random empty files
jungaretti Aug 4, 2022
82cf979
Update comments
jungaretti Aug 4, 2022
25a7dc7
Merge branch 'main' into jungaretti/nvidia-feature
joshspicer Aug 5, 2022
0d6fe41
Rename feature to nvidia-cuda
jungaretti Aug 8, 2022
f0dee9d
Add feature to tests
jungaretti Aug 8, 2022
c729aef
Add version
jungaretti Aug 8, 2022
ce560c7
Merge branch 'main' of https://github.com/devcontainers/features into…
jungaretti Aug 8, 2022
f329740
Move test to match new name
jungaretti Aug 8, 2022
ccc5282
Add final output message
jungaretti Aug 8, 2022
2a7d079
Fix capitalization of NVIDIA
jungaretti Aug 8, 2022
9aef242
Remove option for base CUDA
jungaretti Aug 11, 2022
67309af
Use camelCase
jungaretti Aug 11, 2022
8a44491
Check for required packages
jungaretti Aug 15, 2022
30a4aa6
Use os-release instead of lsb_release
jungaretti Aug 15, 2022
90afb4a
Clean up keyring variables
jungaretti Aug 15, 2022
ba1da2a
Collapse keyring lines
jungaretti Aug 15, 2022
62bceba
Always install CUDA libraries
jungaretti Aug 15, 2022
96423df
Add option to install NVTX
jungaretti Aug 15, 2022
caf879a
Merge branch 'main' of https://github.com/devcontainers/features into…
jungaretti Aug 16, 2022
3ab6580
Always use ubuntu2004 repo
jungaretti Aug 16, 2022
c1a051d
Use test instead of brackets
jungaretti Aug 17, 2022
8a6d52b
Add default values to feature
jungaretti Aug 17, 2022
f246115
Add version options for CUDA and cuDNN
jungaretti Aug 17, 2022
52da1e5
Rename CUDA version option
jungaretti Aug 17, 2022
d3e145b
Add scenario to test specific CUDA/cuDNN version
jungaretti Aug 17, 2022
287aa65
Rename cuDNN scenario
jungaretti Aug 17, 2022
ef45b35
Fix typo in test scenario
jungaretti Aug 17, 2022
e6b9356
Update variable casing
jungaretti Aug 17, 2022
11d217a
Add more helpful error messages
jungaretti Aug 18, 2022
a89dac0
Remove default values from script
jungaretti Aug 19, 2022
d17069e
Use enum for version option
jungaretti Aug 20, 2022
9e6255c
Polish new scenarios
jungaretti Aug 20, 2022
c7a61d3
Remove apt_get_update_if_needed and check_packages
jungaretti Aug 20, 2022
f4b62d2
Merge branch 'main' of https://github.com/devcontainers/features into…
jungaretti Aug 20, 2022
ae65577
Add more versions
jungaretti Aug 20, 2022
7d57487
Improve error messages
jungaretti Aug 20, 2022
fe2bada
Comments and feature description
jungaretti Aug 21, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/test-all.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ jobs:
"java",
"kubectl-helm-minikube",
"node",
"nvidia-cuda",
"oryx",
"php",
"powershell",
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ jobs:
java: ./**/java/**
kubectl-helm-minikube: ./**/kubectl-helm-minikube/**
node: ./**/node/**
nvidia-cuda: ./**/nvidia-cuda/**
oryx: ./**/oryx/**
php: ./**/php/**
powershell: ./**/powershell/**
Expand Down
51 changes: 51 additions & 0 deletions src/nvidia-cuda/devcontainer-feature.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
{
"id": "nvidia-cuda",
"name": "NVIDIA CUDA",
"description": "Installs shared libraries for NVIDIA CUDA.",
"version": "1.0.0",
"options": {
"installCudnn": {
"type": "boolean",
"default": false,
"description": "Additionally install CUDA Deep Neural Network (cuDNN) shared library"
},
"installNvtx": {
"type": "boolean",
"default": false,
"description": "Additionally install NVIDIA Tools Extension (NVTX)"
},
"cudaVersion": {
"type": "string",
"enum": [
"11.7",
"11.6",
"11.5",
"11.4",
"11.3",
"11.2"
],
"default": "11.7",
"description": "Version of CUDA to install"
},
"cudnnVersion": {
"type": "string",
"enum": [
"8.5.0.96",
"8.4.1.50",
"8.4.0.27",
"8.3.3.40",
"8.3.2.44",
"8.3.1.22",
"8.3.0.98",
"8.2.4.15",
"8.2.2.26",
"8.2.1.32",
"8.2.0.53",
"8.1.1.33",
"8.1.0.77"
],
"default": "8.5.0.96",
"description": "Version of cuDNN to install"
}
}
}
58 changes: 58 additions & 0 deletions src/nvidia-cuda/install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/usr/bin/env bash

set -e

INSTALL_CUDNN=${INSTALLCUDNN}
INSTALL_NVTX=${INSTALLNVTX}
CUDA_VERSION=${CUDAVERSION}
CUDNN_VERSION=${CUDNNVERSION}

if [ "$(id -u)" -ne 0 ]; then
echo -e 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
exit 1
fi

jungaretti marked this conversation as resolved.
Show resolved Hide resolved
# Install dependencies
apt-get update -yq
apt-get install -yq wget ca-certificates

# Add NVIDIA's package repository to apt so that we can download packages
# Always use the ubuntu2004 repo because the other repos (e.g., debian11) are missing packages
NVIDIA_REPO_URL="https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64"
KEYRING_PACKAGE="cuda-keyring_1.0-1_all.deb"
KEYRING_PACKAGE_URL="$NVIDIA_REPO_URL/$KEYRING_PACKAGE"
KEYRING_PACKAGE_PATH="$(mktemp -d)"
KEYRING_PACKAGE_FILE="$KEYRING_PACKAGE_PATH/$KEYRING_PACKAGE"
wget -O "$KEYRING_PACKAGE_FILE" "$KEYRING_PACKAGE_URL"
apt-get install -yq "$KEYRING_PACKAGE_FILE"
apt-get update -yq

# Ensure that the requested version of CUDA is available
cuda_pkg="cuda-libraries-${CUDA_VERSION/./-}"
nvtx_pkg="cuda-nvtx-${CUDA_VERSION/./-}"
if ! apt-cache show "$cuda_pkg"; then
echo "The requested version of CUDA is not available: CUDA $CUDA_VERSION"
exit 1
fi

# Ensure that the requested version of cuDNN is available AND compatible
cudnn_pkg_version="libcudnn8=${CUDNN_VERSION}-1+cuda${CUDA_VERSION}"
if ! apt-cache show "$cudnn_pkg_version"; then
echo "The requested version of cuDNN is not available: cuDNN $CUDNN_VERSION for CUDA $CUDA_VERSION"
exit 1
fi

echo "Installing CUDA libraries..."
apt-get install -yq "$cuda_pkg"

if [ "$INSTALL_CUDNN" = "true" ]; then
echo "Installing cuDNN libraries..."
apt-get install -yq "$cudnn_pkg_version"
fi

if [ "$INSTALL_NVTX" = "true" ]; then
echo "Installing NVTX..."
apt-get install -yq "$nvtx_pkg"
fi

echo "Done!"
15 changes: 15 additions & 0 deletions test-scenarios/install_cudnn_nvxt.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash

set -e

# Optional: Import test library
source dev-container-features-test-lib

# Check installation of libcudnn8
check "libcudnn.so.8" test 1 -eq "$(find /usr -name 'libcudnn.so.8' | wc -l)"

# Check installation of cuda-nvtx-11-<version>
check "cuda-11+nvtx" test -e '/usr/local/cuda-11/targets/x86_64-linux/include/nvtx3'

# Report result
reportResults
16 changes: 16 additions & 0 deletions test-scenarios/install_cudnn_nvxt_version.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash

set -e

# Optional: Import test library
source dev-container-features-test-lib

# Check installation of libcudnn8 (8.3.2)
check "libcudnn.so.8.3.2" test 1 -eq "$(find /usr -name 'libcudnn.so.8.3.2' | wc -l)"

# Check installation of cuda-nvtx-11-5 (11.5)
check "cuda-11-5+nvtx" test -e '/usr/local/cuda-11.5/targets/x86_64-linux/include/nvtx3'

# Report result
reportResults

20 changes: 20 additions & 0 deletions test-scenarios/scenarios.json
Original file line number Diff line number Diff line change
Expand Up @@ -91,5 +91,25 @@
"version": "3"
}
}
},
"install_cudnn_nvxt": {
"image": "debian",
"features": {
"nvidia-cuda": {
"installCudnn": true,
"installNvtx": true
}
}
},
"install_cudnn_nvxt_version": {
"image": "debian",
"features": {
"nvidia-cuda": {
"installCudnn": true,
"installNvtx": true,
"cudaVersion": "11.5",
"cudnnVersion": "8.3.2.44"
}
}
}
}
18 changes: 18 additions & 0 deletions test/nvidia-cuda/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/bash

set -e

# Optional: Import test library
source dev-container-features-test-lib

# Check installation of cuda-libraries-11-<version>
check "libcudart.so.11.0" test 1 -eq "$(find /usr -name 'libcudart.so.11.0' | wc -l)"
check "libcublas.so.11" test 1 -eq "$(find /usr -name 'libcublas.so.11' | wc -l)"
check "libcublasLt.so.11" test 1 -eq "$(find /usr -name 'libcublasLt.so.11' | wc -l)"
check "libcufft.so.10" test 1 -eq "$(find /usr -name 'libcufft.so.10' | wc -l)"
check "libcurand.so.10" test 1 -eq "$(find /usr -name 'libcurand.so.10' | wc -l)"
check "libcusolver.so.11" test 1 -eq "$(find /usr -name 'libcusolver.so.11' | wc -l)"
check "libcusparse.so.11" test 1 -eq "$(find /usr -name 'libcusparse.so.11' | wc -l)"

# Report result
reportResults