Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build with CUDA 12 #117

Merged
merged 47 commits into from
May 31, 2023
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
d16d621
MNT: Re-rendered with conda-build 3.24.0, conda-smithy 3.23.1, and co…
pentschev Apr 12, 2023
1ca8c69
MNT: Re-rendered with nvidia channel
pentschev Apr 12, 2023
42a9a26
Update from `$CUDA_HOME` to `$PREFIX`
pentschev Apr 12, 2023
89adb8c
Add CUDA packages to `host` section
pentschev Apr 12, 2023
8498339
Add `cuda-nvcc` and `cuda-driver-dev`
pentschev Apr 12, 2023
61f1bba
Merge remote-tracking branch 'upstream/main' into cuda-12-nvidia-channel
pentschev Apr 12, 2023
a77f07b
Add `cuda-nvml-devel`
pentschev Apr 12, 2023
e59fac3
Fix `cuda-nvml-dev` package name
pentschev Apr 12, 2023
dc60ad1
Add CUDA 12 UCX patch
pentschev Apr 12, 2023
76f3b53
Enable upload of artifacts to Azure
pentschev Apr 12, 2023
cb2e44b
Add aarch64 and ppc64le builds
pentschev Apr 12, 2023
0e8cd96
Update skip build condition
pentschev Apr 12, 2023
8a32084
Drop nvidia channel
pentschev Apr 18, 2023
86dc036
MNT: Re-rendered with conda-build 3.24.0, conda-smithy 3.23.1, and co…
pentschev Apr 18, 2023
9408e60
Set `CFLAGS` with missing includes/libraries
pentschev Apr 20, 2023
3bb8cf8
Add C/C++ compilers to `host` section
pentschev Apr 20, 2023
3534d6e
Add CUDA < 12 build support back
pentschev Apr 20, 2023
c312d19
MNT: Re-rendered with conda-build 3.24.0, conda-smithy 3.23.1, and co…
pentschev Apr 21, 2023
25a94e4
Fix linting
pentschev Apr 21, 2023
83feef7
Do not skip CUDA != 12.0 builds
pentschev Apr 21, 2023
f1c292a
MNT: Re-rendered with conda-build 3.24.0, conda-smithy 3.23.1, and co…
pentschev Apr 21, 2023
fa5d9dd
Fix mixed CUDA version builds
pentschev Apr 21, 2023
b9fb742
Fix install script conditional
pentschev Apr 21, 2023
f5f672d
Fixed docker image for CUDA 10.2 build
pentschev Apr 21, 2023
b23950b
MNT: Re-rendered with conda-build 3.24.0, conda-smithy 3.23.1, and co…
pentschev Apr 21, 2023
ffa1144
Fix meta.yaml linting
pentschev Apr 21, 2023
704da6a
Drop conda_build_config changes
pentschev May 3, 2023
e8b3344
Add CUDA 12 migrators
pentschev May 3, 2023
87b55ab
MNT: Re-rendered with conda-build 3.24.0, conda-smithy 3.23.1, and co…
pentschev May 3, 2023
5f31b59
Merge https://github.com/conda-forge/ucx-split-feedstock into cuda-12…
pentschev May 3, 2023
ce28170
Skip non-CUDA builds
pentschev May 3, 2023
c085338
MNT: Re-rendered with conda-build 3.24.0, conda-smithy 3.23.1, and co…
pentschev May 3, 2023
809874c
Drop CUDA selector
pentschev May 3, 2023
9eb17e1
try simplified migrator
leofang May 9, 2023
5719cf2
Merge remote-tracking branch 'upstream/main' into cuda-12-nvidia-channel
pentschev May 22, 2023
cf2168d
Merge remote-tracking branch 'origin/cuda-12-nvidia-channel' into cud…
pentschev May 22, 2023
711d222
Merge remote-tracking branch 'upstream/main' into cuda-12-nvidia-channel
pentschev May 25, 2023
e9a6f07
Remove CUDA packages that are not needed anymore
pentschev May 25, 2023
85372ce
MNT: Re-rendered with conda-build 3.23.3, conda-smithy 3.23.1, and co…
pentschev May 25, 2023
11412c5
Fix paths
pentschev May 26, 2023
de5a5a5
Remove unnecessary compiler/cuda-driver-dev packages
pentschev May 26, 2023
fa80326
Add `run_constrained` section back
pentschev May 26, 2023
212af63
Use `cuda-version` and remove redundant `target_platform`
pentschev May 30, 2023
7009fc0
Add missing `cuda-cudart-dev` to `host` section
pentschev May 30, 2023
b07ce79
Remove CUDA 12 migrator files
pentschev May 31, 2023
3ea1685
Add `cuda120.yaml` from `conda-forge-pinning`
jakirkham May 31, 2023
a5d6b45
MNT: Re-rendered with conda-build 3.25.0, conda-smithy 3.23.1, and co…
May 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .azure-pipelines/azure-pipelines-linux.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
c_compiler:
- gcc
c_compiler_version:
- '12'
cdt_name:
- cos7
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cuda_compiler:
- cuda-nvcc
cuda_compiler_version:
- '12.0'
cxx_compiler:
- gxx
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-cos7-x86_64
rdma_core:
- '28'
target_platform:
- linux-64
ucx:
- 1.14.0
zip_keys:
- - c_compiler_version
- cxx_compiler_version
- cuda_compiler
- cuda_compiler_version
- cdt_name
- docker_image
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
BUILD:
- aarch64-conda_cos7-linux-gnu
c_compiler:
- gcc
c_compiler_version:
- '10'
cdt_arch:
- aarch64
cdt_name:
- cos7
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cuda_compiler:
- cuda-nvcc
cuda_compiler_version:
- '12.0'
cxx_compiler:
- gxx
cxx_compiler_version:
- '10'
docker_image:
- quay.io/condaforge/linux-anvil-aarch64
target_platform:
- linux-aarch64
ucx:
- 1.14.0
zip_keys:
- - c_compiler_version
- cxx_compiler_version
- cuda_compiler
- cuda_compiler_version
- cdt_name
- docker_image
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
c_compiler:
- gcc
c_compiler_version:
- '10'
cdt_name:
- cos7
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cuda_compiler:
- cuda-nvcc
cuda_compiler_version:
- '12.0'
cxx_compiler:
- gxx
cxx_compiler_version:
- '10'
docker_image:
- quay.io/condaforge/linux-anvil-ppc64le
target_platform:
- linux-ppc64le
ucx:
- 1.14.0
zip_keys:
- - c_compiler_version
- cxx_compiler_version
- cuda_compiler
- cuda_compiler_version
- cdt_name
- docker_image
39 changes: 39 additions & 0 deletions .ci_support/migrations/cuda120.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
migrator_ts: 1682985063
__migrator:
kind:
version
use_local: true # TODO: Remove this line after completing testing
migration_number:
1
build_number:
1
paused: false
override_cbc_keys:
- cuda_compiler_stub
operation: key_add
primary_key: cuda_compiler_version
commit_message: "Rebuild for CUDA 12"

cuda_compiler: # [linux64 or win]
- cuda-nvcc # [linux64 or win]

cuda_compiler_version: # [linux64 or win]
- 12.0 # [linux64 or win]

c_compiler_version: # [linux64]
- 12 # [linux64]

cxx_compiler_version: # [linux64]
- 12 # [linux64]

fortran_compiler_version: # [linux64]
- 12 # [linux64]

cudnn: # [linux64 or win]
- 8 # [linux64 or win]

cdt_name: # [linux64]
- cos7 # [linux64]

docker_image: # [os.environ.get("BUILD_PLATFORM", "").startswith("linux")]
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
48 changes: 48 additions & 0 deletions .ci_support/migrations/cuda120_ppc64le_aarch64.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
migrator_ts: 1683090168
__migrator:
kind:
version
use_local: true # TODO: Remove this line after completing testing
migration_number:
1
build_number:
1
paused: false
override_cbc_keys:
- cuda_compiler_stub
operation: key_add
check_solvable: false
primary_key: cuda_compiler_version
wait_for_migrators:
- aarch64 and ppc64le addition
exclude_pinned_pkgs: False
commit_message: "Rebuild for cuda 12 on ppc64le and aarch64"

arm_variant_type: # [aarch64]
- sbsa # [aarch64]

c_compiler_version: # [ppc64le or aarch64]
- 10 # [ppc64le or aarch64]
cxx_compiler_version: # [ppc64le or aarch64]
- 10 # [ppc64le or aarch64]
fortran_compiler_version: # [ppc64le or aarch64]
- 10 # [ppc64le or aarch64]

cuda_compiler: # [ppc64le or aarch64]
- cuda-nvcc # [ppc64le or aarch64]
cuda_compiler_version: # [ppc64le or aarch64]
- 12.0 # [ppc64le or aarch64]

cudnn: # [ppc64le or aarch64]
- 8 # [ppc64le or aarch64]

cdt_name: # [ppc64le or aarch64]
- cos7 # [ppc64le or aarch64]

docker_image: # [os.environ.get("BUILD_PLATFORM", "").startswith("linux") and (ppc64le or aarch64)]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-aarch64 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-comp7 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-comp7 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
10 changes: 10 additions & 0 deletions .travis.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 21 additions & 0 deletions README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 11 additions & 2 deletions recipe/install_ucx.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,17 @@
set -xeuo pipefail

EXTRA_ARGS=""
if [ "${cuda_compiler_version}" != "None" ]; then
EXTRA_ARGS="${EXTRA_ARGS} --with-cuda=${CUDA_HOME}"

if [[ "${cuda_compiler_version}" =~ 12.* ]]; then
EXTRA_ARGS="${EXTRA_ARGS} --with-cuda=${PREFIX}"

[[ ${target_platform} == "linux-64" ]] && targetsDir="targets/x86_64-linux"
[[ ${target_platform} == "linux-ppc64le" ]] && targetsDir="targets/ppc64le-linux"
[[ ${target_platform} == "linux-aarch64" ]] && targetsDir="targets/sbsa-linux"

export CFLAGS="${CFLAGS} -I${BUILD_PREFIX}/${targetsDir}/include -L${BUILD_PREFIX}/${targetsDir}/lib/stubs"
pentschev marked this conversation as resolved.
Show resolved Hide resolved
elif [[ "${cuda_compiler_version}" != "None" ]]; then
EXTRA_ARGS="${EXTRA_ARGS} --with-cuda=${CUDA_HOME}"
fi

if [ "${cuda_compiler_version}" != "10.2" ] && [ "${target_platform}" == "linux-64" ]; then
Expand Down
9 changes: 6 additions & 3 deletions recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% set ucx_version = "1.14.1" %}
{% set number = "1" %}
{% set number = "2" %}


package:
Expand All @@ -26,21 +26,24 @@ outputs:
build:
- {{ compiler("c") }}
- {{ compiler("cxx") }}
- {{ compiler("cuda") }} # [cuda_compiler_version != "None"]
- {{ compiler("cuda") }}
- {{ cdt("libnl") }} # [cdt_name == "cos6"]
- {{ cdt("libnl3") }} # [cdt_name == "cos7"]
- cuda-nvml-dev ={{ cuda_compiler_version }} # [cuda_compiler_version == "12.0"]
pentschev marked this conversation as resolved.
Show resolved Hide resolved
- automake
- autoconf
- libtool
- make
- pkg-config
host:
- cuda-nvml-dev ={{ cuda_compiler_version }} # [cuda_compiler_version == "12.0"]
pentschev marked this conversation as resolved.
Show resolved Hide resolved
- libnuma
- rdma-core # [linux64 and cuda_compiler_version != "10.2"]
run:
run_constrained:
pentschev marked this conversation as resolved.
Show resolved Hide resolved
- cudatoolkit {{ cuda_compiler_version }}|{{ cuda_compiler_version }}.* # [cuda_compiler_version not in ("None", "11.2")]
- cudatoolkit {{ cuda_compiler_version }}|{{ cuda_compiler_version }}.* # [cuda_compiler_version not in ("None", "11.2", "12.0")]
- cudatoolkit >=11.2,<12 # [cuda_compiler_version == "11.2"]
pentschev marked this conversation as resolved.
Show resolved Hide resolved

script: install_ucx.sh
test:
commands:
Expand Down