RTC models do not compile for unknown future CUDA Architectures

[ptheywood]

RTC models are compiled for the device's compute capability, i.e when running on a consumer Ampere GPU, nvrtc is passed --gpu-architecture=compute_86.

However, if the version of NVRTC does not know about the GPU architecture this will fail to compile, and the user can do nothing about this (other than use a more recent NVRTC)

This means that RTC models will not run on newer GPUs (without using newer features), unlike non-RTC models which will (via PTX embedding / JITing).

To reproduce this, CUDA 11.0 knows SM_80 but not SM_86, so attempting to run a CUDA 11.0 RTC model on consume ampere will fail RTC compialtion, with an error during RTC compilation such as:

Compiler options: --gpu-architecture=compute_86 --generate-line-info -DNDEBUG --std=c++17 --define-macro=SEATBELTS=0 --pre-include=/usr/local/cuda-11.0/include//cuda.h 

[ptheywood]

The fix for this is to make use of nvrtcGetNumSupportedArchs and nvrtcGetSupportedArchs (docs) to find the arch's supported by the current nvrtc, and only pass the device's specific arch if it is in the list of supported arch's.

If it is not in the list, passing the latest arch that is supported should work (i.e. the last value returned by nvrtcGetSupportedArchs).

[ptheywood]

nvrtcGetNumSupportedArchs and nvrtcGetSupportedArchs were introduced in CUDA 11.0, so are not available in (the deprecated but not yet removed) CUDA 10.x.

In this case we have no idea about what CUDA arch's would work (other than the minimum configured at cmake time) so the only safe thing to do is remove setting the gencode if CUDA < 11.0.

As this is deprecated and can be removed at any time now, that's the easier option than worrying about a workaround.

Edit:
nvrtcGetNumSupportedArchs and nvrtcGetSupportedArchs were introduced in CUDA 11.2, so not available in CUDA 11.1 and older.

Support for these older CUDA versions could be:

• Don't set the gencode if we can't query if it exists
• Try an nvrtc compilation with the current gencode, if it fails, don't set a gencode
• Hardcode the earliest and latest supported version based on the CUDA version macro(s) and / or the nvrtc version.