Apple Clang 16 build issue, requires ld_classic

[adamjstewart]

🐛 Describe the bug

When building from source with the new Apple Clang 16, I see the following error:

  ld: multiple errors: duplicate LC_RPATH '/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/cpuinfo-2023-11-04-4sylnhqsvndbdpgwvv3mydc6xhudikqh/lib' in '/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/py-torch-2.4.1-wsy5qo376pu3gqk6sazvaff3wf6uo6fl/lib/python3.11/site-packages/torch/lib/libc10.dylib'; duplicate LC_RPATH '/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/protobuf-3.13.0-sqetzfmrwohnuex4orfxr5loztdqj6rq/lib' in '/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/py-torch-2.4.1-wsy5qo376pu3gqk6sazvaff3wf6uo6fl/lib/python3.11/site-packages/torch/lib/libtorch.dylib'; duplicate LC_RPATH '/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/protobuf-3.13.0-sqetzfmrwohnuex4orfxr5loztdqj6rq/lib' in '/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/py-torch-2.4.1-wsy5qo376pu3gqk6sazvaff3wf6uo6fl/lib/python3.11/site-packages/torch/lib/libtorch_cpu.dylib'; duplicate LC_RPATH '/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/protobuf-3.13.0-sqetzfmrwohnuex4orfxr5loztdqj6rq/lib' in '/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/py-torch-2.4.1-wsy5qo376pu3gqk6sazvaff3wf6uo6fl/lib/python3.11/site-packages/torch/lib/libtorch_python.dylib'
  clang++: error: linker command failed with exit code 1 (use -v to see invocation)

Current workaround is to inject -Wl,-ld_classic into the linker, but would be good to fix this as ld_classic is deprecated and will be removed in the future.

Versions

• torchvision: 0.19.1
• clang: 16.0.0

[adamjstewart]

We implemented a compiler wrapper in Spack that removes duplicate RPATHs, but we're still seeing the following issue:

ld: duplicate LC_RPATH '@loader_path' in '/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/py-torch-2.4.1-hlpsikrtvukd4ww7jyghncl63ux7f2pl/lib/python3.11/site-packages/torch/lib/libtorch_cpu.dylib'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/Users/Adam/spack/lib/spack/env/clang/clang++' failed with exit code 1

It's possible the bug is in PyTorch itself, but I'm not familiar enough with CMake to report it.