native_test and native_binary break shared libraries with remote execution

[cameron-martin]

In many implementation of remote execution services, the symlink that is created to the executable in native_test is not actually a symlink, and is effectively a copy. For resolving dynamic libraries, bazel uses rpaths relative to $ORIGIN. $ORIGIN refers to the resolved location of the symlink, and so these relative paths work fine when the executable is a symlink. However, when this file is actually a copy these relative rpaths no longer work. Note that the same thing happens when using the experimental hermetic linux sandbox, since this uses hardlinks and copying instead of symlinks.

I have created a reproducer, using the hermetic linux sandbox.

[cameron-martin]

Related discussion: https://bazelbuild.slack.com/archives/CA31HN1T3/p1679661215222949

[tetromino]

Thanks, the slack discussion + linked rules_python issue (bazelbuild/rules_python#691) are very informative. Not sure how to solve this, but I'd classify this as a bug in bazel rather than skylib.

One solution might be along the lines of falling back to a wrapper script if ctx.actions.symlink is a copy and not a symlink - but currently Bazel does not tell us whether that is the case.

Best we can do here for now is update our docs to caution against use of native_test / native_binary ...

[cameron-martin]

One of the bazelcon talks this year was about a cross-platform native launcher, kickoff, which is a potential solution to this.