ABI incompatibility: shared library linked to libstdc++, but program linking to libc++

[lahwaacz]

I've built Ginkgo 1.7.0 with GCC + libstdc++ and installed it as a system library. But then when I try to build a project using Ginkgo with Clang + LLVM's libc++ instead of GCC's libstdc++, I get a linker error:

FAILED: bin/tnl-ginkgo-converted-ellpack-dbg 
200: && /usr/sbin/clang++ -stdlib=libc++ -Wall -Werror=vla -Wextra-semi -Wextra-semi-stmt -Werror -Wno-error=deprecated -Wno-error=deprecated-declarations -DHAVE_DCMTK_H -DHAVE_PNG_H -DHAVE_JPEG_H -g -fuse-ld=lld -lc++ -lc++abi    -Wl,-rpath -Wl,/usr/lib -Wl,--enable-new-dtags src/Examples/Ginkgo/CMakeFiles/tnl-ginkgo-converted-ellpack.dir/tnl-ginkgo-converted-ellpack.cpp.o -o bin/tnl-ginkgo-converted-ellpack-dbg  /usr/lib/libginkgo.so.1.7.0  /usr/lib64/libomp.so  /usr/lib64/libpthread.a  /usr/lib/libginkgo_omp.so.1.7.0  /usr/lib/libginkgo_cuda.so.1.7.0  -ldl  /usr/lib/libginkgo_reference.so.1.7.0  /usr/lib/libginkgo_hip.so.1.7.0  /usr/lib/libginkgo_dpcpp.so.1.7.0  /usr/lib/libginkgo_device.so.1.7.0  /usr/lib/libhwloc.so  /usr/lib/libhwloc.so  /usr/lib64/libmpi.so && :
201ld.lld: error: undefined symbol: gko::CudaExecutor::create(int, std::__1::shared_ptr<gko::Executor>, bool, gko::allocation_mode, CUstream_st*)
202>>> referenced by tnl-ginkgo-converted-ellpack.cpp:58 (/builds/tnl-project/tnl/src/Examples/Ginkgo/tnl-ginkgo-converted-ellpack.cpp:58)
203>>>               src/Examples/Ginkgo/CMakeFiles/tnl-ginkgo-converted-ellpack.dir/tnl-ginkgo-converted-ellpack.cpp.o:(main)
204ld.lld: error: undefined symbol: gko::HipExecutor::create(int, std::__1::shared_ptr<gko::Executor>, bool, gko::allocation_mode, CUstream_st*)
205>>> referenced by tnl-ginkgo-converted-ellpack.cpp:61 (/builds/tnl-project/tnl/src/Examples/Ginkgo/tnl-ginkgo-converted-ellpack.cpp:61)
206>>>               src/Examples/Ginkgo/CMakeFiles/tnl-ginkgo-converted-ellpack.dir/tnl-ginkgo-converted-ellpack.cpp.o:(main)
207ld.lld: error: undefined symbol: gko::DpcppExecutor::create(int, std::__1::shared_ptr<gko::Executor>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, dpcpp_queue_property)
208>>> referenced by tnl-ginkgo-converted-ellpack.cpp:63 (/builds/tnl-project/tnl/src/Examples/Ginkgo/tnl-ginkgo-converted-ellpack.cpp:63)
209>>>               src/Examples/Ginkgo/CMakeFiles/tnl-ginkgo-converted-ellpack.dir/tnl-ginkgo-converted-ellpack.cpp.o:(main)
210ld.lld: error: undefined symbol: gko::stop::ResidualNormBase<double>::ResidualNormBase(std::__1::shared_ptr<gko::Executor const>, gko::stop::CriterionArgs const&, double, gko::stop::mode)
211>>> referenced by residual_norm.hpp:167 (/usr/include/ginkgo/core/stop/residual_norm.hpp:167)
212>>>               src/Examples/Ginkgo/CMakeFiles/tnl-ginkgo-converted-ellpack.dir/tnl-ginkgo-converted-ellpack.cpp.o:(gko::stop::ResidualNorm<double>::ResidualNorm(gko::stop::ResidualNorm<double>::Factory const*, gko::stop::CriterionArgs const&))
213ld.lld: error: undefined symbol: gko::solver::workspace_traits<gko::solver::Cg<double>>::op_names(gko::solver::Cg<double> const&)
214>>> referenced by solver_base.hpp:633 (/usr/include/ginkgo/core/solver/solver_base.hpp:633)
215>>>               src/Examples/Ginkgo/CMakeFiles/tnl-ginkgo-converted-ellpack.dir/tnl-ginkgo-converted-ellpack.cpp.o:(gko::solver::EnableSolverBase<gko::solver::Cg<double>, gko::LinOp>::get_workspace_op_names() const)
216ld.lld: error: undefined symbol: gko::stop::Combined::Factory::Factory(std::__1::shared_ptr<gko::Executor const>, gko::stop::Combined::parameters_type const&)
217>>> referenced by abstract_factory.hpp:267 (/usr/include/ginkgo/core/base/abstract_factory.hpp:267)
218>>>               src/Examples/Ginkgo/CMakeFiles/tnl-ginkgo-converted-ellpack.dir/tnl-ginkgo-converted-ellpack.cpp.o:(gko::enable_parameters_type<gko::stop::Combined::parameters_type, gko::stop::Combined::Factory>::on(std::__1::shared_ptr<gko::Executor const>) const)
219clang++: error: linker command failed with exit code 1 (use -v to see invocation)

So Ginkgo does not have binary compatibility for the shared library linked to libstdc++ and downstream executable linking to libc++. Would it be hard to support this case? Can you give some hints how to detect this case with cmake if it can't be supported?

[upsj]

Achieving binary compatibility with different stdlib implementations is hard to impossible if we use standard library types across the ABI boundary. We might be able to achieve this in a future major version, but it would require major restructuring.

So for now, I think this is not something we can support. I would imagine that linking against both libstdc++ and libc++ is also dangerous in terms of ODR violations, so I would not recommend it.

[upsj]

Can you give some hints how to detect this case with cmake if it can't be supported?

CMake still doesn't have native support for specifying stdlib implementations, so I don't really see a way to do this properly, unless you want to parse compiler flags for the standard library