crash in absl::~Flag when used in python along with PyTorch

[mandeeplearning]

FLAGS_version and FLAGS_minloglevel are exported in the sentencepiece shared library and can be interposed with the same named symbols from other shared libraries such as pytorch's c10. When using sentencepiece from python along with pytorch, this causes a crash in the Flag destructor due to the sentencepiece Flag object being larger than the standard absl Flag object.

For example: #905 #560 #579

One possible fix is to use -Bsymbolic

diff --git a/python/setup.py b/python/setup.py
index fdf9394..9b8cb42 100755
--- a/python/setup.py
+++ b/python/setup.py
@@ -96,6 +96,8 @@ class build_ext(_build_ext):
     else:
       cflags.append('-Wl,-strip-all')
       libs.append('-Wl,-strip-all')
+    if sys.plaform == 'linux:
+      libs.append('-Wl,-Bsymbolic')
     print('## cflags={}'.format(' '.join(cflags)))
     print('## libs={}'.format(' '.join(libs)))
     ext.extra_compile_args = cflags

Reference: https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic

[h-vetinari]

The abseil situation is IMO quite problematic. See also #869 and #876. I have a branch that fixes some of this (based on 0.1.99, not master), at least for use with an external abseil (which is the basis of the packaging in conda-forge). I've been trying to get some guidance on how and in what form I could contribute back some of those changes, but so far no response in #876 😑

[taku910]

Fixed in v0.2.0