Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cannot use syntaxdot for symbol lookup error #220

Open
tondach01 opened this issue Sep 27, 2022 · 11 comments
Open

cannot use syntaxdot for symbol lookup error #220

tondach01 opened this issue Sep 27, 2022 · 11 comments

Comments

@tondach01
Copy link

Hello, I installed Syntaxdot as it was described in documentation, no error occurred, but when I tried to execute the syntaxdot binary, I got this error:

syntaxdot: symbol lookup error: syntaxdot: undefined symbol: _ZN2at3mulERKNS_6TensorERKN3c106ScalarE

I am using Ubuntu 22.04 on VirtualBox
Thanks for your help, Ondra
@danieldk
Copy link
Member

Could you give some more information, such as which libtorch archive you downloaded? Usually such errors come from downloading libtorch with the wrong C++ ABI.

(I should really provide pre-built CUDA binaries in the future.)

@tondach01
Copy link
Author

I downloaded libtorch from
https://download.pytorch.org/libtorch/cu116/libtorch-cxx11-abi-shared-with-deps-1.12.1%2Bcu116.zip
Stable (1.12.1) > Linux > LIbtorch > C++/Java > CUDA 11.6 > the second link (cxx11 ABI)

CUDA 11.7, rustup 1.24.3 (rustc 1.64.0), cmake 3.22.1, pkg-config 0.29.2, OpenSSL 3.0.2 15 Mar 2022, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0

@danieldk
Copy link
Member

danieldk commented Oct 1, 2022

Thanks! I’ll try and see if I can reproduce this.

@danieldk
Copy link
Member

danieldk commented Oct 8, 2022

That's really odd, I installed Ubuntu 22.04 using distrobox and downloaded the same libtorch version as you did and it works fine. Could you show the ldd output for the syntaxdot binary. Here is mine:

% ldd target/release/syntaxdot
	linux-vdso.so.1 (0x00007ffd9fd67000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8d744ff000)
	libtorch_cuda_cu.so => /home/daniel/syntaxdot-issue/libtorch/lib/libtorch_cuda_cu.so (0x00007f8d4be00000)
	libtorch_cuda_cpp.so => /home/daniel/syntaxdot-issue/libtorch/lib/libtorch_cuda_cpp.so (0x00007f8d39400000)
	libtorch_cpu.so => /home/daniel/syntaxdot-issue/libtorch/lib/libtorch_cpu.so (0x00007f8d20000000)
	libc10.so => /home/daniel/syntaxdot-issue/libtorch/lib/libc10.so (0x00007f8d1fc00000)
	libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f8d744b3000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8d74493000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8d1ff19000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8d1f9d8000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f8d74ca7000)
	libcudart-45da57e3.so.11.0 => /home/daniel/syntaxdot-issue/libtorch/lib/libcudart-45da57e3.so.11.0 (0x00007f8d1f600000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8d7448e000)
	libc10_cuda.so => /home/daniel/syntaxdot-issue/libtorch/lib/libc10_cuda.so (0x00007f8d1f200000)
	libnvToolsExt-847d78f2.so.1 => /home/daniel/syntaxdot-issue/libtorch/lib/libnvToolsExt-847d78f2.so.1 (0x00007f8d1ee00000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8d74487000)
	libcublas-2854e16e.so.11 => /home/daniel/syntaxdot-issue/libtorch/lib/libcublas-2854e16e.so.11 (0x00007f8d15600000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8d74482000)
	libcudnn.so.8 => /home/daniel/syntaxdot-issue/libtorch/lib/libcudnn.so.8 (0x00007f8d15200000)
	libgomp-52f2fd74.so.1 => /home/daniel/syntaxdot-issue/libtorch/lib/libgomp-52f2fd74.so.1 (0x00007f8d14e00000)
	libcublasLt-b015978e.so.11 => /home/daniel/syntaxdot-issue/libtorch/lib/libcublasLt-b015978e.so.11 (0x00007f8cffc00000)

@tondach01
Copy link
Author

Hi, I ran the 'ldd' and there were some libraries missing e.g 'libtorch_cuda_cu.so'. So I tried to reboot the whole virtual machine, installed all the required programs, but this time I got an error right during compilation. I will sent the output log via e-mail, it is quite long.

@tondach01
Copy link
Author

tondach01 commented Oct 10, 2022 via email

@tondach01
Copy link
Author

Update: I figured out that if LIBTORCH variable is undefined, Syntaxdot completes compilation, but throws this symbol undefined error. If defined, the compilation fails ať torch-sys. I tried all versions of LibTorch available (including the CPU) and the result is always the same. Can this be because of outdated torch-sys used during compilation?

@josefkr
Copy link

josefkr commented Apr 23, 2023

Hi, I was wondering if you had found a solution to the issue regarding the compilation of torch-sys.

I have been trying to install the syntaxdot git code on ubuntu 20.04. I tried both both install types, the one with support for training and the one without training but neither succeeds.

As far as I can tell, I have all the required bits.

  • nvidia-smi shows: | NVIDIA-SMI 515.105.01 Driver Version: 515.105.01 CUDA Version: 11.7
  • Paths
    • LD_LIBRARY_PATH=/usr/local/cuda/bin:/mnt/big/tools/libtorch_abi/lib
    • LIBTORCH=/mnt/big/tools/libtorch_abi
  • I ran apt-get install -y build-essential cmake libssl-dev pkg-config
  • rustup default stable shows
info: using existing install for 'stable-x86_64-unknown-linux-gnu'
info: default toolchain set to 'stable-x86_64-unknown-linux-gnu'

  stable-x86_64-unknown-linux-gnu unchanged - rustc 1.68.2 (9eb3afe9e 2023-03-27)

I tried two setups:

  • Version A where I try to use files from the zip archive libtorch-cxx11-abi-shared-with-deps-2.0.0

  • Version B where I try to use libtorch from the torch python package. This is modeled on the recommendation for macOS ARM64 on the syntaxdot install page (though I am not on a mac).

The error messages with both versions look as shown below.

    Updating crates.io index
  Installing syntaxdot-cli v0.4.1
   Compiling libc v0.2.142
   Compiling proc-macro2 v1.0.56
   Compiling quote v1.0.26

...

  
error: failed to run custom build command for `torch-sys v0.5.0`

Caused by:
  process didn't exit successfully: `/tmp/cargo-installm7Mpjz/release/build/torch-sys-fb561cb5c3453def/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=TORCH_CUDA_VERSION
  cargo:rerun-if-env-changed=LIBTORCH
  cargo:rustc-link-search=native=/mnt/big/tools/libtorch_abi/lib
  cargo:rerun-if-changed=libtch/torch_api.cpp
  cargo:rerun-if-changed=libtch/torch_api.h
  cargo:rerun-if-changed=libtch/torch_api_generated.cpp.h
  cargo:rerun-if-changed=libtch/torch_api_generated.h
  cargo:rerun-if-changed=libtch/stb_image_write.h
  cargo:rerun-if-changed=libtch/stb_image_resize.h
  cargo:rerun-if-changed=libtch/stb_image.h
  cargo:rerun-if-env-changed=LIBTORCH_CXX11_ABI
  TARGET = Some("x86_64-unknown-linux-gnu")
  
  ...
      |                                                                                                     |
  cargo:warning=      |                                                                                                     c10::optional<c10::ArrayRef<long int> >
  cargo:warning=libtch/torch_api.h:16:5: note: in definition of macro ‘PROTECT’
  cargo:warning=   16 |     x \
  cargo:warning=      |     ^
  exit status: 1

  --- stderr


  error occurred: Command "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "/mnt/big/tools/libtorch_abi/include" "-I" "/mnt/big/tools/libtorch_abi/include/torch/csrc/api/include" "-Wl,-rpath=/mnt/big/tools/libtorch_abi/lib" "-std=c++14" "-D_GLIBCXX_USE_CXX11_ABI=1" "-o" "/tmp/cargo-installKy08Tc/release/build/torch-sys-10cb96feddf7928c/out/libtch/torch_api.o" "-c" "libtch/torch_api.cpp" with args "c++" did not execute successfully (status code exit status: 1).

I subsequently also tried to install syntaxdot from crates.io.

I get the same kind of error when doing cargo install --version 0.5.0-beta.2 syntaxdot-cli except that it is torch-sys v0.9.0 that's not being built . When trying to install beta1, it fails for torch-sys v0.8.0.

@danieldk
Copy link
Member

libtorch 2.0.0 was not yet supported until I merged the 2.0.0 support PR a few minutes ago. The issue is that the Torch C++ API (which the binding uses) is not stable, so the Torch binding and SyntaxDot can only work with one particular Torch version.

I am sorry that this was not clear in the documentation. I will update the documentation to mention this explicitly in the documentation. We will also provide a precompiled CUDA build for the next release, so that there is a ready-to-run binary for training as well.

@josefkr
Copy link

josefkr commented Apr 26, 2023

Thank you so much for all the recent work on this. I have now been able to install syntaxdot from crates.io.

If it's ok, I'd like to tack on another question: could you provide a minimal example of how to call syntaxdot from the commandline? I am not quite sure what needs to go in the required conf file; where the files named in conf need to be
located; and what info needs to be in them .

@danieldk
Copy link
Member

danieldk commented Apr 26, 2023 •
edited

If it's ok, I'd like to tack on another question: could you provide a minimal example of how to call syntaxdot from the commandline? I am not quite sure what needs to go in the required conf file; where the files named in conf need to be located; and what info needs to be in them .

I'd recommend you to download one of the existing finetuned models to get an idea of what a configuration could look like, eg.:

https://github.com/tensordot/syntaxdot-models/releases/download/de-ud-2021/de-ud-huge-20210307.tar.gz

The finetuning command is then something like:

syntaxdot prepare syntaxdot.conf train.conll
syntaxdot finetune --gpu 0 --mixed-precision --batch-size 32 --label-smoothing 0.03 --warmup 10000 --maxlen 100 --keep-best 3 syntaxdot.conf xlm-roberta-checkpoint.pt train.conll dev.conll

The checkpoint needs to be converted to SyntaxDot format using one of the scripts in:

https://github.com/tensordot/syntaxdot/tree/main/scripts

I still want to add support for using the checkpoints that are provided through Huggingface Hub directly, but I didn't have time for that yet.

I am currently on vacation, but I'll try to do a more complete write-up and add more examples later.

Apologies for the incomplete documentation, I haven't had much time to work on SyntaxDot for the last 1.5 years or so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@danieldk @josefkr @tondach01