ImportError: gpu_nmscpython-36m-x86_64-linux-gnu.so: undefined symbol: __cudaPopCallConfiguration

[JosePeeterson]

Hi,
I am facing the following error when I run demo_video_mobile.py .

Traceback (most recent call last): File "demo_video_mobile.py", line 28, in <module> from nms.gpu_nms import gpu_nms ImportError: /home/peeterson/Desktop/lighttrack/HPE/../lib/nms/gpu_nms.cpython-36m-x86_64-linux-gnu.so: undefined symbol: __cudaPopCallConfiguration

module versions inside py36 environment:

_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 1_llvm conda-forge
_tflow_180_select 1.0 gpu anaconda
absl-py 0.2.2 py36_0 anaconda
astor 0.6.2 py36_0 anaconda
blas 1.0 mkl conda-forge
bleach 1.5.0 py36_0 conda-forge
bzip2 1.0.6 1 conda-forge
ca-certificates 2020.12.5 ha878542_0 conda-forge
cairo 1.14.12 h77bcde2_0 anaconda
certifi 2020.12.5 py36h5fab9bb_1 conda-forge
cloudpickle 0.5.3 pypi_0 pypi
cmake 3.12.0 h307fef2_0 anaconda
cudatoolkit 9.0 h13b8566_0 anaconda
cudnn 7.1.2 cuda9.0_0 anaconda
cupti 9.0.176 0 anaconda
cycler 0.10.0 py36_0 conda-forge
cython 0.28.3 pypi_0 pypi
dask 0.17.5 pypi_0 pypi
dataclasses 0.8 pypi_0 pypi
datetime 4.2 pypi_0 pypi
dbus 1.13.0 h3a4f0e9_0 conda-forge
decorator 4.3.0 pypi_0 pypi
easydict 1.7 pypi_0 pypi
expat 2.2.5 0 conda-forge
ffmpeg 3.2.4 3 conda-forge
fontconfig 2.13.0 0 conda-forge
freetype 2.8.1 0 conda-forge
gast 0.2.0 py36_0 anaconda
giflib 5.1.4 0 conda-forge
glib 2.53.6 h5d9569c_2 anaconda
graphite2 1.3.11 0 conda-forge
grpcio 1.12.0 py36hdbcaa40_0 anaconda
gst-plugins-base 1.12.4 h33fb286_0 anaconda
gstreamer 1.12.4 hb53b477_0 anaconda
h5py 2.9.0 pypi_0 pypi
harfbuzz 1.7.6 hc5b324e_0 anaconda
hdf5 1.10.2 0 conda-forge
html5lib 0.9999999 py36_0 conda-forge
icu 58.2 0 conda-forge
imageio 2.3.0 py36_0 conda-forge
jasper 1.900.1 4 conda-forge
jpeg 9b h024ee3a_2
kiwisolver 1.0.1 py36_1 conda-forge
libcurl 7.61.0 h1ad7b7a_0 anaconda
libedit 3.1.20170329 h6b74fdf_2 anaconda
libffi 3.2.1 hd88cf55_4 anaconda
libgcc 7.2.0 h69d50b8_2 conda-forge
libgcc-ng 7.2.0 hdf63c60_3 anaconda
libgfortran 3.0.0 1 conda-forge
libgfortran-ng 7.2.0 hdf63c60_3 anaconda
libiconv 1.15 h63c8f33_5 anaconda
libopenblas 0.2.20 h9ac9557_7 anaconda
libpng 1.6.34 0 conda-forge
libprotobuf 3.5.2 h6f1eeef_0 anaconda
libssh2 1.8.0 h9cfc8f7_4 anaconda
libstdcxx-ng 7.2.0 hdf63c60_3 anaconda
libtiff 4.0.9 he6b73bb_1 conda-forge
libuuid 1.0.3 h1bed415_2 anaconda
libwebp 0.5.2 7 conda-forge
libxcb 1.13 0 conda-forge
libxml2 2.9.8 h26e45fe_1 anaconda
llvm-openmp 11.0.1 h4bd325d_0 conda-forge
markdown 2.6.11 py36_0 anaconda
matplotlib 2.2.2 py36h0e671d2_0 anaconda
mkl 2020.4 h726a3e6_304 conda-forge
msgpack 0.5.6 pypi_0 pypi
msgpack-numpy 0.4.3 pypi_0 pypi
munkres 1.0.12 pypi_0 pypi
ncurses 6.1 hf484d3e_0 anaconda
networkx 2.1 pypi_0 pypi
numpy 1.14.3 py36h28100ab_1 anaconda
numpy-base 1.14.3 py36h0ea5e3f_1 anaconda
olefile 0.45.1 py36_0 conda-forge
openblas 0.2.20 8 conda-forge
opencv 3.4.1 py36h6fd60c2_1 anaconda
opencv-python 3.4.1.15 pypi_0 pypi
openssl 1.0.2o 0 conda-forge
pcre 8.41 1 conda-forge
pillow 5.1.0 py36_0 conda-forge
pip 21.0.1 pypi_0 pypi
pixman 0.34.0 2 conda-forge
protobuf 3.5.2 py36hf484d3e_0 anaconda
pyarrow 0.9.0 pypi_0 pypi
pycocotools 2.0.0 pypi_0 pypi
pyparsing 2.2.0 py36_0 conda-forge
pyqt 5.6.0 py36_5 conda-forge
pyqt5 5.12.1 pypi_0 pypi
pyqt5-sip 4.19.15 pypi_0 pypi
python 3.6.5 hc3d631a_2 anaconda
python-dateutil 2.7.3 py_0 conda-forge
python_abi 3.6 1_cp36m conda-forge
pytz 2018.4 py_0 conda-forge
pywavelets 0.5.2 pypi_0 pypi
pyyaml 3.12 pypi_0 pypi
pyzmq 17.0.0 pypi_0 pypi
qt 5.6.2 hd25b39d_14 anaconda
readline 7.0 ha6073c6_4 anaconda
rhash 1.3.6 hb7f436b_0 anaconda
scikit-image 0.14.0 pypi_0 pypi
scipy 1.1.0 pypi_0 pypi
setproctitle 1.1.10 pypi_0 pypi
setuptools 39.2.0 py36_0 conda-forge
shapely 1.6.4.post1 pypi_0 pypi
sip 4.18 py36_1 conda-forge
six 1.11.0 py36h372c433_1 anaconda
sqlite 3.23.1 he433501_0 anaconda
tabulate 0.8.2 pypi_0 pypi
tensorboard 1.8.0 py36hf484d3e_0 anaconda
tensorflow 1.8.0 hb11d968_0 anaconda
tensorflow-base 1.8.0 py36hc1a7637_0 anaconda
tensorflow-gpu 1.8.0 h7b35bdc_0 anaconda
termcolor 1.1.0 py36_1 conda-forge
tk 8.6.7 hc745277_3 anaconda
toolz 0.9.0 pypi_0 pypi
torch 1.0.1 pypi_0 pypi
torchvision 0.2.2 pypi_0 pypi
tornado 5.0.2 py36_0 conda-forge
tqdm 4.19.9 pypi_0 pypi
typing-extensions 3.7.4.3 pypi_0 pypi
webencodings 0.5.1 py_1 conda-forge
werkzeug 0.14.1 py36_0 anaconda
wheel 0.31.1 py36_0 conda-forge
x264 20131218 0 conda-forge
xorg-libxau 1.0.8 3 conda-forge
xorg-libxdmcp 1.1.2 3 conda-forge
xz 5.2.4 h14c3975_4 anaconda
zlib 1.2.11 ha838bed_2 anaconda
zope-interface 4.5.0 pypi_0 pypi

I have installed the same cuda and cudnn versions above in my system.

I have exported the path as well.
export PATH=/usr/local/cuda-9.0/bin export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64:/usr/local/cuda-9.0/extras/CUPTI/lib64:/lib/nccl/cuda-9

I also tried to use an earlier version of gcc as per this:
[https://stackoverflow.com/questions/6622454/cuda-incompatible-with-my-gcc-version]

I still get the same error. could someone suggest how to fix this?

thanks.

[gaoyelu1996]

hello I have this problem . could you solve it?

[ankithagithub]

ImportError: /mnt/DATA/Ankita/lighttrack-master/HPE/../lib/nms/gpu_nms.cpython-36m-x86_64-linux-gnu.so: undefined symbol: __cudaRegisterFatBinaryEnd
I'm also getting the same error.could someone suggest how to fix this?