evaluate error

[pradeepkc11]

2023-11-20 12:22:45.003935: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-11-20 12:22:45.005021: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2211] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at
When i tried to install and load en_core_web_sm from spacy it did not work and when i tried to run the
evaluate_ooc.py using python i got the below error
https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
Traceback (most recent call last):
File "/home/pradeep/COSMOS/evaluate_ooc.py", line 5, in
from utils.config import *
File "/home/pradeep/COSMOS/utils/init.py", line 3, in
from .dataset import *
File "/home/pradeep/COSMOS/utils/dataset.py", line 5, in
from utils.dataset_utils import modify_caption_replace_entities
File "/home/pradeep/COSMOS/utils/dataset_utils.py", line 8, in
nlp = spacy.load("en")
File "/home/pradeep/.local/lib/python3.10/site-packages/spacy/init.py", line 51, in load
return util.load_model(
File "/home/pradeep/.local/lib/python3.10/site-packages/spacy/util.py", line 471, in load_model
raise IOError(Errors.E941.format(name=name, full=OLD_MODEL_SHORTCUTS[name])) # type: ignore[index]
OSError: [E941] Can't find model 'en'. It looks like you're trying to load a model from a shortcut, which is obsolete as of spaCy v3.0. To load the model, use its full name instead:

nlp = spacy.load("en_core_web_sm")

For more details on the available models, see the models directory: https://spacy.io/models and if you want to create a blank model, use spacy.blank: nlp = spacy.blank("en")
Exception ignored in: <function AtomicFunction.del at 0x7fd2c7181120>
Traceback (most recent call last):
File "/home/pradeep/.local/lib/python3.10/site-packages/tensorflow/python/eager/polymorphic_function/atomic_function.py", line 283, in del
TypeError: 'NoneType' object is not subscriptable

[pradeepkc11]

absl-py==2.0.0
aicsimageio==4.14.0
aiobotocore==2.5.4
aiohttp==3.8.6
aioitertools==0.11.0
aiosignal==1.3.1
albumentations==1.3.1
annotated-types==0.6.0
asciitree==0.3.3
astunparse==1.6.3
async-timeout==4.0.3
attrs==23.1.0
audioread==3.0.1
av==11.0.0
beautifulsoup4==4.12.2
blinker==1.4
blis==0.7.11
botocore==1.31.17
bottle==0.12.25
bottle-websocket==0.2.9
Brotli==1.1.0
bs4==0.0.1
cachetools==5.3.2
catalogue==2.0.10
certifi==2023.7.22
cffi==1.16.0
charset-normalizer==3.3.2
chromedriver-binary==74.0.3729.6.0
click==8.1.7
cloudpathlib==0.16.0
cloudpickle==3.0.0
cmake==3.27.7
colorama==0.4.6
command-not-found==0.3
confection==0.1.3
contourpy==1.2.0
cryptography==3.4.8
cycler==0.12.1
cymem==2.0.8
dask==2023.10.1
dbus-python==1.2.18
decorator==5.1.1
densenet-pytorch==0.2.0
distributed==2023.10.1
distro==1.7.0
distro-info===1.1build1
dlib==19.24.2
Eel==0.16.0
efficientnet-pytorch==0.7.1
einops==0.7.0
elementpath==4.1.5
et-xmlfile==1.1.0
exceptiongroup==1.1.3
fasteners==0.19
filelock==3.13.1
flatbuffers==23.5.26
fonttools==4.44.0
frozenlist==1.4.0
fsspec==2023.6.0
future==0.18.3
gast==0.5.4
gdown==4.7.1
gevent==23.9.1
gevent-websocket==0.10.1
google==3.0.0
google-auth==2.23.4
google-auth-oauthlib==1.0.0
google-pasta==0.2.0
greenlet==3.0.1
grpcio==1.59.2
h11==0.14.0
h5py==3.10.0
httplib2==0.20.2
huggingface-hub==0.18.0
idna==3.4
imagecodecs==2023.9.18
imageio==2.31.6
importlib-metadata==6.8.0
imutils==0.5.4
jeepney==0.7.1
Jinja2==3.1.2
jmespath==1.0.1
joblib==1.3.2
keras==2.14.0
keyring==23.5.0
kiwisolver==1.4.5
langcodes==3.3.0
launchpadlib==1.10.16
lazr.restfulclient==0.14.4
lazr.uri==1.0.6
lazy_loader==0.3
libclang==16.0.6
librosa==0.10.1
lightning-utilities==0.10.0
llvmlite==0.41.1
locket==1.0.0
lxml==4.9.3
Markdown==3.5.1
MarkupSafe==2.1.3
matplotlib==3.8.1
ml-dtypes==0.2.0
more-itertools==8.10.0
mpmath==1.3.0
msgpack==1.0.7
multidict==6.0.4
munch==4.0.0
murmurhash==1.0.10
mutagen==1.47.0
netifaces==0.11.0
networkx==3.2.1
nltk==3.8.1
numba==0.58.1
numcodecs==0.12.1
numpy==1.26.1
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.18.1
nvidia-nvjitlink-cu12==12.3.52
nvidia-nvtx-cu12==12.1.105
oauthlib==3.2.0
ome-types==0.4.2
ome-zarr==0.8.2
opencv-python==4.8.1.78
opencv-python-headless==4.8.1.78
openpyxl==3.1.2
opt-einsum==3.3.0
outcome==1.3.0.post0
packaging==23.2
pafy==0.5.5
pandas==2.1.2
partd==1.4.1
piexif==1.1.3
Pillow==10.0.1
platformdirs==3.11.0
pooch==1.8.0
preshed==3.0.9
pretrainedmodels==0.7.4
protobuf==4.23.4
psutil==5.9.6
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycparser==2.21
pycryptodomex==3.19.0
pydantic==2.4.2
pydantic_core==2.10.1
pyDeprecate==0.3.2
pydub==0.25.1
PyGObject==3.42.1
PyJWT==2.3.0
pyparsing==2.4.7
PySocks==1.7.1
pyTelegramBotAPI==4.14.0
python-apt==2.4.0+ubuntu1
python-dateutil==2.8.2
pytorch-lightning @ file:///home/pradeep/pytorch_lightning-1.7.7-py3-none-any.whl#sha256=4438b8284d7f7fdb06cf3566a7b5b6f102ac8971cf7bb6d3f1b1de64628241f3
pytz==2023.3.post1
PyYAML==6.0.1
qudida==0.0.4
regex==2023.10.3
requests==2.31.0
requests-oauthlib==1.3.1
resource-backed-dask-array==0.1.0
rpycolors==0.0.5
rsa==4.9
s3fs==2023.6.0
safetensors==0.4.0
scikit-image==0.22.0
scikit-learn==1.3.2
scipy==1.11.3
searchtweets==1.7.6
SecretStorage==3.3.1
segmentation-models-pytorch==0.3.3
selenium==4.15.2
six==1.16.0
smart-open==6.4.0
sniffio==1.3.0
snscrape==0.7.0.20230622
sortedcontainers==2.4.0
soundfile==0.12.1
soupsieve==2.5
soxr==0.3.7
spacy==3.7.2
spacy-legacy==3.0.12
spacy-loggers==1.0.5
srsly==2.4.8
sympy==1.12
systemd-python==234
tblib==3.0.0
telebot==0.0.5
telegram==0.0.1
tensorboard==2.14.1
tensorboard-data-server==0.7.2
tensorflow==2.14.0
tensorflow-estimator==2.14.0
tensorflow-hub==0.15.0
tensorflow-io-gcs-filesystem==0.34.0
termcolor==2.3.0
textblob==0.17.1
thinc==8.2.1
threadpoolctl==3.2.0
tifffile==2023.2.28
timm==0.9.2
tk==0.1.0
toml==0.10.2
toolz==0.12.0
torch==2.1.1
torchaudio==2.1.1
torchmetrics==0.11.4
torchvision==0.16.0
tornado==6.3.3
tqdm==4.66.1
trio==0.23.1
trio-websocket==0.11.1
triton==2.1.0
tweepy==4.14.0
tweet-parser==1.13.2
tweety==0.1.6
typer==0.9.0
typing_extensions==4.8.0
tzdata==2023.3
ubuntu-advantage-tools==8001
ufw==0.36.1
unattended-upgrades==0.1
UNKNOWN @ git+https://github.com/JustAnotherArchivist/snscrape.git@614d4c2029a62d348ca56598f87c425966aaec66
urllib3==1.26.18
vaderSentiment==3.3.2
wadllib==1.3.6
wasabi==1.1.2
weasel==0.3.4
websockets==12.0
Werkzeug==3.0.1
whichcraft==0.6.1
wrapt==1.14.1
wsproto==1.2.0
xarray==2023.10.1
xmlschema==2.5.0
xsdata==23.8
yarl==1.9.2
youtube-dl==2021.12.17
youtube-extract==1.4.0
yt-dlp==2023.10.13
zarr==2.15.0
zict==3.0.0
zipp==1.0.0
zope.event==5.0
zope.interface==6.1

dependencies with versions installed

[pradeepkc11]

Traceback (most recent call last):
File "/home/pradeep/COSMOS/trainer_scipt.py", line 232, in
train_joint_model()
File "/home/pradeep/COSMOS/trainer_scipt.py", line 156, in train_joint_model
train_model(epoch)
File "/home/pradeep/COSMOS/trainer_scipt.py", line 85, in train_model
z_img, z_t_match, z_t_diff = combined_model(img, text_match, text_diff, batch, seq_len_match, seq_len_diff,
File "/home/pradeep/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/pradeep/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/pradeep/COSMOS/model_archs/models.py", line 51, in forward
img = self.maskrcnn_extractor(img, bboxes, bbox_classes)
File "/home/pradeep/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/pradeep/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/pradeep/COSMOS/model_archs/image/image_models.py", line 47, in forward
[self.maskrcnn_predictor(img.cpu().numpy(), target) for img, target in zip(imgs, targets)])
File "/home/pradeep/COSMOS/model_archs/image/image_models.py", line 47, in
[self.maskrcnn_predictor(img.cpu().numpy(), target) for img, target in zip(imgs, targets)])
TypeError: DefaultPredictor.call() takes 2 positional arguments but 3 were given

[codewithriza]

Issue Analysis and Solutions

TensorFlow GPU Error:

The error message suggests an issue with GPU libraries and a potential problem with NUMA support. This might be due to missing or incorrectly installed GPU libraries. Ensure that you've followed the TensorFlow GPU installation guide for your platform and that the required libraries are installed properly. The specific error messages related to CUDA and GPU support need to be resolved.

Spacy Model Loading Error:

The error occurred while loading the 'en' model using Spacy. In recent versions of Spacy (v3.0+), model loading syntax has changed. Instead of 'en', use the full model name 'en_core_web_sm' to load the English model:

nlp = spacy.load("en_core_web_sm")