-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Receiving a JSONDecodeError
when running tevatron.driver.encode
on WQ dataset
#53
Comments
Hi @xhluca,
let me know if you still having the issue Xueguang |
I'm not sure what json environment means here. I'm using the standard python 3.7 library in a fresh virtualenv |
I tried different datasets and the problem seems to be present |
Could you see if a simple jsonl file can be read in your environment? or could you try conda environment? My environment is python3.8 with conda |
Yes, I tried the following example: https://stackoverflow.com/questions/50475635/loading-jsonl-file-as-json-objects ANd it works fine in my environment |
@MXueguang My bad, I was indeed using conda. However, do you think there should be a difference whether I"m using conda or virtualenv since the libraries were installed with pip and there's no conda-specific dependency? |
I have first used tevatron to train DPR from bert-based-uncased:
After the model was saved to
model_wq/
(see footnote), I continued to follow the instructions to encode the passages:I saved that inside a bash file and ran the bash file, but I multiple
JSONDecodeError
along the way, which does not seem to be expected (which is why I stopped the process):Is this normal?
Libraries
This is my requirements file:
Footnote
model_nq
but renamed it tomodel_wq
, I don't think this makes a difference but if it does let me know.master
and also with the 0.1 version on pypi and I'm getting the same error.The text was updated successfully, but these errors were encountered: