-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
error when using the -untie_encoder param for DPR - config.json not in the correct path. #69
Comments
Hi @lboesen |
Hi @MXueguang, |
Hi @MXueguang, (I was a little too quick closing the issues :)) I tried including the --untie_encoder param when encoding the corpus and queries, but get following error message: OSError: /home/model_runs/DPR does not appear to have a file named config.json. Checkout 'https://huggingface.co//home/model_runs/DPR/None' for available files. Since passage_model/config.json and query_model/config.json are identical I move the config.json file from one of them out to the path where the path containing all relevant files for a BertTokenizerFast tokenizer. Tried to check if I needed to direct the path directly to the passage_model folder: If I directed the --model_name_or_path .../passage_model (for corpus encoding and verse visa for the queries encoding) it gives following error: OSError: Can't load tokenizer for '../passage_model'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '../passage_model' is the correct path to a directory containing all relevant files for a BertTokenizerFast tokenizer. So currently I can only get it to work if I include the --untie_encoder params and move the config.json out of either passage_model or query_model folder and out the the path containing the tokenizer. |
Same problem |
Hi,
I had the following issue today:
When using the -untie_encoder params in this guide: https://github.com/texttron/tevatron/blob/main/examples/example_dpr.md
When done training the DPR model it does not include the config.json in the --output_dir.
This gives issues when trying the encode corpus and queries.
I moved the config.json from either the passage_model/ or the query_model/ folder to the path stated in the "--output_dir":
The text was updated successfully, but these errors were encountered: