Add initial ONNX regressions #2094
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Just want to make sure we are referring to the same error (on different OSes), only the RM3 runs failed? |
|
@ArthurChen189 yea, I think that's the concurrency bug... I try running again, and I think you'll observe different runs failing. And with |
|
The weirdest part for me is that is not just a small loss of effectiveness, is almost like it is losing the original representation |
I think it's a weird concurrency bug... |
|
lgtm! |
ArthurChen189
approved these changes
Apr 5, 2023
lintool
added a commit
that referenced
this pull request
May 30, 2023
+ Regressions themselves were added in #2094 - this patch adds the documentation. + Fix download paths for SPLADE++ ED/SD corpora. + Minor tweaks for other docs
RMaarefdoust
pushed a commit
to RMaarefdoust/anserini
that referenced
this pull request
Sep 16, 2024
There appears to be a concurrency bug associated with regressions, which we address with python src/main/python/run_regression.py --search-pool 1 ... but we should circle back and look into later.
RMaarefdoust
pushed a commit
to RMaarefdoust/anserini
that referenced
this pull request
Sep 16, 2024
+ Regressions themselves were added in castorini#2094 - this patch adds the documentation. + Fix download paths for SPLADE++ ED/SD corpora. + Minor tweaks for other docs
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Building on @ArthurChen189 's recent PR.
Some initial ONNX regressions so we can start playing around with it...
These work:
As well as:
I'm encountering some concurrency issues, so without
--search-pool 1, the regressions fail. Will need to debug more.cc @cadurosar