Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Align RunMsMarco with Fatjar regression instructions #2473

Closed
lintool opened this issue Apr 28, 2024 · 3 comments
Closed

Align RunMsMarco with Fatjar regression instructions #2473

lintool opened this issue Apr 28, 2024 · 3 comments

Comments

@lintool
Copy link
Member

lintool commented Apr 28, 2024

Today, you can run regressions with the fatjar directly:
https://github.com/castorini/anserini/blob/master/docs/fatjar-regressions-v0.35.1.md

Today, you can also do this:

java -cp `ls target/*-fatjar.jar` io.anserini.reproduce.RunMsMarco

These aren't aligned. RunMsMarco only runs a subset of the cases in the fatjar instructions. We should bring the two into alignment - i.e., add the missing cases into RunMsMarco. It probably makes sense to run have then generate the same output to be consistent.

@lintool
Copy link
Member Author

lintool commented Apr 28, 2024

Separately:

  • we have fatjar instructions for BEIR, but no io.anserini.reproduce.RunBeir or something similar.
  • we have no repro for V2.1 runs, e.g., add option to io.anserini.reproduce.RunMsMarco.

@lintool
Copy link
Member Author

lintool commented Apr 30, 2024

Initial commit #2477 - some errors noted #2480

@lintool
Copy link
Member Author

lintool commented Aug 8, 2024

I believe this issue has been resolved.

@lintool lintool closed this as completed Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant