Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The run_shark_tank.yml workflow is broken #15623

Closed
ScottTodd opened this issue Nov 16, 2023 · 0 comments · Fixed by #15825
Closed

The run_shark_tank.yml workflow is broken #15623

ScottTodd opened this issue Nov 16, 2023 · 0 comments · Fixed by #15825
Labels
bug 🐞 Something isn't working

Comments

@ScottTodd
Copy link
Member

What happened?

The workflow defined at .github/workflows/run_shark_tank.yml has been silently skipped for months (workflow history).

I'm pretty sure that workflow is the source for data displayed at https://storage.googleapis.com/shark-benchmark-artifacts/latest/summary.html

I have a patch here: 52bf41c to re-enable the workflow, but it has bitrotted. Sample workflow results: https://github.com/openxla/iree/actions/runs/6896441651, errors:

+ pytest --benchmark --update_tank --maxfail=500 -k cpu tank/test_models.py
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: argument --benchmark: expected one argument

Steps to reproduce your issue

  1. Create a branch in this repo with changes to test (no need to create a PR)
  2. Navigate to https://github.com/openxla/iree/actions/workflows/run_shark_tank.yml
  3. Dispatch the workflow with the "Run workflow" button -> pick your branch

What component(s) does this issue relate to?

Other

Version information

I suspect this has been skipped since 8ae0f52 (July 10, 2023). The dashboard linked above was last updated on April 3, 2023... maybe something else broke first.

Additional context

We discussed this here on Discord

It sounds like people still find value in this workflow - specifically for the model coverage and dashboard UI. However, if this workflow is no longer useful, we could opt to delete it and any scripts it uses instead.

@ScottTodd ScottTodd added the bug 🐞 Something isn't working label Nov 16, 2023
ScottTodd added a commit that referenced this issue Nov 16, 2023
Spotted these while debugging
#15623

skip-ci: comment-only change
Groverkss pushed a commit that referenced this issue Nov 17, 2023
Spotted these while debugging
#15623

skip-ci: comment-only change
ramiro050 pushed a commit to ramiro050/iree that referenced this issue Dec 19, 2023
Spotted these while debugging
iree-org#15623

skip-ci: comment-only change
ScottTodd added a commit that referenced this issue Jan 18, 2024
Fixes #15623 . If the workflow is
still useful then (IMO) it should find another repository to live in.

Progress towards #15332 (fewer
Dockerfiles to port/maintain)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐞 Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant