Add results to sotabench

[Randl]

🚀 Feature

Add the model results on COCO to sotabench

Motivation

Sotabench is arguably the best independent model benchmark which allows to obtain fair comparison between models.

Pitch

What is required is to add sotabench.py to the repository. The website has both guides and code examples. Existing submissions can be used as a guideline too.

[github-actions]

Hello @Randl, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Jupyter Notebook , Docker Image, and Google Cloud Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom model or data training question, please note that Ultralytics does not provide free personal support. As a leader in vision ML and AI, we do offer professional consulting, from simple expert advice up to delivery of fully customized, end-to-end production solutions for our clients, such as:

• Cloud-based AI systems operating on hundreds of HD video streams in realtime.
• Edge AI integrated into custom iOS and Android apps for realtime 30 FPS video inference.
• Custom data training, hyperparameter evolution, and model exportation to any destination.

For more information please visit https://www.ultralytics.com.

[glenn-jocher]

@Randl great idea, thank you! I've not used sotabench before unfortunately, but I browsed the site just now and it seems to be a worthwhile effort. Have you done submissions to the site yourself?

[Randl]

I've done some submissions but mostly to classification leaderboard. Detection appears a bit more complicated, but you might find https://github.com/tensorpack/tensorpack/blob/master/sotabench/sotabench.py and https://github.com/Randl/DetectoRS/blob/master/sotabench.py useful.

[glenn-jocher]

their sotabench.py looks decently complicated, though I can't tell if that's simply due to the detectorrs implementation or the sotabench requirements.

If anyone reading this has any experience submitting repositories to sotabench we could use help with this. This looks like a great idea, especially since we will be updating the models as the year goes on, so it would be nice to see realtime-ish results update after applying changes, but unfortunately I won't be able to free up any time to work on it until July perhaps.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Randl]

I don't think it should be closed

[glenn-jocher]

@Randl hey buddy. We don't have resources to do this ourselves, we are stretched very thin, but if you'd like to submit a PR we'd be happy to review!

[glenn-jocher]

@Randl I've created a sotabench.py file now in the root directory, which is essentially a slight modification of the existing test.py file. sotabench.py defaults to testing YOLOv5s. I've registered the repo on sotabench.com, but something is not working correctly, as all builds get skipped. Any ideas why this might be happening?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.