-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add metric logging #223
Add metric logging #223
Conversation
I like this. thanks a lot @Dref360. I was only wondering don't you think it is more descriptive to have "active_step" as the key for metrics and have "datasize" as a field ? |
This looks great, will give it a try as soon as it's ready. |
Good idea, not sure how to keep track of the active step tho. Do you have an idea? Like just a field in ModelWrapper? |
I think this is ready for review. |
Is there an entry in the docs for this? Regardless of that, this looks good! |
@rafapi just added an example in our documentation. |
Quick video showcasing the feature. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Summary:
Add some utils to keep track of metrics over time.
This is WIP, but what do you think of this API?
It would be quite simple for the users to get a metrics for each dataset size.
We can also add utilities for that such as
MetricMixin.get_active_metric(metric_name="precision") : Dict[int, float]
Features:
Fixes #220
Checklist:
tests/documentation_test.py
).