Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add versioning the HubEvaluationModuleFactory #314

Merged
merged 1 commit into from
Oct 12, 2022
Merged

Conversation

lvwerra
Copy link
Member

@lvwerra lvwerra commented Oct 7, 2022

This PR couples the module repositories on the hub with the main repository in evaluate, based on tags that are propagated from evaluate to each space when they are pushed to the hub via actions (see evaluate/.github/hub/push_evaluations_to_hub.py).

The logic is:

  • if user provides revision, this is used.
  • else: there is no revision is provided then,
    • if evaluate has a dev tag main is used
    • else version tag of evaluate is used (e.g. v0.2.0)

This addresses the massive issues when merging #188. This will not solve the issues for existing versions but at least for newer versions this should not be an issue anymore.

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Oct 7, 2022

The documentation is not available anymore as the PR was closed or merged.

@lhoestq
Copy link
Member

lhoestq commented Oct 7, 2022

Alternatively it can use main by default, but if it contains a requirements.txt with an old version of evaluate it can check the tags for release-* ?

(tbh I was thinking of something like that for datasets)

This way we stay consistent with datasets and models: use main by default + no need to make a release to update something

@lvwerra
Copy link
Member Author

lvwerra commented Oct 10, 2022

The issue with that is that we'll never be able to have a breaking change between the hub modules and the github repository without breaking every older version of the library.

this is what's happening with #188: since changed the module structure a bit all the older version of evaluate will try to load them.

@lhoestq
Copy link
Member

lhoestq commented Oct 10, 2022

If we use release tags release-* we can resolve to the latest tag with a requirements.txt that is compatible with the user's local version of evaluate no ?

@lvwerra lvwerra merged commit c258f0a into main Oct 12, 2022
@lvwerra lvwerra deleted the fix-hub-versioning branch October 12, 2022 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants