Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'CombinedEvaluations' object has no attribute 'evaluation_modules' #603

Open
shunk031 opened this issue Jun 23, 2024 · 2 comments · May be fixed by #604
Open

AttributeError: 'CombinedEvaluations' object has no attribute 'evaluation_modules' #603

shunk031 opened this issue Jun 23, 2024 · 2 comments · May be fixed by #604

Comments

@shunk031
Copy link

Hi, I tried to run the following script by combining two metrics, but the error occurred:

Python 3.9.16 (main, Apr  5 2023, 13:58:15)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import evaluate

In [2]: evaluations = ("accuracy", "f1", "precision", "recall")

In [3]: clf_metrics = evaluate.combine(evaluations=evaluations)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[3], line 1
----> 1 clf_metrics = evaluate.combine(evaluations=evaluations)

File ~/.pyenv/versions/3.9.16/envs/my-dev/lib/python3.9/site-packages/evaluate/module.py:1034, in combine(evaluations, force_prefix)
   1008 def combine(evaluations, force_prefix=False):
   1009     """Combines several metrics, comparisons, or measurements into a single `CombinedEvaluations` object that
   1010     can be used like a single evaluation module.
   1011
   (...)
   1031     ```
   1032     """
-> 1034     return CombinedEvaluations(evaluations, force_prefix=force_prefix)

File ~/.pyenv/versions/3.9.16/envs/my-dev/lib/python3.9/site-packages/evaluate/module.py:884, in CombinedEvaluations.__init__(self, evaluation_modules, force_prefix)
    881     self.evaluation_module_names = list(evaluation_modules.keys())
    882 loaded_modules = []
--> 884 for module in self.evaluation_modules:
    885     if isinstance(module, str):
    886         module = load(module)

AttributeError: 'CombinedEvaluations' object has no attribute 'evaluation_modules'

If we use a list format for evaluations, it works fine, but I think we should also support tuple format, which work in the same way as list. This can be easily achieved by modifying the following section.

https://github.com/huggingface/evaluate/blob/main/src/evaluate/module.py#L877

May I create a PR?

@pzdkn
Copy link

pzdkn commented Jun 28, 2024

@shunk031 did you?

@shunk031 shunk031 linked a pull request Jun 28, 2024 that will close this issue
@shunk031
Copy link
Author

Hi, I created the PR at #604.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants