Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImportError: cannot import name 'initialize_tasks' from 'lm_eval.tasks' #565

Open
kunzeng-ch opened this issue Aug 1, 2024 · 1 comment

Comments

@kunzeng-ch
Copy link

When I run the examples/evaluator. py file, the code runs from awq.evaluation import(
evaluate_perplexity,
eval_librispeech,
eval_mmlu,
eval_humaneval,
eval_kl_divergence,)An ImportError occurred: cannot import name 'initialized tasks' from' lm_ eval. tasks' error. Can you help me?

Here is my environment:
cuda: 12.4
torch: 2.3.1
transformers: 4.42.4
vllm: 0.5.3
lm_eval: 0.4.3

@casper-hansen
Copy link
Owner

You can pip install -e .[eval] to get the right dependencies

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants