Add deprecation warnings for ICL datasets/helper functions/metrics

[bmosaicml]

What does this PR do?

We are migrating ICL tasks from composer to foundry and need to deprecate the existing composer implementations. The migration PR is here: mosaicml/llm-foundry#936

What issue(s) does this change relate to?

Before submitting

• Have you read the contributor guidelines?
• Is this change a documentation change or typo fix? If so, skip the rest of this checklist.
• Was this change discussed/approved in a GitHub issue first? It is much more likely to be merged if so.
• Did you update any related docs and document your change?
• Did you update any related tests and add any new tests related to your change? (see testing)
• Did you run the tests locally to make sure they pass?
• Did you run pre-commit on your change? (see the pre-commit section of prerequisites)

[DhruvDh]

Not sure I understand why this change was needed. As composer is meant for "composing" training runs for models, language models, being one of them; and running evals during training is important. Why would ICL evals need to be in llm-foundry?

Is training language models using composer a deprecated use-case?

[maxisawesome]

@DhruvDh Committing changes to ICL Datasets and metrics required PRs to both foundry and composer which caused not-insignificant delays and complications in the development process. The ICL tasks are solely designed for LLMs and llm-foundry was always intended as the main place for LLM based eval code. The migration of ICL tasks to llm-foundry also helped us streamline code and consolidate defaults, which were two major pain points we had found internally.

[DhruvDh]

Thank you for your response. I did end finding that doing ICL eval with composer/llm-foundry was too painful and up rolling my own. I can see now that it is a bit complicated especially if you want to do distributed evaluation, so this change is probably a good idea.

Though ICL like context-continuation evaluation does not be restricted to natural language and llms. I think you usually end up with better abstractions when you try to be general. I think a "composer" purely for evaluation with a similar events/callback api could be ideal.