Point to composer.callback.Generate

[aspfohl]

This removes the llm-foundry Generate callback in favor of composer.callback.Generate, which has newer features like logging to MLflow. I've deleted the existing Generate callback logic but kept it around just in case someone is directly importing from llm-foundry. The builder callback also supports converting batch_log_interval to interval for backwards compatibility

In a future llmfoundry release, should remove this callback entirely

example of logging to mlflow & wandb

[dakinggg]

Can you please manually test this? both with old args and new args. Also I know there isn't a unit test for this callback right now...but could you please add one?

[aspfohl]

Can you please manually test this? both with old args and new args. Also I know there isn't a unit test for this callback right now...but could you please add one?

Yep, will do! Planning to test with https://github.com/mosaicml/mcloud/pull/2529, and I'll add some unit tests too

Although since the Generate callback should now be in composer, not sure it makes sense to add tests here?

[irenedea]

Although since the Generate callback should now be in composer, not sure it makes sense to add tests here?

There are unit tests for Generate in composer that test for functionality. Is there something else we should be unit testing in foundry for this callback? I think the builder tests that are in this PR may be sufficient. cc: @dakinggg

[dakinggg]

@irenedea @aspfohl the "new" thing that foundry should test is that the callback works e2e with MPT