v0.7.0
🚀 LLM Foundry v0.7.0
LLM Foundry is an efficient codebase for training, evaluating, and deploying Large Language Models (LLMs) and serves as the foundation for the MPT model series.
In addition to the usual bug fixes and performance improvements, we've made foundry more customizable and extensible!
New Features
Registerable Components (#975, #1043, #1052, #1057)
We've made key components of LLM Foundry registrable, such as models, loggers, and callbacks. You can use the registry to easily customize and extend your training workflows.
This means that you can register new options for these components, and then use them in your yaml config.
Check out the README for detailed instructions and code examples!
Breaking Changes and Deprecations
Deprecated Feature Removals (#1063)
We've removed support for deprecated features: triton attention, Prefix LMs, Llama attention patch, z-loss, and text denoising. These features were little used, and we removed them to focus on the core features that are heavily used.
If you were using these features please let us know how you were using them in a GitHub issue. We're happy to add things back that are in heavy usage.
What's Changed
- Fix typo in monolithic chkpt callback docs by @sashaDoubov in #1024
- Allow code-quality workflow to be callable by @b-chu in #1026
- Fix llama attention patch by @dakinggg in #1036
- Adds a decorator for experimental features by @dakinggg in #1038
- Finish 0.6.0 release by @dakinggg in #1040
- Remove reference to attn_impl: triton by @dakinggg in #1041
- Registry based config - Part 1 by @dakinggg in #975
- Deprecate attention patching for llama by @dakinggg in #1047
- Compile GLU by @josejg in #1049
- log details to metadata for run analytics by @angel-ruiz7 in #992
- Update README.md by @dennyglee in #1056
- Add chat schema example for mlflow by @dakinggg in #1054
- Metrics registry by @dakinggg in #1052
- LLM Foundry CLI (just registry) by @dakinggg in #1043
- Bump Composer to 0.21.1 by @jjanezhang in #1053
- Dataloaders registry by @dakinggg in #1044
- Fix multi model eval by @dakinggg in #1055
- Remove unnecessary test workflow by @dakinggg in #1058
- Fix peft llama test by @dakinggg in #1059
- Models registry by @dakinggg in #1057
- Remove under construction from registry by @dakinggg in #1060
- Custom Exceptions for Mosaic Logger by @jjanezhang in #1014
- Bump version to 0.7.0 by @irenedea in #1063
- Fix file filter by @dakinggg in #1067
- Fix context printing by @irenedea in #1068
New Contributors
- @angel-ruiz7 made their first contribution in #992
- @dennyglee made their first contribution in #1056
Full Changelog: v0.6.0...v0.7.0
