Skip to content

ChatLLaMA 0.0.3

Pre-release
Pre-release
Compare
Choose a tag to compare
@diegofiori diegofiori released this 27 Mar 08:16
· 180 commits to main since this release
chatllama0.0.3
9efe338
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Release Notes - ChatLLaMA 0.0.3

Major release of ChatLLaMA solving multiple bugs and expanding the support to distrubuted training.

New Features

  • Training produces a log file containing also the training stats
  • Add Template during the dataset creation
  • Changed default training parameters to InstructGPT paper ones
  • Implemented Cosine Scheduler for LR
  • Add Conversation Logs during RLHF
  • Improved management of checkpoints
  • Add support for Accelerate backend for distributed training
  • Defer import of LLaMA modules, avoiding the LLaMA dependecy while another model is used

Bug Fix

  • Fix bug causing crashes during training due to long sequences
  • Fix multiple bugs on deepspeed MultiGPU training
  • Fix bugs on RLHF training
  • Fix bug with different versions of LangChain during synthetic data generation

New Contributors

Contributors

egrefen, HuangLK, and 8 other contributors
Assets 2
Loading