Skip to content

Latest commit

 

History

History
36 lines (23 loc) · 988 Bytes

CHANGE_LOG.md

File metadata and controls

36 lines (23 loc) · 988 Bytes

Change Log

All notable changes to this project will be documented in this file.

v0.0.2 | 2022-02

Added

  • Unified distributed layers
  • MoE support
  • DevOps tools such as github action, code review automation, etc.
  • New project official website

Changes

  • refactored the APIs for usability, flexibility and modularity
  • adapted PyTorch AMP for tensor parallel
  • refactored utilities for tensor parallel and pipeline parallel
  • Separated benchmarks and examples as independent repositories
  • Updated pipeline parallelism to support non-interleaved and interleaved versions
  • refactored installation scripts for convenience

Fixed

  • zero level 3 runtime error
  • incorrect calculation in gradient clipping

v0.0.1 beta | 2021-10

The first beta version of Colossal-AI. Thanks to all contributors for the effort to implement the system.

Added

  • Initial architecture of the system
  • Features such as tensor parallelism, gradient clipping, gradient accumulation