Skip to content

0.3.0
Compare
Choose a tag to compare
@aallam aallam released this 14 Oct 10:56
· 8 commits to main since this release
0.3.0
55d3485
This commit was signed with the committer’s verified signature.
aallam Mouaad Aallam
GPG key ID: 0FB318F8C9001426
Learn about vigilant mode.

Focused on internal refactoring and API stabilization to enhance usability and facilitate future development.

Breaking Changes

  • Rename Enconding to EncodingConfig.
  • Rework encodings to enhance extensibility:
    • Add a new Encoding interface.
    • Introduce default implementations of Encoding: CL100KBase, P50KBase, R50KBase, and P50KEdit.
  • Rename Tokenizer.encoding and Tokenizer.encodingForModel to Tokenizer.of with overrides.
Assets 2
Loading