v0.4.4

Important changes

• Fixed a bug bf16 + big endian + torch #507
• Lots of minor fixes

What's Changed

• Fixing empty serialization (no tensor) with some metadata. by @Narsil in #472
• Minor correction about Arrow in README.md by @lhoestq in #488
• Upgrading Pyo3 0.22 (with gil-refs) by @Narsil in #504
• Improving the bf16 tests for PT+TF. by @Narsil in #505
• Revert "Improving the bf16 tests for PT+TF." by @Narsil in #506
• Change doc-builder workflow container by @mishig25 in #508
• feat(ci): add trufflehog secrets detection by @McPatate in #484
• Add support for xla devices by @BitPhinix in #494
• Improving bf16 tests by @Narsil in #507
• Fixing zero-width tensor for in memory loading. by @Narsil in #510
• Respects torch.device(0) new behavior without breaking backward compatibility by @Narsil in #509
• Upgrading the version. by @Narsil in #511
• Fixing release workflow. by @Narsil in #513
• Forcing empty list/empty tuple behavior. by @Narsil in #514

New Contributors

• @lhoestq made their first contribution in #488
• @McPatate made their first contribution in #484
• @BitPhinix made their first contribution in #494

Full Changelog: v0.4.3...v0.4.4

v0.4.4rc0

What's Changed

• Fixing empty serialization (no tensor) with some metadata. by @Narsil in #472
• Minor correction about Arrow in README.md by @lhoestq in #488
• Upgrading Pyo3 0.22 (with gil-refs) by @Narsil in #504
• Improving the bf16 tests for PT+TF. by @Narsil in #505
• Revert "Improving the bf16 tests for PT+TF." by @Narsil in #506
• Change doc-builder workflow container by @mishig25 in #508
• feat(ci): add trufflehog secrets detection by @McPatate in #484
• Add support for xla devices by @BitPhinix in #494
• Improving bf16 tests by @Narsil in #507
• Fixing zero-width tensor for in memory loading. by @Narsil in #510
• Respects torch.device(0) new behavior without breaking backward compatibility by @Narsil in #509
• Upgrading the version. by @Narsil in #511

New Contributors

• @lhoestq made their first contribution in #488
• @McPatate made their first contribution in #484
• @BitPhinix made their first contribution in #494

Full Changelog: v0.4.3...v0.4.4rc0

v0.4.3

What's Changed

• Updating minor after release 0.4.2 by @Narsil in #433
• Adding support for integer indexing [0, :2, -1]. by @Narsil in #440
• Updating doc. by @Narsil in #465
• Add support for device in safetensors.torch.load_model by @Wauplin in #449
• Upgrading pyo3 to 0.21. by @Narsil in #468

Full Changelog: v0.4.2...v0.4.3

v0.4.3rc0

What's Changed

• Updating minor after release 0.4.2 by @Narsil in #433
• Adding support for integer indexing [0, :2, -1]. by @Narsil in #440
• Updating doc. by @Narsil in #465
• Add support for device in safetensors.torch.load_model by @Wauplin in #449
• Upgrading pyo3 to 0.21. by @Narsil in #468

Full Changelog: v0.4.2...v0.4.3rc0

v0.4.2

What's Changed

• Back on dev. by @Narsil in #313
• Clarify specification around endianness and header padding by @akx in #306
• Adding new target for m1 by @Narsil in #317
• Fixing macos 13 release script (by prebuilding). by @Narsil in #315
• Cache dir all the time. by @Narsil in #319
• Supporting whisper conversion. by @Narsil in #320
• add tensorshare to featured projects by @chainyo in #322
• Fix Bench GH action. by @Narsil in #323
• Yaml is nice. by @Narsil in #325
• Fixing big endian. by @Narsil in #327
• Fixing boolean + numpy > 1.20 by @Narsil in #326
• Add NumPy for all backends dependencies by @chainyo in #332
• impl View for TensorView by @coreylowman in #329
• Reworked the release script entirely by @Narsil in #334
• Preparing small patch release (Big Endian fix). by @Narsil in #335
• Temporary revert of the breaking change (keep it for 0.4.0). by @Narsil in #336
• Don't release rust versions on RC by @Narsil in #337
• Fixing release script. by @Narsil in #338
• Python 3.8 for arm by @Narsil in #339
• Win64. by @Narsil in #342
• win64 by @Narsil in #343
• Trying out maturin. by @Narsil in #344
• Reduce memory allocations by @cospectrum in #350
• [docs] Convert weights by @stevhliu in #345
• Update types for load_model filename by @kevinhu in #356
• Fixing release script for windows aarch64 and 3.12 by @Narsil in #348
• Minor docstring edit to flax.py by @paulbricman in #359
• Stop recreating the hashmap all the time. by @Narsil in #363
• fix: add py.typed by @b-kamphorst in #365
• Preparing a new release (0.4.0). by @Narsil in #366
• Supporting bfloat16 for tensorflow + jax (was failing because of intermediary numpy). by @Narsil in #382
• Fix typo by @cccntu in #377
• Support fp8_e4m3/fp8_e5m2 by @Narsil in #383
• Better convert. by @Narsil in #384
• Ignore closed PRs to avoid spam. by @Narsil in #385
• Adding support for Ascend NPU by @statelesshz in #372
• Fix convert. by @Narsil in #390
• Upgrade dev version. by @Narsil in #392
• Fix/errors for some models including Mistral-7b and Flan-ul2 by @lachlancahill in #388
• Releasing for python 3.12/3.13 and testing on 3.11. by @Narsil in #393
• Adding stale action. by @Narsil in #398
• Document huggingface_hub.get_safetensors_metadata by @Wauplin in #417
• [docs] Update list of projects using safetensors by @mishig25 in #418
• Adding MLX backend. by @Narsil in #419
• [docs, untested] Attempt to fix links to source by @pcuenca in #420
• [docs] Use hfoptions for metadata by @mishig25 in #424
• Fix path in version_tag_suffix by @pcuenca in #426
• Removing old doc. by @Narsil in #427
• Fix missing Apache-2.0 license file in the published crate by @musicinmybrain in #416
• Implement PyTorch support for float8 types (F8_E5M2 and F8_E4M3) by @zeux in #404
• Fix mispelling of dictionary. by @hayden-donnelly in #410
• Add "xpu" device tag to the device list to support Intel GPU by @Narsil in #428
• Skip MLX bench for the bench runner. by @Narsil in #429
• Remove py313 windows. by @Narsil in #432

New Contributors

• @akx made their first contribution in #306
• @chainyo made their first contribution in #322
• @coreylowman made their first contribution in #329
• @cospectrum made their first contribution in #350
• @stevhliu made their first contribution in #345
• @kevinhu made their first contribution in #356
• @paulbricman made their first contribution in #359
• @b-kamphorst made their first contribution in #365
• @cccntu made their first contribution in #377
• @statelesshz made their first contribution in #372
• @lachlancahill made their first contribution in #388
• @Wauplin made their first contribution in #417
• @pcuenca made their first contribution in #420
• @musicinmybrain made their first contribution in #416
• @zeux made their first contribution in #404
• @hayden-donnelly made their first contribution in #410

Full Changelog: v0.3.2...v0.4.2

v0.4.2rc0

What's Changed

• Back on dev. by @Narsil in #313
• Clarify specification around endianness and header padding by @akx in #306
• Adding new target for m1 by @Narsil in #317
• Fixing macos 13 release script (by prebuilding). by @Narsil in #315
• Cache dir all the time. by @Narsil in #319
• Supporting whisper conversion. by @Narsil in #320
• add tensorshare to featured projects by @chainyo in #322
• Fix Bench GH action. by @Narsil in #323
• Yaml is nice. by @Narsil in #325
• Fixing big endian. by @Narsil in #327
• Fixing boolean + numpy > 1.20 by @Narsil in #326
• Add NumPy for all backends dependencies by @chainyo in #332
• impl View for TensorView by @coreylowman in #329
• Reworked the release script entirely by @Narsil in #334
• Preparing small patch release (Big Endian fix). by @Narsil in #335
• Temporary revert of the breaking change (keep it for 0.4.0). by @Narsil in #336
• Don't release rust versions on RC by @Narsil in #337
• Fixing release script. by @Narsil in #338
• Python 3.8 for arm by @Narsil in #339
• Win64. by @Narsil in #342
• win64 by @Narsil in #343
• Trying out maturin. by @Narsil in #344
• Reduce memory allocations by @cospectrum in #350
• [docs] Convert weights by @stevhliu in #345
• Update types for load_model filename by @kevinhu in #356
• Fixing release script for windows aarch64 and 3.12 by @Narsil in #348
• Minor docstring edit to flax.py by @paulbricman in #359
• Stop recreating the hashmap all the time. by @Narsil in #363
• fix: add py.typed by @b-kamphorst in #365
• Preparing a new release (0.4.0). by @Narsil in #366
• Supporting bfloat16 for tensorflow + jax (was failing because of intermediary numpy). by @Narsil in #382
• Fix typo by @cccntu in #377
• Support fp8_e4m3/fp8_e5m2 by @Narsil in #383
• Better convert. by @Narsil in #384
• Ignore closed PRs to avoid spam. by @Narsil in #385
• Adding support for Ascend NPU by @statelesshz in #372
• Fix convert. by @Narsil in #390
• Upgrade dev version. by @Narsil in #392
• Fix/errors for some models including Mistral-7b and Flan-ul2 by @lachlancahill in #388
• Releasing for python 3.12/3.13 and testing on 3.11. by @Narsil in #393
• Adding stale action. by @Narsil in #398
• Document huggingface_hub.get_safetensors_metadata by @Wauplin in #417
• [docs] Update list of projects using safetensors by @mishig25 in #418
• Adding MLX backend. by @Narsil in #419
• [docs, untested] Attempt to fix links to source by @pcuenca in #420
• [docs] Use hfoptions for metadata by @mishig25 in #424
• Fix path in version_tag_suffix by @pcuenca in #426
• Removing old doc. by @Narsil in #427
• Fix missing Apache-2.0 license file in the published crate by @musicinmybrain in #416
• Implement PyTorch support for float8 types (F8_E5M2 and F8_E4M3) by @zeux in #404
• Fix mispelling of dictionary. by @hayden-donnelly in #410
• Add "xpu" device tag to the device list to support Intel GPU by @Narsil in #428
• Skip MLX bench for the bench runner. by @Narsil in #429

New Contributors

• @akx made their first contribution in #306
• @chainyo made their first contribution in #322
• @coreylowman made their first contribution in #329
• @cospectrum made their first contribution in #350
• @stevhliu made their first contribution in #345
• @kevinhu made their first contribution in #356
• @paulbricman made their first contribution in #359
• @b-kamphorst made their first contribution in #365
• @cccntu made their first contribution in #377
• @statelesshz made their first contribution in #372
• @lachlancahill made their first contribution in #388
• @Wauplin made their first contribution in #417
• @pcuenca made their first contribution in #420
• @musicinmybrain made their first contribution in #416
• @zeux made their first contribution in #404
• @hayden-donnelly made their first contribution in #410

Full Changelog: v0.3.2...v0.4.2rc0

v0.4.1

What's Changed

• Back on dev. by @Narsil in #313
• Clarify specification around endianness and header padding by @akx in #306
• Adding new target for m1 by @Narsil in #317
• Fixing macos 13 release script (by prebuilding). by @Narsil in #315
• Cache dir all the time. by @Narsil in #319
• Supporting whisper conversion. by @Narsil in #320
• add tensorshare to featured projects by @chainyo in #322
• Fix Bench GH action. by @Narsil in #323
• Yaml is nice. by @Narsil in #325
• Fixing big endian. by @Narsil in #327
• Fixing boolean + numpy > 1.20 by @Narsil in #326
• Add NumPy for all backends dependencies by @chainyo in #332
• impl View for TensorView by @coreylowman in #329
• Reworked the release script entirely by @Narsil in #334
• Preparing small patch release (Big Endian fix). by @Narsil in #335
• Temporary revert of the breaking change (keep it for 0.4.0). by @Narsil in #336
• Don't release rust versions on RC by @Narsil in #337
• Fixing release script. by @Narsil in #338
• Python 3.8 for arm by @Narsil in #339
• Win64. by @Narsil in #342
• win64 by @Narsil in #343
• Trying out maturin. by @Narsil in #344
• Reduce memory allocations by @cospectrum in #350
• [docs] Convert weights by @stevhliu in #345
• Update types for load_model filename by @kevinhu in #356
• Fixing release script for windows aarch64 and 3.12 by @Narsil in #348
• Minor docstring edit to flax.py by @paulbricman in #359
• Stop recreating the hashmap all the time. by @Narsil in #363
• fix: add py.typed by @b-kamphorst in #365
• Preparing a new release (0.4.0). by @Narsil in #366
• Supporting bfloat16 for tensorflow + jax (was failing because of intermediary numpy). by @Narsil in #382
• Fix typo by @cccntu in #377
• Support fp8_e4m3/fp8_e5m2 by @Narsil in #383
• Better convert. by @Narsil in #384
• Ignore closed PRs to avoid spam. by @Narsil in #385
• Adding support for Ascend NPU by @statelesshz in #372
• Fix convert. by @Narsil in #390

New Contributors

• @akx made their first contribution in #306
• @chainyo made their first contribution in #322
• @coreylowman made their first contribution in #329
• @cospectrum made their first contribution in #350
• @stevhliu made their first contribution in #345
• @kevinhu made their first contribution in #356
• @paulbricman made their first contribution in #359
• @b-kamphorst made their first contribution in #365
• @cccntu made their first contribution in #377
• @statelesshz made their first contribution in #372

Full Changelog: v0.3.2...v0.4.1

v0.4.1rc1

What's Changed

• Back on dev. by @Narsil in #313
• Clarify specification around endianness and header padding by @akx in #306
• Adding new target for m1 by @Narsil in #317
• Fixing macos 13 release script (by prebuilding). by @Narsil in #315
• Cache dir all the time. by @Narsil in #319
• Supporting whisper conversion. by @Narsil in #320
• add tensorshare to featured projects by @chainyo in #322
• Fix Bench GH action. by @Narsil in #323
• Yaml is nice. by @Narsil in #325
• Fixing big endian. by @Narsil in #327
• Fixing boolean + numpy > 1.20 by @Narsil in #326
• Add NumPy for all backends dependencies by @chainyo in #332
• impl View for TensorView by @coreylowman in #329
• Reworked the release script entirely by @Narsil in #334
• Preparing small patch release (Big Endian fix). by @Narsil in #335
• Temporary revert of the breaking change (keep it for 0.4.0). by @Narsil in #336
• Don't release rust versions on RC by @Narsil in #337
• Fixing release script. by @Narsil in #338
• Python 3.8 for arm by @Narsil in #339
• Win64. by @Narsil in #342
• win64 by @Narsil in #343
• Trying out maturin. by @Narsil in #344
• Reduce memory allocations by @cospectrum in #350
• [docs] Convert weights by @stevhliu in #345
• Update types for load_model filename by @kevinhu in #356
• Fixing release script for windows aarch64 and 3.12 by @Narsil in #348
• Minor docstring edit to flax.py by @paulbricman in #359
• Stop recreating the hashmap all the time. by @Narsil in #363
• fix: add py.typed by @b-kamphorst in #365
• Preparing a new release (0.4.0). by @Narsil in #366
• Supporting bfloat16 for tensorflow + jax (was failing because of intermediary numpy). by @Narsil in #382
• Fix typo by @cccntu in #377
• Support fp8_e4m3/fp8_e5m2 by @Narsil in #383

New Contributors

• @akx made their first contribution in #306
• @chainyo made their first contribution in #322
• @coreylowman made their first contribution in #329
• @cospectrum made their first contribution in #350
• @stevhliu made their first contribution in #345
• @kevinhu made their first contribution in #356
• @paulbricman made their first contribution in #359
• @b-kamphorst made their first contribution in #365
• @cccntu made their first contribution in #377

Full Changelog: v0.3.2...v0.4.1rc1

v0.4.0

Highlights

• Revamped the build pipeline to use maturin
• Reduced allocations (speedup for small safetensors files with many small tensors)

What's Changed

• Back on dev. by @Narsil in #313
• Clarify specification around endianness and header padding by @akx in #306
• Adding new target for m1 by @Narsil in #317
• Fixing macos 13 release script (by prebuilding). by @Narsil in #315
• Cache dir all the time. by @Narsil in #319
• Supporting whisper conversion. by @Narsil in #320
• add tensorshare to featured projects by @chainyo in #322
• Fix Bench GH action. by @Narsil in #323
• Yaml is nice. by @Narsil in #325
• Fixing big endian. by @Narsil in #327
• Fixing boolean + numpy > 1.20 by @Narsil in #326
• Add NumPy for all backends dependencies by @chainyo in #332
• impl View for TensorView by @coreylowman in #329
• Reworked the release script entirely by @Narsil in #334
• Preparing small patch release (Big Endian fix). by @Narsil in #335
• Temporary revert of the breaking change (keep it for 0.4.0). by @Narsil in #336
• Don't release rust versions on RC by @Narsil in #337
• Fixing release script. by @Narsil in #338
• Python 3.8 for arm by @Narsil in #339
• Win64. by @Narsil in #342
• win64 by @Narsil in #343
• Trying out maturin. by @Narsil in #344
• Reduce memory allocations by @cospectrum in #350
• [docs] Convert weights by @stevhliu in #345
• Update types for load_model filename by @kevinhu in #356
• Fixing release script for windows aarch64 and 3.12 by @Narsil in #348
• Minor docstring edit to flax.py by @paulbricman in #359
• Stop recreating the hashmap all the time. by @Narsil in #363
• fix: add py.typed by @b-kamphorst in #365
• Preparing a new release (0.4.0). by @Narsil in #366

New Contributors

• @akx made their first contribution in #306
• @chainyo made their first contribution in #322
• @coreylowman made their first contribution in #329
• @cospectrum made their first contribution in #350
• @stevhliu made their first contribution in #345
• @kevinhu made their first contribution in #356
• @paulbricman made their first contribution in #359
• @b-kamphorst made their first contribution in #365

Full Changelog: v0.3.2...v0.4.0

v0.4.0rc1

Highlights

• Revamped entire build pipeline
• Fixed a bug on small files containing many small tensors allocating too much

What's Changed

• Back on dev. by @Narsil in #313
• Clarify specification around endianness and header padding by @akx in #306
• Adding new target for m1 by @Narsil in #317
• Fixing macos 13 release script (by prebuilding). by @Narsil in #315
• Cache dir all the time. by @Narsil in #319
• Supporting whisper conversion. by @Narsil in #320
• add tensorshare to featured projects by @chainyo in #322
• Fix Bench GH action. by @Narsil in #323
• Yaml is nice. by @Narsil in #325
• Fixing big endian. by @Narsil in #327
• Fixing boolean + numpy > 1.20 by @Narsil in #326
• Add NumPy for all backends dependencies by @chainyo in #332
• impl View for TensorView by @coreylowman in #329
• Reworked the release script entirely by @Narsil in #334
• Preparing small patch release (Big Endian fix). by @Narsil in #335
• Temporary revert of the breaking change (keep it for 0.4.0). by @Narsil in #336
• Don't release rust versions on RC by @Narsil in #337
• Fixing release script. by @Narsil in #338
• Python 3.8 for arm by @Narsil in #339
• Win64. by @Narsil in #342
• win64 by @Narsil in #343
• Trying out maturin. by @Narsil in #344
• Reduce memory allocations by @cospectrum in #350
• [docs] Convert weights by @stevhliu in #345
• Update types for load_model filename by @kevinhu in #356
• Fixing release script for windows aarch64 and 3.12 by @Narsil in #348
• Minor docstring edit to flax.py by @paulbricman in #359
• Stop recreating the hashmap all the time. by @Narsil in #363
• fix: add py.typed by @b-kamphorst in #365
• Preparing a new release (0.4.0). by @Narsil in #366

New Contributors

• @akx made their first contribution in #306
• @chainyo made their first contribution in #322
• @coreylowman made their first contribution in #329
• @cospectrum made their first contribution in #350
• @stevhliu made their first contribution in #345
• @kevinhu made their first contribution in #356
• @paulbricman made their first contribution in #359
• @b-kamphorst made their first contribution in #365

Full Changelog: v0.3.2...v0.4.0rc1