What's Changed
❗ Breaking changes
CID in CAR block keys and CID DAG-CBOR are now in raw byte form instead of stringified one!
⚡ Bunch of new optimizations
- Optimize duplicate check by @MarshalX in #34
- Optimize type checking by @MarshalX in #37
- Optimize keys duplication check by @MarshalX in #39
- Optimize array creation by @MarshalX in #40
- Optimize string encoding by @MarshalX in #41
🆕 Features
- Add CID encoding (
encode_cid(bytes) -> str) by @MarshalX in #38 - Add DAG-CBOR profiling by @MarshalX in #35
Other
- Fix upload to PyPI by @MarshalX in #33
- Update CodSpeed to v3 by @MarshalX in #47
- Update PyO3 to 0.22.2 by @MarshalX in #48
- Lock rust toolchain on v1.76.0 because higher versions segfaults during PGO gather by @MarshalX in #46
🏎️ Performance comparison. Before (v1.2.3) VS After (v2.0.0):
Raw bench results
Before (v1.2.3):
Hello World Decode:
===================
libipld : 144 ns
Hello World Encode:
===================
libipld : 150 ns
Realistic Decode Tests:
=======================
canada.json.dagcbor libipld : 5.11 ms (197.31 MB/s)
citm_catalog.json.dagcbor libipld : 3.43 ms (95.31 MB/s)
github.json.dagcbor libipld : 0.17 ms (270.36 MB/s)
twitter.json.dagcbor libipld : 2.15 ms (178.96 MB/s)
Realistic Decode Car Tests:
===========================
atproto_repo.car libipld : 119.97 ms (154.22 MB/s)
Realistic Encode Tests:
=======================
canada.json.dagcbor libipld : 1.19 ms (848.65 MB/s)
citm_catalog.json.dagcbor libipld : 1.66 ms (196.42 MB/s)
github.json.dagcbor libipld : 0.11 ms (429.15 MB/s)
twitter.json.dagcbor libipld : 0.87 ms (443.77 MB/s)
After (v2.0.0):
Hello World Decode:
===================
libipld : 124 ns
Hello World Encode:
===================
libipld : 116 ns
Realistic Decode Tests:
=======================
canada.json.dagcbor libipld : 4.65 ms (216.57 MB/s)
citm_catalog.json.dagcbor libipld : 2.96 ms (110.37 MB/s)
github.json.dagcbor libipld : 0.15 ms (311.00 MB/s)
twitter.json.dagcbor libipld : 1.75 ms (219.74 MB/s)
Realistic Decode Car Tests:
===========================
atproto_repo.car libipld : 92.44 ms (200.13 MB/s)
Realistic Encode Tests:
=======================
canada.json.dagcbor libipld : 1.00 ms (1011.47 MB/s)
citm_catalog.json.dagcbor libipld : 1.60 ms (203.57 MB/s)
github.json.dagcbor libipld : 0.06 ms (733.96 MB/s)
twitter.json.dagcbor libipld : 0.56 ms (686.31 MB/s)
Test stand: M1 Pro, Python 3.12.
I highly recommend using Python 3.12. The lower version will be slower.
P.S. Breaking change not only gives a performance boost but also aligns implementations with other languages.
Full Changelog: v1.2.3...v2.0.0
Contributors
MarshalX