v2.0.0

What's Changed

❗ Breaking changes

CID in CAR block keys and CID DAG-CBOR are now in raw byte form instead of stringified one!

• Encode and decode CID as bytes (major 2) instead of string (major 3) by @MarshalX in #36

⚡ Bunch of new optimizations

• Optimize duplicate check by @MarshalX in #34
• Optimize type checking by @MarshalX in #37
• Optimize keys duplication check by @MarshalX in #39
• Optimize array creation by @MarshalX in #40
• Optimize string encoding by @MarshalX in #41

🆕 Features

• Add CID encoding (encode_cid(bytes) -> str) by @MarshalX in #38
• Add DAG-CBOR profiling by @MarshalX in #35

Other

• Fix upload to PyPI by @MarshalX in #33
• Update CodSpeed to v3 by @MarshalX in #47
• Update PyO3 to 0.22.2 by @MarshalX in #48
• Lock rust toolchain on v1.76.0 because higher versions segfaults during PGO gather by @MarshalX in #46

🏎️ Performance comparison. Before (v1.2.3) VS After (v2.0.0):

Raw bench results

Before (v1.2.3):

Hello World Decode:
===================
libipld  : 144 ns

Hello World Encode:
===================
libipld  : 150 ns

Realistic Decode Tests:
=======================
canada.json.dagcbor            libipld  : 5.11 ms (197.31 MB/s)
citm_catalog.json.dagcbor      libipld  : 3.43 ms (95.31 MB/s)
github.json.dagcbor            libipld  : 0.17 ms (270.36 MB/s)
twitter.json.dagcbor           libipld  : 2.15 ms (178.96 MB/s)

Realistic Decode Car Tests:
===========================
atproto_repo.car               libipld  : 119.97 ms (154.22 MB/s)

Realistic Encode Tests:
=======================
canada.json.dagcbor            libipld  : 1.19 ms (848.65 MB/s)
citm_catalog.json.dagcbor      libipld  : 1.66 ms (196.42 MB/s)
github.json.dagcbor            libipld  : 0.11 ms (429.15 MB/s)
twitter.json.dagcbor           libipld  : 0.87 ms (443.77 MB/s)

After (v2.0.0):

Hello World Decode:
===================
libipld  : 124 ns

Hello World Encode:
===================
libipld  : 116 ns

Realistic Decode Tests:
=======================
canada.json.dagcbor            libipld  : 4.65 ms (216.57 MB/s)
citm_catalog.json.dagcbor      libipld  : 2.96 ms (110.37 MB/s)
github.json.dagcbor            libipld  : 0.15 ms (311.00 MB/s)
twitter.json.dagcbor           libipld  : 1.75 ms (219.74 MB/s)

Realistic Decode Car Tests:
===========================
atproto_repo.car               libipld  : 92.44 ms (200.13 MB/s)

Realistic Encode Tests:
=======================
canada.json.dagcbor            libipld  : 1.00 ms (1011.47 MB/s)
citm_catalog.json.dagcbor      libipld  : 1.60 ms (203.57 MB/s)
github.json.dagcbor            libipld  : 0.06 ms (733.96 MB/s)
twitter.json.dagcbor           libipld  : 0.56 ms (686.31 MB/s)

Test stand: M1 Pro, Python 3.12.

I highly recommend using Python 3.12. The lower version will be slower.

P.S. Breaking change not only gives a performance boost but also aligns implementations with other languages.

Full Changelog: v1.2.3...v2.0.0

v1.2.3

What's Changed

⚡ Thanks to PGO and new Bound API in pyo3 v0.21 I achieved huge performance boost ones again! The pyo3 team did an awesome work! Many thanks to all contributors! Also, I implemented own CAR reader that gives +11%

• Migrate to pyo3 v0.21 Bound API by @MarshalX in #23
• Add PGO-optimized wheels by @MarshalX in #30, #21
• Add own implementation of CAR reader by @MarshalX in #28
• Add IPLD Codec Fixtures by @MarshalX in #19
• Add CAR and CID benchmarks by @MarshalX in #20
• Improve work with numbers by @MarshalX in #25
• Use refs in args and Bound in return type where possible by @MarshalX in #31

🏎️ Performance comparison. Before (v1.2.2) VS After (v1.2.3):

Raw bench results

Before (v1.2.2):

Hello World Decode:
===================
libipld  : 275 ns

Hello World Encode:
===================
libipld  : 237 ns

Realistic Decode Tests:
=======================
canada.json.dagcbor            libipld  : 7.51 ms (134.15 MB/s)
citm_catalog.json.dagcbor      libipld  : 4.97 ms (65.71 MB/s)
github.json.dagcbor            libipld  : 0.23 ms (201.43 MB/s)
twitter.json.dagcbor           libipld  : 2.70 ms (142.51 MB/s)

Realistic Decode Car Tests:
===========================
atproto_repo.car               libipld  : 170.09 ms (108.77 MB/s)

Realistic Encode Tests:
=======================
canada.json.dagcbor            libipld  : 1.89 ms (533.01 MB/s)
citm_catalog.json.dagcbor      libipld  : 2.86 ms (114.02 MB/s)
github.json.dagcbor            libipld  : 0.14 ms (335.64 MB/s)
twitter.json.dagcbor           libipld  : 1.07 ms (358.30 MB/s)

After (v1.2.3):

Hello World Decode:
===================
libipld  : 130 ns

Hello World Encode:
===================
libipld  : 143 ns

Realistic Decode Tests:
=======================
canada.json.dagcbor            libipld  : 5.24 ms (192.11 MB/s)
citm_catalog.json.dagcbor      libipld  : 3.49 ms (93.51 MB/s)
github.json.dagcbor            libipld  : 0.18 ms (257.51 MB/s)
twitter.json.dagcbor           libipld  : 2.11 ms (182.31 MB/s)

Realistic Decode Car Tests:
===========================
atproto_repo.car               libipld  : 123.72 ms (149.55 MB/s)

Realistic Encode Tests:
=======================
canada.json.dagcbor            libipld  : 1.22 ms (823.65 MB/s)
citm_catalog.json.dagcbor      libipld  : 1.69 ms (192.80 MB/s)
github.json.dagcbor            libipld  : 0.11 ms (410.94 MB/s)
twitter.json.dagcbor           libipld  : 0.80 ms (481.72 MB/s)

Test stand: M1 Pro, Python 3.12.

I highly recommend using Python 3.12. The lower version will be slower.

Full Changelog: v1.2.2...v1.2.3

v1.2.2

What's Changed

• Add unit tests and benchmarks by @MarshalX in #11
• Add CodSpeed by @MarshalX in #14
• Fix encode_multibase input data type by @MarshalX in #12
• Fix DAG CBOR string encoding by @MarshalX in #13
• Fix duplicated map keys check in decode_dag_cbor by @MarshalX in #16
• Fix non-string map keys check in decode_dag_cbor by @MarshalX in #17
• Fix DAG-CBOR map keys order for encoding and decoding by @MarshalX in #18

🏎️ Performance comparison

Before (v1.2.1):

canada.json.dagcbor            libipld  : 1.98 ms (508.99 MB/s)
citm_catalog.json.dagcbor      libipld  : 3.96 ms (82.46 MB/s)
twitter.json.dagcbor           libipld  : 1.92 ms (200.10 MB/s)

After (v1.2.2):

canada.json.dagcbor            libipld  : 2.04 ms (493.43 MB/s)
citm_catalog.json.dagcbor      libipld  : 2.76 ms (118.31 MB/s)
twitter.json.dagcbor           libipld  : 1.03 ms (371.93 MB/s)

CodSpeed Performance Report: #18 (comment)

Full Changelog: v1.2.1...v1.2.2

v1.2.1

What's Changed

• Improve performance of DAG-CBOR encoding by @MarshalX in #10

Encoding results compared to v1.2.0:

• canada.json.dagcbor x20.6
• citm_catalog.json.dagcbor x4.7
• twitter.json.dagcbor x4.1

Full Changelog: v1.2.0...v1.2.1

v1.2.0

What's Changed

• Add encode_dag_cbor by @MarshalX in #5
• Add profiling by @MarshalX in #6
• Improve performance of DAG-CBOR decoding (remove a lot of unnecessary allocs and drops) by @MarshalX in #8

Decoding results compared to v1.1.1:

• citm_catalog.json.dagcbor x2
• canada.json.dagcbor x1.64
• twitter.json.dagcbor x1.29

Decoding results compared to v1.1.0:

• citm_catalog.json.dagcbor x4.9
• canada.json.dagcbor x4.6
• twitter.json.dagcbor x4.37

Full Changelog: v1.1.1...v1.2.0

v1.1.1

What's Changed

• Add error handling by @MarshalX in #4
• Improve performance (at least x2 boost for CAR and DAG-CBOR decoding) by @MarshalX in #4
• Fix CID decoding in DAG-CBOR by @MarshalX in #4

Full Changelog: v1.1.0...v1.1.1

v1.1.0

What's Changed

• Add multibase encode and decode by @MarshalX in #3

Full Changelog: v1.0.1...v1.1.0

v1.0.1

What's Changed

• Add Python 3.12 by @MarshalX in #2

Full Changelog: v1.0.0...v1.0.1

v1.0.0

The first stable release

Features

• Decode DAG CBOR (decode_cid(str) -> dict)
• Decode CID (decode_dag_cbor(bytes) -> dict, decode_dag_cbor_multi(bytes) -> list[dict])
• Decode CAR (decode_car(bytes) -> tuple[dict, dict[str, dict]]). Returns a header and blocks mapped by CID.

by @MarshalX