Add CARv2 MultihashIndexSorted codec
#227
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Define a new codec for CARv2 `MultihashIndexSorted`. See: - ipld/go-car#217 - ipld/go-car#214
masih
force-pushed
the
masih/carv2-mh-idx-srtd
branch
from
37658aa
to
b708c24
Compare
masih
added a commit
to ipld/go-car
that referenced
this pull request
Sep 2, 2021
Implement a new CARv2 index that contains enough information to reconstruct the multihashes of the data payload, since `CarIndexSorted` only includes multihash digests. Note, this index intentionally ignores any given record with `multihash.IDENTITY` CID hash. Add a test that asserts offsets for the same CID across sorted index and new multihash sorted index are consistent. Add tests that assert marshal unmarshalling of the new index type is as expected, and it does not load records with `multihash.IDENTITY` digest. Note, there is a need for a multicodec to be defined for the new index type. For now TODOs are left since it requires coordination across repos. Relates to: - multiformats/multicodec#227 Fixes: - #214
masih
added a commit
to ipld/go-car
that referenced
this pull request
Sep 2, 2021
Implement a new CARv2 index that contains enough information to reconstruct the multihashes of the data payload, since `CarIndexSorted` only includes multihash digests. Note, this index intentionally ignores any given record with `multihash.IDENTITY` CID hash. Add a test that asserts offsets for the same CID across sorted index and new multihash sorted index are consistent. Add tests that assert marshal unmarshalling of the new index type is as expected, and it does not load records with `multihash.IDENTITY` digest. Note, there is a need for a multicodec to be defined for the new index type. For now TODOs are left since it requires coordination across repos. Relates to: - multiformats/multicodec#227 Fixes: - #214
rvagg
approved these changes
Sep 2, 2021
willscott
approved these changes
Sep 2, 2021
masih
added a commit
to multiformats/go-multicodec
that referenced
this pull request
Sep 2, 2021
Update submodule to `1bcdc083898abb3e92b132f951e0a2fe0dcd485b`. Run `go generate`. Note, the code generation includes other changes to the codec table that have been merged but not generated here. See: - multiformats/multicodec#227 - multiformats/multicodec@1bcdc08 - ipld/go-car#214
masih
added a commit
to multiformats/go-multicodec
that referenced
this pull request
Sep 2, 2021
Update submodule to `1bcdc083898abb3e92b132f951e0a2fe0dcd485b`. Run `go generate`. Note, the code generation includes other changes to the codec table that have been merged but not generated here. See: - multiformats/multicodec#227 - multiformats/multicodec@1bcdc08 - ipld/go-car#214
masih
added a commit
to multiformats/go-multicodec
that referenced
this pull request
Sep 2, 2021
Update submodule to `1bcdc083898abb3e92b132f951e0a2fe0dcd485b`. Run `go generate`. Note, the code generation includes other changes to the codec table that have been merged but not generated here. See: - multiformats/multicodec#227 - multiformats/multicodec@1bcdc08 - ipld/go-car#214
masih
added a commit
to ipld/go-car
that referenced
this pull request
Sep 2, 2021
Implement a new CARv2 index that contains enough information to reconstruct the multihashes of the data payload, since `CarIndexSorted` only includes multihash digests. Note, this index intentionally ignores any given record with `multihash.IDENTITY` CID hash. Add a test that asserts offsets for the same CID across sorted index and new multihash sorted index are consistent. Add tests that assert marshal unmarshalling of the new index type is as expected, and it does not load records with `multihash.IDENTITY` digest. Relates to: - multiformats/multicodec#227 Fixes: - #214
masih
added a commit
to ipld/go-car
that referenced
this pull request
Sep 7, 2021
Implement a new CARv2 index that contains enough information to reconstruct the multihashes of the data payload, since `CarIndexSorted` only includes multihash digests. The new index builds on top of the existing `IndexSorted` by adding an additional layer of grouping the multi-width indices by their multihash code. Note, this index intentionally ignores any given record with `multihash.IDENTITY` CID hash. Add a test that asserts offsets for the same CID across sorted index and new multihash sorted index are consistent. Add tests that assert marshal unmarshalling of the new index type is as expected, and it does not load records with `multihash.IDENTITY` digest. Relates to: - multiformats/multicodec#227 Fixes: - #214
masih
added a commit
to ipld/go-car
that referenced
this pull request
Sep 7, 2021
Implement a new CARv2 index that contains enough information to reconstruct the multihashes of the data payload, since `CarIndexSorted` only includes multihash digests. The new index builds on top of the existing `IndexSorted` by adding an additional layer of grouping the multi-width indices by their multihash code. Note, this index intentionally ignores any given record with `multihash.IDENTITY` CID hash. Add a test that asserts offsets for the same CID across sorted index and new multihash sorted index are consistent. Add tests that assert marshal unmarshalling of the new index type is as expected, and it does not load records with `multihash.IDENTITY` digest. Relates to: - multiformats/multicodec#227 Fixes: - #214
Jorropo
pushed a commit
to ipfs/boxo
that referenced
this pull request
Mar 22, 2023
Implement a new CARv2 index that contains enough information to reconstruct the multihashes of the data payload, since `CarIndexSorted` only includes multihash digests. The new index builds on top of the existing `IndexSorted` by adding an additional layer of grouping the multi-width indices by their multihash code. Note, this index intentionally ignores any given record with `multihash.IDENTITY` CID hash. Add a test that asserts offsets for the same CID across sorted index and new multihash sorted index are consistent. Add tests that assert marshal unmarshalling of the new index type is as expected, and it does not load records with `multihash.IDENTITY` digest. Relates to: - multiformats/multicodec#227 Fixes: - ipld/go-car#214 This commit was moved from ipld/go-car@42b9e28
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Define a new codec for CARv2
MultihashIndexSorted.See: