-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: switch to DAG-CBOR #93
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>
lidel
force-pushed
the
refactor/dag-cbor
branch
from
September 18, 2022 21:17
ed39804
to
4b4e7ad
Compare
We now can fetch raw blocks from a public gateway. This will use public gateway if no API is provided. License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>
lidel
force-pushed
the
refactor/dag-cbor
branch
from
September 18, 2022 21:21
4b4e7ad
to
6c7d074
Compare
License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>
lidel
force-pushed
the
refactor/dag-cbor
branch
7 times, most recently
from
September 19, 2022 00:20
84802ca
to
76f1c2a
Compare
lidel
force-pushed
the
refactor/dag-cbor
branch
2 times, most recently
from
September 28, 2022 01:27
de35eff
to
51c9e99
Compare
License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>
lidel
force-pushed
the
refactor/dag-cbor
branch
from
September 28, 2022 01:43
51c9e99
to
4fabe56
Compare
This was referenced Oct 18, 2022
Merged
* Using async-await * feat: support IPFS_GATEWAY env ipfs/specs#280 Signed-off-by: Marcin Rataj <lidel@lidel.org> Signed-off-by: Marcin Rataj <lidel@lidel.org> Co-authored-by: Marcin Rataj <lidel@lidel.org>
I'm merging this to unblock @whizzzkid to clean up remaining items before we can release:
When we have that, I'll regenerate dataset and make the release. |
This was referenced Oct 19, 2022
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR aims to close #85 and switch b-tree away from expensive JSON stored in
Data
field ofdag-pb
and deprecatedobject
RPC API.TLDR: ~30% DAG size reduction
DAG-PB was 399424 bytes (~400MiB)
DAG-CBOR is 281089143 bytes (~281MiB)
👉 this means ~30% reduction 👌
TODO
ipfs.object.get
toipfs.block.get
ipfs.object.put
to CAR writerlinks
so b-tree can be traversed by generic IPLD libraries andipfs dag stat
in Kubodata
andlinks
to reason about node type (storage savings)node example/lookup.js 9.9.9.9
returns same resultipfs
is not CoreAPI instance, but a string, and if it is a valid gateway URL, use it for fetching raw blocks as application/vnd.ipld.rawip
dependency requiring native node moduleTests
npm test
-[ ] TBD - maybe we should only support gateways?
ipfs.block.get
from Kuboipfs.block.get
from JS-IPFSavailableGatewayUrl
and pass it similar to how we do it on explore page)