Skip to content
This repository has been archived by the owner on Aug 11, 2021. It is now read-only.

feat: support passing AbortSignal instances to the configured blockservice #267

Merged
merged 2 commits into from
Apr 23, 2020
Merged

feat: support passing AbortSignal instances to the configured blockservice #267

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c9fe11c
feat: support passing AbortSignals to the configured blockservice
achingbrain Apr 15, 2020
6488b8a
chore: wait a little longer before aborting request
achingbrain Apr 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 31 additions & 18 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<a href="https://ipld.io"><img src="https://ipld.io/img/ipld-logo.png" alt="IPLD hex logo" /></a>
</div>

# The JavaScript implementation of the IPLD
# The JavaScript implementation of the IPLD <!-- omit in toc -->

[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://protocol.ai)
[![](https://img.shields.io/badge/project-ipld-blue.svg?style=flat-square)](http://ipld.io/)
Expand All @@ -17,23 +17,23 @@

> The JavaScript implementation of the IPLD, InterPlanetary Linked-Data

## Project Status
## Project Status <!-- omit in toc -->

This project is considered stable, but alpha quality implementation. The IPLD strategy for persistence and integration with IPFS has evolved
since this package was created. This package will be deprecated once the new strategy is fully implemented. You can read more about
This project is considered stable, but alpha quality implementation. The IPLD strategy for persistence and integration with IPFS has evolved
since this package was created. This package will be deprecated once the new strategy is fully implemented. You can read more about
the new strategy in [Issue #260](https://github.com/ipld/js-ipld/issues/260)

[**IPLD Team Management**](https://github.com/ipld/team-mgmt)

## Tech Lead
## Tech Lead <!-- omit in toc -->

[Volker Mische](https://github.com/vmx)

## Lead Maintainer
## Lead Maintainer <!-- omit in toc -->

[Volker Mische](https://github.com/vmx)

## Table of Contents
## Table of Contents <!-- omit in toc -->

- [Install](#install)
- [Usage](#usage)
Expand All @@ -43,12 +43,12 @@ the new strategy in [Issue #260](https://github.com/ipld/js-ipld/issues/260)
- [`options.formats`](#optionsformats)
- [`options.loadFormat(codec)`](#optionsloadformatcodec)
- [`.put(node, format, options)`](#putnode-format-options)
- [`.putMany(nodes, format, options)`](#putmanynode-format-options)
- [`.resolve(cid, path)`](#resolvecid-path)
- [`.get(cid)`](#getcid)
- [`.getMany(cids)`](#getmanycids)
- [`.remove(cid)`](#removecid)
- [`.removeMany(cids)`](#removemanycids)
- [`.putMany(nodes, format, options)`](#putmanynodes-format-options)
- [`.resolve(cid, path, options)`](#resolvecid-path-options)
- [`.get(cid, options)`](#getcid-options)
- [`.getMany(cids, options)`](#getmanycids-options)
- [`.remove(cid, options)`](#removecid-options)
- [`.removeMany(cids, options)`](#removemanycids-options)
- [`.tree(cid, [path], [options])`](#treecid-path-options)
- [`.addFormat(ipldFormatImplementation)`](#addformatipldformatimplementation)
- [`.removeFormat(codec)`](#removeformatcodec)
Expand Down Expand Up @@ -181,6 +181,7 @@ const ipld = new Ipld({
- `hashAlg` (`multicodec`, default: hash algorithm of the given multicodec): the hashing algorithm that is used to calculate the CID.
- `cidVersion` (`number`, default: 1): the CID version to use.
- `onlyHash` (`boolean`, default: false): if true the serialized form of the IPLD Node will not be passed to the underlying block store.
- `signal` ([`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)): a signal that can be used to abort any long-lived operations that are started as a result of this operation.

Returns a Promise with the CID of the serialized IPLD Node.

Expand All @@ -195,58 +196,69 @@ Returns a Promise with the CID of the serialized IPLD Node.
- `hashAlg` (`multicodec`, default: hash algorithm of the given multicodec): the hashing algorithm that is used to calculate the CID.
- `cidVersion` (`number`, default: 1): the CID version to use.
- `onlyHash` (`boolean`, default: false): if true the serialized form of the IPLD Node will not be passed to the underlying block store.
- `signal` ([`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)): a signal that can be used to abort any long-lived operations that are started as a result of this operation.

Returns an async iterator with the CIDs of the serialized IPLD Nodes. The iterator will throw an exception on the first error that occurs.


### `.resolve(cid, path)`
### `.resolve(cid, path, options)`

> Retrieves IPLD Nodes along the `path` that is rooted at `cid`.

- `cid` (`CID`, required): the CID the resolving starts.
- `path` (`IPLD Path`, required): the path that should be resolved.
- `options` an optional object that may have the following properties:
- `signal` ([`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)): a signal that can be used to abort any long-lived operations that are started as a result of this operation.

Returns an async iterator of all the IPLD Nodes that were traversed during the path resolving. Every element is an object with these fields:
- `remainderPath` (`string`): the part of the path that wasn’t resolved yet.
- `value` (`*`): the value where the resolved path points to. If further traversing is possible, then the value is a CID object linking to another IPLD Node. If it was possible to fully resolve the path, `value` is the value the `path` points to. So if you need the CID of the IPLD Node you’re currently at, just take the `value` of the previously returned IPLD Node.


### `.get(cid)`
### `.get(cid, options)`

> Retrieve an IPLD Node.

- `cid` (`CID`): the CID of the IPLD Node that should be retrieved.
- `options` an optional object that may have the following properties:
- `signal` ([`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)): a signal that can be used to abort any long-lived operations that are started as a result of this operation.

Returns a Promise with the IPLD Node that correspond to the given `cid`.

Throws an error if the IPLD Node can’t be retrieved.


### `.getMany(cids)`
### `.getMany(cids, options)`

> Retrieve several IPLD Nodes at once.

- `cids` (`AsyncIterable<CID>`): the CIDs of the IPLD Nodes that should be retrieved.
- `options` an optional object that may have the following properties:
- `signal` ([`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)): a signal that can be used to abort any long-lived operations that are started as a result of this operation.

Returns an async iterator with the IPLD Nodes that correspond to the given `cids`.

Throws an error if a IPLD Node can’t be retrieved.


### `.remove(cid)`
### `.remove(cid, options)`

> Remove an IPLD Node by the given `cid`

- `cid` (`CID`): the CIDs of the IPLD Node that should be removed.
- `options` an optional object that may have the following properties:
- `signal` ([`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)): a signal that can be used to abort any long-lived operations that are started as a result of this operation.

Throws an error if the IPLD Node can’t be removed.


### `.removeMany(cids)`
### `.removeMany(cids, options)`

> Remove IPLD Nodes by the given `cids`

- `cids` (`AsyncIterable<CID>`): the CIDs of the IPLD Nodes that should be removed.
- `options` an optional object that may have the following properties:
- `signal` ([`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)): a signal that can be used to abort any long-lived operations that are started as a result of this operation.

Throws an error if any of the Blocks can’t be removed. This operation is not atomic, some Blocks might have already been removed.

Expand All @@ -259,6 +271,7 @@ Throws an error if any of the Blocks can’t be removed. This operation is not a
- `path` (`IPLD Path`, default: ''): the path to start to retrieve the other paths from.
- `options`:
- `recursive` (`bool`, default: false): whether to get the paths recursively or not. `false` resolves only the paths of the given CID.
- `signal` ([`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)): a signal that can be used to abort any long-lived operations that are started as a result of this operation.

Returns an async iterator of all the paths (as Strings) you could resolve into.

Expand Down
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
"homepage": "https://github.com/ipld/js-ipld#readme",
"license": "MIT",
"devDependencies": {
"abort-controller": "^3.0.0",
"aegir": "^21.0.1",
"bitcoinjs-lib": "^5.1.6",
"chai": "^4.2.0",
Expand Down
37 changes: 25 additions & 12 deletions src/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,11 +74,13 @@ class IPLDResolver {
*
* @param {CID} cid - the CID the resolving starts.
* @param {string} path - the path that should be resolved.
* @param {Object} [options] - Options is an object with the following properties.
* @param {AbortSignal} [options.signal] - A signal that can be used to abort any long-lived operations that are started as a result of this operation.
* @returns {Iterable.<Promise.<{remainderPath: string, value}>>} - Returns an async iterator of all the IPLD Nodes that were traversed during the path resolving. Every element is an object with these fields:
* - `remainderPath`: the part of the path that wasn’t resolved yet.
* - `value`: the value where the resolved path points to. If further traversing is possible, then the value is a CID object linking to another IPLD Node. If it was possible to fully resolve the path, value is the value the path points to. So if you need the CID of the IPLD Node you’re currently at, just take the value of the previously returned IPLD Node.
*/
resolve (cid, path) {
resolve (cid, path, options) {
if (!CID.isCID(cid)) {
throw new Error('`cid` argument must be a CID')
}
Expand All @@ -94,7 +96,7 @@ class IPLDResolver {
// get block
// use local resolver
// update path value
const block = await this.bs.get(cid)
const block = await this.bs.get(cid, options)
const result = format.resolver.resolve(block.data, path)

// Prepare for the next iteration if there is a `remainderPath`
Expand Down Expand Up @@ -125,10 +127,12 @@ class IPLDResolver {
* Get a node by CID.
*
* @param {CID} cid - The CID of the IPLD Node that should be retrieved.
* @param {Object} [options] - Options is an object with the following properties.
* @param {AbortSignal} [options.signal] - A signal that can be used to abort any long-lived operations that are started as a result of this operation.
* @returns {Promise.<Object>} - Returns a Promise with the IPLD Node that correspond to the given `cid`.
*/
async get (cid) {
const block = await this.bs.get(cid)
async get (cid, options) {
const block = await this.bs.get(cid, options)
const format = await this._getFormat(block.cid.codec)
const node = format.util.deserialize(block.data)

Expand All @@ -139,17 +143,19 @@ class IPLDResolver {
* Get multiple nodes back from an array of CIDs.
*
* @param {Iterable.<CID>} cids - The CIDs of the IPLD Nodes that should be retrieved.
* @param {Object} [options] - Options is an object with the following properties.
* @param {AbortSignal} [options.signal] - A signal that can be used to abort any long-lived operations that are started as a result of this operation.
* @returns {Iterable.<Promise.<Object>>} - Returns an async iterator with the IPLD Nodes that correspond to the given `cids`.
*/
getMany (cids) {
getMany (cids, options) {
if (!typical.isIterable(cids) || typeof cids === 'string' ||
Buffer.isBuffer(cids)) {
throw new Error('`cids` must be an iterable of CIDs')
}

const generator = async function * () {
for await (const cid of cids) {
yield this.get(cid)
yield this.get(cid, options)
}
}.bind(this)

Expand All @@ -165,6 +171,7 @@ class IPLDResolver {
* @param {number} [userOtions.hashAlg=hash algorithm of the given multicodec] - The hashing algorithm that is used to calculate the CID.
* @param {number} [userOptions.cidVersion=1] - The CID version to use.
* @param {boolean} [userOptions.onlyHash=false] - If true the serialized form of the IPLD Node will not be passed to the underlying block store.
* @param {AbortSignal} [userOptions.signal] - A signal that can be used to abort any long-lived operations that are started as a result of this operation.
* @returns {Promise.<CID>} - Returns the CID of the serialized IPLD Nodes.
*/
async put (node, format, userOptions) {
Expand Down Expand Up @@ -193,7 +200,7 @@ class IPLDResolver {

if (!options.onlyHash) {
const block = new Block(serialized, cid)
await this.bs.put(block)
await this.bs.put(block, options)
}

return cid
Expand All @@ -208,6 +215,7 @@ class IPLDResolver {
* @param {number} [userOtions.hashAlg=hash algorithm of the given multicodec] - The hashing algorithm that is used to calculate the CID.
* @param {number} [userOptions.cidVersion=1] - The CID version to use.
* @param {boolean} [userOptions.onlyHash=false] - If true the serialized form of the IPLD Node will not be passed to the underlying block store.
* @param {AbortSignal} [userOptions.signal] - A signal that can be used to abort any long-lived operations that are started as a result of this operation.
* @returns {Iterable.<Promise.<CID>>} - Returns an async iterator with the CIDs of the serialized IPLD Nodes.
*/
putMany (nodes, format, userOptions) {
Expand Down Expand Up @@ -251,10 +259,12 @@ class IPLDResolver {
* Remove an IPLD Node by the given CID.
*
* @param {CID} cid - The CID of the IPLD Node that should be removed.
* @param {Object} [options] - Options is an object with the following properties.
* @param {AbortSignal} [options.signal] - A signal that can be used to abort any long-lived operations that are started as a result of this operation.
* @return {Promise.<CID>} The CID of the removed IPLD Node.
*/
async remove (cid) { // eslint-disable-line require-await
return this.bs.delete(cid)
async remove (cid, options) { // eslint-disable-line require-await
return this.bs.delete(cid, options)
}

/**
Expand All @@ -264,17 +274,19 @@ class IPLDResolver {
* *not* atomic, some Blocks might have already been removed.
*
* @param {Iterable.<CID>} cids - The CIDs of the IPLD Nodes that should be removed.
* @param {Object} [options] - Options is an object with the following properties.
* @param {AbortSignal} [options.signal] - A signal that can be used to abort any long-lived operations that are started as a result of this operation.
* @return {Iterable.<Promise.<CID>>} Returns an async iterator with the CIDs of the removed IPLD Nodes.
*/
removeMany (cids) {
removeMany (cids, options) {
if (!typical.isIterable(cids) || typeof cids === 'string' ||
Buffer.isBuffer(cids)) {
throw new Error('`cids` must be an iterable of CIDs')
}

const generator = async function * () {
for await (const cid of cids) {
yield this.remove(cid)
yield this.remove(cid, options)
}
}.bind(this)

Expand All @@ -288,6 +300,7 @@ class IPLDResolver {
* @param {string} [offsetPath=''] - the path to start to retrieve the other paths from.
* @param {Object} [userOptions]
* @param {number} [userOptions.recursive=false] - whether to get the paths recursively or not. `false` resolves only the paths of the given CID.
* @param {AbortSignal} [userOptions.signal] - A signal that can be used to abort any long-lived operations that are started as a result of this operation.
* @returns {Iterable.<Promise.<String>>} - Returns an async iterator with paths that can be resolved into
*/
tree (cid, offsetPath, userOptions) {
Expand Down Expand Up @@ -335,7 +348,7 @@ class IPLDResolver {
if (treePaths.length === 0 && queue.length > 0) {
({ cid, basePath } = queue.shift())
const format = await this._getFormat(cid.codec)
block = await this.bs.get(cid)
block = await this.bs.get(cid, options)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to forward only the signal value and not all of the options object? This is really just a question.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a fair point, but the way we wrap modules inside modules, if we let the wrapping module be the gatekeeper of what gets passed on, it becomes tied to the API of that module more strongly.

This may cause problems down the line as we're potentially passing more arguments to the wrapped module than strictly necessary, but we trade off against flexibility in that case.


const paths = format.resolver.tree(block.data)
treePaths.push(...paths)
Expand Down
Loading