refactor: pass in IPLD Formats into the constructor

[vmx]

Instead of having IPLD using all currently available IPLD Format
implementations automatically, pass the Formats that should be
used into the constructor as options.

This will also decrease the bundle size as every user of IPLD can
decide which IPLD Formats should be used. By default
ipld-dag-cbor and ipld-dag-pb are used as those are the ones
also heavily used within the IPFS ecosystem.

BREAKING CHANGE: Not all IPLD Formats are included by default

By default only the ipld-dag-cbor and ipld-dag-pb IPLD Formats
are included. If you want to use other IPLD Formats you need
to pass them into the constructor.

The code to restore the old behaviour could look like this:

const ipldDagPb = require('ipld-dag-pb')
const ipldDagCbor = require('ipld-dag-cbor')
const ipldGit = require('ipld-git')
const ipldBitcoin = require('ipld-bitcoin')
const ipldEthAccountSnapshot = require('ipld-ethereum').ethAccountSnapshot
const ipldEthBlock = require('ipld-ethereum').ethBlock
const ipldEthBlockList = require('ipld-ethereum').ethBlockList
const ipldEthStateTrie = require('ipld-ethereum').ethStateTrie
const ipldEthStorageTrie = require('ipld-ethereum').ethStorageTrie
const ipldEthTx = require('ipld-ethereum').ethTx
const ipldEthtrie = require('ipld-ethereum').ethTxTrie
const ipldRaw = require('ipld-raw')
const ipldZcash = require('ipld-zcash')

…

const ipld = new Ipld({
  blockService: blockService,
  formats: [
    pldDagPb, ipldDagCbor, ipldGit, ipldBitcoin, ipldEthAccountSnapshot,
    ipldEthBlock, ipldEthBlockList, ipldEthStateTrie, ipldEthStorageTrie,
    ipldEthTx, ipldEthtrie, ipldRaw, ipldZcash
  ]
})

/cc @alanshaw @hugomrdias

[vmx]

It's not clear, but the parts of the CI that matter passed (and the failed part can't be retriggered).

[hugomrdias]

This should be enough here right ?

Suggested change

	const options = mergeOptions(IPLDResolver.defaultOptions, userOptions)
	const options = Object.assign({}, IPLDResolver.defaultOptions, userOptions)

[vmx]

I want a to have a deep copy. Example:

'use strict'

const mergeOptions = require('merge-options')

const oa1 = {main: {sub: 1}}
const oa2 = {main: {sub: 2}}
const oa = Object.assign({}, oa1, oa2)
oa2.main.sub = 3
console.log('oa:', oa)

const mo1 = {main: {sub: 1}}
const mo2 = {main: {sub: 2}}
const mo = mergeOptions(mo1, mo2)
mo2.main.sub = 3
console.log('mo:',  mo)

Output is:

oa: { main: { sub: 3 } }
mo: { main: { sub: 2 } }

See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign#Deep_Clone for more information about Object.assign().

[hugomrdias]

ok just wanted to make sure we really need deep copy here.

[vmx]

K, cool. I prefer code which doesn't lead to weird surprises in the long run (+ I've coded functional programming language for a looong tim)

[hugomrdias]

@vmx can i suggest https://www.npmjs.com/package/assign-deep instead of merge-options. Check https://codesandbox.io/s/742p0rjvy0, assign-deep is smaller and handles non-plain-objects values better like promises (merge-options gives empty object when you should get a Promise). We should use assign-deep everywhere.

[vmx]

@hugomrdias: I want to have a deep copy, that makes things easier to debug/reason about. assign-deep does only a shallow copy.

Though I'd love to see that we agree on something decent across all libs.

[vmx]

I just read ipfs/js-ipfs#1663 (comment). Perhaps it's not always a good idea to copy things. I think I need to put more thought into this. Though I'll merge this for now to keep things going.

[hugomrdias]

LGTM

[daviddias]

These PR should be done in tandem to PR to IPLD examples in js-ipfs and also add an example to this repo that shows how to add the multple formats.

[vmx]

@diasdavid I've done the js-ipfs change already locally. Once this change is merged and released. I will make PRs to those repos updating js-ipld with the corresponding changes. There's not point of keep too many PRs open for a long time.

[daviddias]

@vmx we've been working and encouraging on working with Open PRs since the beginning of the project. It helps track work and get others to see and experiment as well. When there is a PR that requires many other PRs, then we link then and make sure that we explain that on the top level PR.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@f1c2e12). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master     #164   +/-   ##
=========================================
  Coverage          ?   89.13%           
=========================================
  Files             ?        1           
  Lines             ?      184           
  Branches          ?        0           
=========================================
  Hits              ?      164           
  Misses            ?       20           
  Partials          ?        0

Impacted Files	Coverage Δ
src/index.js	89.13% <100%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f1c2e12...669ee0c. Read the comment docs.

[vmx]

@diasdavid Projects I've identified that need changes if this change is released:

• js-ipfs: [WIP] chore: update IPLD to latest version ipfs/js-ipfs#1652
• js-ipfs-unixfs-engine: chore: update IPLD to latest version ipfs-inactive/js-ipfs-unixfs-engine#232
• ipld-explorer: doesn't need changes, only ipld-explorer-components does
• ipld-explorer-components:
  • Once Make it work with js-ipld 0.18.0 ipfs/ipld-explorer-components#5 is fixed, I'll look into making it work with a release where this PR is merged
• ipld-explorer-cli: wait for Tests are failing on master tableflip/ipld-explorer-cli#10 to be fixed. Then I can check if my local changes really work.

There is already an example on how to add other formats in the README.

[alanshaw]

This'll undo the perf work here #145 as IPFS (and potentially others) will have to require these formats upfront.

Can we load these on demand? This would actually be good for IPFS in browsers because we could exclude the implementations from the bundle but add loaders to load in the code when/if needed.

Something like this?:

// In IPFS:
ipld.support.add('git-raw', () => new Promise((resolve, reject) => {
  // Could replace with a XHR request. In Node.js we just:
  resolve(require('ipld-git'))
}))

// In ipld
let format
if (typeof this.support[codec] === 'function') {
  format = await this.support[codec]()
  this.support[codec] = format
} else if (this.support[codec]) {
  format = this.support[codec]
} else {
  throw new Error('missing format')
}

[vmx]

My idea was that you'd normally pass in only the things you need anyway. But then I found out that this is certainly not the case for the js-ifps CLI, where you want support for all formats.

That's a good idea. Thanks for also flushing out the code.