refactor(core): NewIPFSNode constructor

[btc]

RFC/RFCR

This PR is a gradual refactor of the core.IPFSNode constructor.

Primary objectives:

• Improve reliability by sharing more code between integration tests and the real node (dev/prod parity)
• Pave the way for centralized routing Grandcentral Routing (temporary while DHT is shown to be correct) #454 by making the Routing pluggable in a way that doesn't cause a complexity explosion

Before

core.NewIpfsNode(context.Context, *config.Config, online bool) (*core.IpfsNode, error)

After

core.NewIPFSNode(context.Context, ConfigOption) (*core.IpfsNode, error)

The old signature can be emulated as follows:

// Standard is a higher order function. (this is simply partial application)
n, err := core.NewIPFSNode(ctx, core.Standard(config, online))

The crux of ConfigOption is that it allows for any number of custom configuration options. It takes ideas from Pike's self-referential functions, but is tailored to our specific needs and constraints (we have a lot of these).

http://commandcenter.blogspot.nl/2014/01/self-referential-functions-and-design.html

Some type signatures I've considered so far:

// extend the node and return it 
// PITFALL: unclear responsibilities for field initialization
type ConfigOption func(context.Context, *core.IpfsNode) (*core.IpfsNode, error)
type ConfigOption func(context.Context, *core.IpfsNode) (error) // alternative

// return some smaller building block and use it to construct the node in NewIPFSNode
// PITFALL: verbosity/duplication, ownership concerns
type ConfigOption func(context.Context) (someNodeComponents/Repo, error)

I've hit a point in the implementation where it is no longer possible to share more of the constructor code without making tough trade-offs. This is a good time to invite others into the design process.

(big refactor with lots of commits but all commits should pass all tests)

[jbenet]

@briantigerchow this is great stuff! very much needed.

[jbenet]

this is beautiful. and yes, +1 on free functions. btw, lots of it can probably be abstracted into an interface (i.e. cat shouldnt need to know what a path.Resolver is, and instead could call ipfs.Resolve(path))

[jbenet]

❤️ want to merge this in asap.

[jbenet]

(btw build err)

[btc]

About to rebase on master and fixup the build error. Responding to your comments here since they'll disappear after the rebase:

+1 on free functions. btw, lots of it can probably be abstracted into an interface (i.e. cat shouldnt need to know what a path.Resolver is, and instead could call ipfs.Resolve(path))

• add Resolve method 98181e6

For the free functions, would you prefer to see them in another package or here in core?

Can we get rid of this online bool?

• change mode to an enum 3e10bec

not sure if this is still WIP, but the "connecting to bootstrap peers" part should be in this function too.

It originally used the bootstrap method, but was causing tests to hang. I'll put a TODO comment in the function as a reminder to address before merging.

func bootstrap(ctx context.Context, h host.Host, r *dht.IpfsDHT, ps peer.Peerstore, bootstrapPeers []peer.PeerInfo) error

• find bug and use bootstrap

(btw build err)

doh!

[jbenet]

For the free functions, would you prefer to see them in another package or here in core?

Another package.

but was causing tests to hang.

interesing-- any idea why?

btw, tests can now use ipfs bootstrap rm --all

[btc]

interesing-- any idea why?

@jbenet it's not hanging anymore. as of b0849fa it works fine.

[jbenet]

@briantigerchow LGTM, merge this in if you're ready.