IPFS paths hard coded in build system make bootstrapping impossible without source hacking

[tidux]

git master, DragonFly BSD, but applies to any system currently not supported

Type: bug
Area: build system
Priority: 4 for DragonFly BSD, 0 for currently supported systems

The build system currently relies on hard coded IPFS paths to precompiled tarballs of the dependencies instead of leveraging the Go language's built in system for fetching dependencies via go get. This results in 694 separate instances across 253 files that need to be patched out in order to bootstrap a port to a new kernel, or to build at all for said new kernel.

tidux@dflyvm:~/go/src/github.com/ipfs/go-ipfs > grep -rF gx/ipfs/Q . | wc -l
     694

tidux@dflyvm:~/go/src/github.com/ipfs/go-ipfs > grep -lrF gx/ipfs/Q . | sort | uniq | wc -l
     253

What exactly was wrong with using go get to build dependencies from source? All that requires is the Go language and any linked C libraries to be ported to the new OS. The gx and gx-go tools already work fine on DragonFly BSD when installed via go get -u, for example:

tidux@dflyvm:~/go/src/github.com/ipfs/go-ipfs > gx --version
gx version 0.8.0
tidux@dflyvm:~/go/src/github.com/ipfs/go-ipfs > gx-go --version
gx-go version 1.2.1

The dependencies appear to all be tracked via Godeps/Godeps.json as well, rendering the hard coded paths useless duplication of effort. I'm currently working on a shell script to traverse the repository and replace the IPFS links with standard go get imports.

[tidux]

Well I feel like an idiot for not checking the gx_bin and gx_go_bin variables set in the top level Makefile. Ignore the shell script mentioned earlier, this could be fixed with requiring "gx" and "gx-go" to be installed as dependencies when running go get -d github.com/ipfs/go-ipfs. Those are the only binaries involved.

HOW TO COMPLETE PORT TO DRAGONFLY BSD:

1. FUSE does not exist for DragonFly BSD yet. Append dragonfly to the first lines of fuse/node/mount_nofuse.go and core/commands/mount_nofuse.go to match uname checks.
2. Add a note to README.md noting special build instructions for DragonFly as work around until gx and gx-go are autoinstalled by go get -d:

#!/bin/sh
go get -u github.com/whyrusleeping/gx && go get -u github.com/whyrusleeping/gx-go
go get -d github.com/ipfs/go-ipfs && cd $GOROOT/src/github.com/ipfs/go-ipfs
gmake nofuse gx_bin=$(which gx) gx-go_bin=$(which gx-go)`

Proof that it works:

tidux@dflyvm:~ > uname -a
DragonFly dflyvm.domain.tld 4.6-RELEASE DragonFly v4.6.0.3.ge4caf-RELEASE #0: Thu Aug  4 14:41:29 CDT 2016     root@dflyvm.domain.tld:/usr/obj/usr/src/sys/X86_64_GENERIC  x86_64
tidux@dflyvm:~ > ipfs version
ipfs version 0.4.3-rc2

[Kubuxu]

Problem with native build system is that any breaking change in dependency chain will break the build, also it is entry point for some vulnerabilities. gx essentially performs version locking for Go dependencies. gx can be easily built on any architecture that supports Go.

Please also see #3051.

We are moving away from Godeps, but it isn't fast process (we have to version locks dependencies of our dependencies using gx).

[whyrusleeping]

closing as #3051 merged