Impl the http endpoints and cli cmds for PubSub

[daviddias]

#610 never really added support for PubSub to the http-api and the cli, just for core.

We do have tests and the impl is underway which should make things easier for someone to pick it up. To try it out you can enable these tests:

• https://github.com/ipfs/js-ipfs/blob/master/test/http-api/spec/pubsub.js
• https://github.com/ipfs/js-ipfs/blob/master/test/http-api/interface/pubsub.js
• https://github.com/ipfs/js-ipfs/blob/master/test/cli/pubsub.js

[richardschneider]

@diasdavid There are incompatibilities between go and js in the published message that is sent by pubsub publish.

• js has renamed the field topicIDs to topicCIDs, see the Message spec
• js encodes the field from in base58 whereas go uses base64

See the code in libp2p-floodsub.

js message

{
	"from": "QmYJ5Cm2dBEdDmCYkSqGoCViFfu3BE3xN8gJydMdtoVsdP",
	"data": "aGVsbG8gd29ybGQh",
	"seqno": "ZmUwMzAxMjUxYmU5ZDVhMzI3YmNiYTRkYWJmOTcwMzk4Y2ZmYTA4Mg==",
	"topicCIDs": [
		"net-ipfs-api-test-ce7465cc-ec60-428b-a5d6-ce783e5a462d"
	]
}

go message

{
	"from": "EiDzOYdzT4BE42JXwxVM8Q19w6tx30Bp2N3T7tOH/a2nCw==",
	"data": "aGVsbG8gd29ybGQh",
	"seqno": "FPX/1KPfLIA=",
	"topicIDs": [
		"net-ipfs-api-test-15924ff2-1748-4630-b896-e282bbf59ee6"
	]
}

[richardschneider]

Note that none of the above tests check a published message.

Also the test for pubsub ls is skipped.

[richardschneider]

Changes will have to be made

• flood-sub
• ipfs
• interace-ipfs-core
• ipfs-api

[daviddias]

@richardschneider great catch!

• https://github.com/libp2p/go-floodsub/blob/master/pb/rpc.proto
• https://github.com/libp2p/js-libp2p-floodsub/blob/master/src/message/rpc.proto.js

No idea how did this happen. This will be a breaking change. Wanna ship it in time for js-ipfs 0.27? An interop test would also be welcome

[richardschneider]

Working on changes now. when is js-ipfs 0.27 scheduled to ship.

[daviddias]

@richardschneider once I finish ipfs-inactive/interface-js-ipfs-core#162 which should be this week (so releasing next week)

[richardschneider]

No worries, will have changes in tonight/tomorrow.

Will the release include jsipfs files ls ...?

[daviddias]

@richardschneider it will :)

[richardschneider]

@diasdavid I need PR 49 reviewed and released before I begin the other tasks.

[daviddias]

@richardschneider you should be able to npm link to do more testing. To make sure that all bugs are detected, could you add some interop tests for pubsub as well? Thank you!

[richardschneider]

Yes, adding/changing tests in interface-ipfs-code.

I don't use npm link, will read up on it. Typically, use npm install to the github branch for testing other packages that are net yet ready for prime time.

But to run these tests, I need to use js-ipfs; which somewhere in the DLL hell uses libp2p-fload-sub.

[daviddias]

@richardschneider you should not need to change tests in interface-ipfs-core for interop, you should just to add some to this set https://github.com/ipfs/js-ipfs/tree/master/test/interop, they are run with https://github.com/ipfs/js-ipfs/blob/master/package.json#L36

[richardschneider]

interface-ipfs-core is WRONG. I'm making test changes to interface-ipfs-core because its checking for the wrong published message field name.

see ipfs-inactive/interface-js-ipfs-core@e750afe

[daviddias]

Understood! :)

[richardschneider]

@diasdavid I think pubsub is truely broken. I enabled https://github.com/ipfs/js-ipfs/blob/master/test/http-api/interface/pubsub.js and most tests fail; see test-log.txt

I'll start looking into it.

[richardschneider]

After a few hours, pubsub is NOT broken. The tests need to enable the pubsub experiment.