Better discovery and dialing heuristic

[dirkmc]

There are some issues with discovery:

• It takes a long time for gossip to start up (around 15 seconds). This is because
  • Rendezvous does not send us peers who are already connected to the rendezvous server (fixed in feat: include existing peers in response to ss-join libp2p/js-libp2p-websocket-star-rendezvous#27, deploy in Deploy latest ws-star.discovery.libp2p.io updates (0.3.0) ipfs/infra#458)
  • To connect the peer to floodsub, we need to dial the peer and then floodsub will receive a peer:connect event and make a connection. Currently we wait a random amount of time (up to 5s) before dialing a peer so as not to overload new peers.
    In practice the peer may already have been dialed by the websocket star transport.
    1. I propose a dialing heuristic that takes into account the number of peers known to the app.
  • When we dial a peer, we poll floodsub to see if it's interested in our app topic because floodsub did not inform us with an event (fixed in Emit an event when a peer's subscriptions change libp2p/js-libp2p-floodsub#60)
    2. We should listen for this event instead of polling
• We repeatedly dial the same peers
  This is because rendezvous repeatedly emits peer:connect (every 10s by default)
  3. I propose adding a wrapper around discovery that remembers which peers have been dialed
  I suggest the wrapper's memory should have a time limit, and a limit on the number of peers
• Occasionally we do not dial on peer events, I think because of a bug in the dialing mechanism
  4. Implementing the wrapper should take care of this

I'm happy to work on a PR to fix these issues

[dirkmc]

@pgte one thing I'm not sure about, what is the purpose of this block?

[victorb]

deploy in ipfs/infra#458)

Been deployed now

[dirkmc]

@victorb wow you guys don't mess around :) Thanks!

[pgte]

@dirkmc this is great, thank you!
Trying to answer your question:
When trying to find if a peer is interested in the app, we need to make sure that he's in the inbound or outbound connection list, so that the connection accounting is correct.

(Thinking about this again, I'm not sure of why the peer would be in the inbound list if it wasn't in the peer ring, but I may be missing something...)

[dirkmc]

Fixed by #176

Note that #176 will not pass tests until release v0.15.2 is included into libp2p2, and we include that version of libp2p2 in peer-star-app

[pgte]

@dirkmc this was fixed by #200, right?

[dirkmc]

@pgte Yes, it was fixed - closing