Transport + Swarm refactor

[Stebalien]

Moved from libp2p/go-libp2p-transport#21 for visibility.

Related to #279 (but this one is just about the refactor itself, not the overall task of QUIC + TLS + refactor).

State: In review. go-ipfs currently builds and runs (well) with these patches.

Branch feat/refactor in:

• ipfs/go-ipfs
  • PR open
  • Reviewed
  • Merged
• libp2p/go-libp2p-metrics
  • PR open - remove stream/conn adapters for transport refactor go-libp2p-metrics#12
  • Reviewed - @whyrusleeping
  • Merged
• libp2p/go-libp2p-swarm
  • PR open - refactor for new transports go-libp2p-swarm#63
  • Reviewed - @Kubuxu, @whyrusleeping, @magik6k
  • Merged
• libp2p/go-libp2p-circuit
  • PR open - Update for transport refactor go-libp2p-circuit#33
  • Reviewed - @vyzo
  • Merged
• libp2p/go-ws-transport
  • PR open - fix for transport refactor go-ws-transport#31
  • Reviewed
  • Merged
• libp2p/go-libp2p-kad-dht -- netutil changes + changed notification guarantees.
  • PR open - fixup for stream refactor go-libp2p-kad-dht#126
  • Reviewed
  • Merged
• libp2p/go-libp2p-netutil
  • PR open - move swarm construction helpers to a subpackage on the swarm go-libp2p-netutil#19
  • Reviewed
  • Merged
• multiformats/go-multiaddr-net
  • PR open - turn NetListener() method on listener interface into a helper method multiformats/go-multiaddr-net#37
  • Reviewed
  • Merged
• libp2p/go-libp2p-net
  • PR open - Updates for swarm refactor go-libp2p-net#23
  • Reviewed
  • Merged
• libp2p/go-libp2p
  • PR open - refactor for transport changes #299
  • Reviewed - @whyrusleeping (although this could use another review)
  • Merged
• libp2p/go-libp2p-transport
  • PR open - refactor transports to be fully featured go-libp2p-transport#29
  • Reviewed
  • Merged
• libp2p/go-floodsub -- netutil changes
  • PR open - update for transport refactor go-libp2p-pubsub#73
  • Reviewed
  • Merged
• libp2p/go-addr-util -- Remove AddrUsable (the swarm should be responsible for this...)
  • PR open - remove AddrUsable (and ws transport dep) go-addr-util#11
  • Reviewed - @whyrusleeping
  • Merged
• libp2p/go-libp2p-secio
  • PR open - make secio a proper go-conn-security transport go-libp2p-secio#28
  • Reviewed - @Kubuxu
  • Merged
• libp2p/go-libp2p-interface-pnet
  • PR open - protect net conns instead of transport conns go-libp2p-interface-pnet#9
  • Reviewed - @Kubuxu, @whyrusleeping
  • Merged
• libp2p/go-tcp-transport
  • PR open - refactor for transport interface changes go-tcp-transport#23
  • Reviewed
  • Merged
• libp2p/go-libp2p-pnet
  • PR open - wrap net conns, not transport conns go-libp2p-pnet#19
  • Reviewed - @whyrusleeping, @Kubuxu
  • Merged

Branch master in the new repos:

• libp2p/go-conn-security-multistream: multistream security multiplexer
  • Reviewed - @whyrusleeping
• libp2p/go-conn-security: stream security interfaces
  • Reviewed - @whyrusleeping
• libp2p/go-reuseport-transport: base transport for all TCP reuseport transports
  • Reviewed - @whurusleeping (done), @Kubuxu
• libp2p/go-libp2p-transport-upgrader: stream transport to full libp2p transport upgrader
  • Reviewed - @whyrusleeping

Deprecated packages:

• libp2p/go-libp2p-interface-conn: will merge what is left with go-libp2p-net, the distinction between this package, go-libp2p-net, and go-libp2p-transport always confused me anyways so I'm glad to see one of them go.
• libp2p/go-libp2p-conn: mostly replaced by go-stream-transport
• libp2p/go-peerstream: merged into swarm

Known Issues:

• Simultaneous dial problems are worse. This is caused by a combination of reuseport and TCP simultaneous open. The fact that it has gotten worse probably stems from a timing issue (lots of code cleanup so timing is probably more predictable).
• We no longer treat half-closed streams as fully closed for the purposes of tracking them in the swarm. This means users must listen for the EOF on a stream after calling close. I've added two new functions to go-libp2p-net to help with this (FullClose and AwaitEOF).

[Stebalien]

Done!

[dignifiedquire]

🎉 congrats @Stebalien for slaying another 🐉

[ghost]

All hail @Stebalien <3

[dirkmc]

Amazing 🎉