It's (beyond) time to sunset the *-star protocols 🌅

[daviddias]

Long story short for context:

• Back in 2016, we created two shortcuts to enable js-ipfs to connect to other Browsers quickly, these were: https://www.npmjs.com/package/libp2p-webrtc-star & https://www.npmjs.com/package/libp2p-websocket-star.
• The star in the name comes from the fact that what they were doing was a pure mesh in which everyone listening to on that transport, would dial to everyone else in that transport.
• These were created in a time where there was no circuit-relay, delegated routing and dht implementation.
• These were never designed to be long term solutions, in fact, they were designed to be quick tools for demos. They have design flaws and worse, they end up re-implementing a lot of what libp2p already does (connection authentication, peer discovery, circuit-relay and rendezvous)

Proposal: Let's sunset the *-star protocols 🌅

To clarify:

• A) https://github.com/libp2p/js-libp2p-websocket-star && https://github.com/libp2p/js-libp2p-websocket-stardust are equivalent to listening in a circuit-relay address for dialing, and rendezvous protocol for discovering
• B) https://github.com/libp2p/js-libp2p-websocket-star-multi is equivalent to listening in multiple circuit-relay addresses
• C) https://github.com/libp2p/js-libp2p-webrtc-star/ is the same as A), but uses that connection to establish a WebRTC connection (SDP exchange)

Direct benefits:

• By cutting of the *-star transports, we would be saving the time from reinventing the wheel and also reduce the confusion from the users.
• Any IPFS node could then be the rendezvous or the relaying address (meaning that we can use the go-libp2p to handle with even more clients). More
  • We avoid spending time creating custom instrumentation and use instead the instrumentation that IPFS & libp2p have.
  • Any security, perf improvement or memory leak fix will be directly on libp2p and/or IPFS rather than external "temporary" code.
• People would not have to rely on a centralized of custom built infrastructure, instead, they could use it with any IPFS node.

Candidate Roadmap:

• Milestone 1: Sunset any variation of the websocket-star (star, stardust, multi, and so on)
  • Verify that https://github.com/libp2p/js-libp2p-rendezvous is complete and well tested
  • Remove any trace of *-star transport from js-libp2p (examples, docs)
  • Remove any trace of *-star transport from js-ipfs (examples, docs, base config)
  • Upgrade PeerPad to use rendezvous & circuit-relay to find other peers (dropping webrtc-star)
  • Make it default with js-ipfs
• Milestone 2: Sunset webrtc-star (easier, if the above is done first)
  • Create webrtc-circuit-relay-exchange (for lack of better name for now) that uses the Circuit Relay to exchange the SDP offer.
  • Test it with PeerPad
  • Make it default on js-ipfs

[backkem]

I suggest to make this effort implementation independent. That should make it fairly straightforward to adapt libp2p/go-libp2p-webrtc-direct and get webrtc working between the browser and Go clients.

cc libp2p/go-libp2p-webrtc-direct#14

[alanshaw]

I'm 👍 on this. Maybe I'm missing something but I think there's a piece missing...

Right now, websocket-star is the most reliable way for JS IPFS nodes in the browser to discover and communicate with each other. It kinda works because we don't have the whole of the IPFS network using it, and it cannot be discovered by the rest of the IPFS network (or used since there's no Go impl). This makes it great for demos, development and testing.

In the proposal , we'd be taking this resource away from the community without providing a replacement. Can we setup some circuit relays that JS IPFS nodes can use for demos, development and testing but the rest of the network doesn't discover and use on a day to day basis? I'm not familiar with how you'd go about doing that, it's possible right? I imagine it's something application developers would really need also.

[daviddias]

In the proposal , we'd be taking this resource away from the community without providing a replacement.

Not the case, let me clarify. The proposal contemplates:

• Deploying one or more IPFS nodes that are relays specifically for this purpose and not for the entire network (equivalent to the rendezvous/signalling point for any -star transport)
• Have js-ipfs nodes by default advertising those relay addresses
• Set up a node with the rendezvous protocol on and register every new js-ipfs node in that rendezvous

This is a direct replacement that will be transparent. In fact, users might see things getting faster and more reliable as they will be able to use go-libp2p (more performant) as a circuit relay. This way everything is a libp2p node, no custom pieces that reimpl the same thing.

[jacobheun]

Milestone 1: Sunset any variation of the websocket-star

https://github.com/libp2p/js-libp2p-rendezvous should be close as the initial spec didn't deviate much but needs to be updated and heavy testing still needs to be done. It doesn't have any federation at this point, but we don't need that to replace the star servers. Federation will need to be added in the future to better support decentralization of the node.

I have been thinking through the flow for doing connection upgrades for WebRTC signaling over the circuit. Part of that potentially uses the DHT (local or delegate) as a means of finding services instead of Rendezvous. This would allow us to spend more of our effort improving that, instead of fragmenting our focus. Here are my notes on how that flow could/would look:

Connection flow

• Node A discovers Relay r
  • How does Node A discovery Relay r?
    • This could be hard coded as a "Bootstrap" node
    • A DHT query could be made to find relays, assuming the Relay has registered as a provider of the relay protocol
    • The relay could use a Rendezvous server to find relays
    • ...?
• Node A connects to Relay r
• Once connected, Node A adds Relay r as a Priority Peer, so that it can maintain this connection for listening.
• Node A adds the Relay r, /p2p-circuit relay address to its list of advertised addresses.
• Node A performs an Identify Push update to all its connected peers to inform them of the address change.
• Node A advertises its purpose/support/content (such as telling the network it's a Peer Pad node)
  • How does this advertisement happen?
    • It could register with a Rendezvous server using the Rendezvous protocol.
    • It could register as a Peer Pad "provider" via the DHT.
    • ...?
• Node B joins the network and looks for other Peer Pad nodes.
  • How does it find them?
    • The same as process as Node A uses to discover Relay r as detailed above
• Node B discovers Node A and its advertised /p2p-circuit address.
• Node B connects to Relay r, to connect to Node A
• Node B and Node A are connected via Relay r
• Both nodes start a Connection Upgrade (such as WebRTC)
  • How is this handled?
    • We need a way to determine available upgrade transports
    • Once we agree on the transport we need to begin the WebRTC SDP exchange, or another upgrade process for the decided transport
• The nodes establish a direct connection to one another over the WebRTC (or other) transport
  • What happens if this fails? Should the nodes continue to leverage the Relay to exchange data? If so, it may be expensive for the relay node. If not, how can the nodes talk to one another?
    • If there was a distinction between nodes behaving as only "signaling relays", and nodes behaving as "data exchange relays", nodes could potentially be customized to support high/low connections or high/low bandwidth.
• Node B, as the dialer, requests a disconnect of the relayed connection
• Node A, ACKs the disconnect and disconnects the relayed connection
  • Node A must maintain its direct connection to Relay r, as it is using it as a listen address
• Node A and B are now free to exchange data over their direct connection

[mxinden]

@p-shahi with /webrtc and /webrtc-direct, can we close here?

[p-shahi]

Yes closing this, we're deprecating them here: #1488