feat: helia-lan-discovery example (#136)

* tmp: helia-lan-server-discovery testing

* feat: mdns discovery example works

* chore: rename example

* chore: conform to example standards

* chore: cleanup .gitignore

.github/workflows/ci.yml

@@ -36,6 +36,7 @@ jobs:
           - helia-vue
           - helia-webpack
           - helia-create-car
+          - helia-lan-discovery
     defaults:
       run:
         working-directory: examples/${{ matrix.project }}
@@ -99,6 +100,7 @@ jobs:
           - helia-vue
           - helia-webpack
           - helia-create-car
+          - helia-lan-discovery
     steps:
       - uses: convictional/trigger-workflow-and-wait@f69fa9eedd3c62a599220f4d5745230e237904be
         with:

examples/helia-lan-discovery/README.md

@@ -0,0 +1,147 @@
+<p align="center">
+  <a href="https://github.com/ipfs/helia" title="Helia">
+    <img src="https://github.com/raw/ipfs/helia/main/assets/helia.png" alt="Helia logo" width="300" />
+  </a>
+</p>
+
+<h3 align="center"><b>Helia LAN discovery</b></h3>
+
+<p align="center">
+  <img src="https://github.com/raw/jlord/forkngo/gh-pages/badges/cobalt.png" width="200">
+  <br>
+  <a href="https://ipfs.github.io/helia/modules/helia.html">Explore the docs</a>
+  ·
+  <a href="https://codesandbox.io/p/sandbox/infallible-haibt-e3lcd4">View Demo</a>
+  ·
+  <a href="https://github.com/ipfs-examples/helia-examples/issues">Report Bug</a>
+  ·
+  <a href="https://github.com/ipfs-examples/helia-examples/issues">Request Feature/Example</a>
+</p>
+
+## Table of Contents
+
+- [Table of Contents](#table-of-contents)
+- [About The Project](#about-the-project)
+- [Getting Started](#getting-started)
+  - [Prerequisites](#prerequisites)
+  - [Installation and Running example](#installation-and-running-example)
+- [Usage](#usage)
+- [Documentation](#documentation)
+- [Contributing](#contributing)
+- [Want to hack on IPFS?](#want-to-hack-on-ipfs)
+
+## About The Project
+
+- Read the [docs](https://ipfs.github.io/helia/modules/helia.html)
+- Look into other [examples](https://github.com/ipfs-examples/helia-examples) to learn how to spawn a Helia node in Node.js and in the Browser
+- Visit https://dweb-primer.ipfs.io to learn about IPFS and the concepts that underpin it
+- Head over to https://proto.school to take interactive tutorials that cover core IPFS APIs
+- Check out https://docs.ipfs.io for tips, how-tos and more
+- See https://blog.ipfs.io for news and more
+- Need help? Please ask 'How do I?' questions on https://discuss.ipfs.io
+
+## Getting Started
+
+### Prerequisites
+
+Make sure you have installed all of the following prerequisites on your development machine:
+
+- Git - [Download & Install Git](https://git-scm.com/downloads). OSX and Linux machines typically have this already installed.
+- Node.js - [Download & Install Node.js](https://nodejs.org/en/download/) and the npm package manager.
+
+### Installation and Running example
+
+```console
+> npm install
+
+# then in one terminal
+> npm run server
+
+# in another terminal
+> npm run client
+```
+
+To run the test
+
+```console
+npm run test
+```
+
+You should see all of the output from the server and client nodes, and the test should pass.
+
+## Usage
+
+This example shows how you can use mdns to connect two nodes. Either server/client node can be run first.
+
+Both scripts (src/server.js & src/client.js) will create a helia node, and subscribe to a known pubsub topic, and shut each other down (for ease of testing).
+
+Note: No WAN functionality is enabled, so only nodes on your local network can help with peer-discovery, and only nodes on your local network can be discovered as the code currently stands.. If you want to enable connecting to nodes outside of your WAN, you will need to connect to a bootstrap node.
+
+### General flow
+
+When you run these two scripts, the general flow works like this:
+
+1. Each node subscribes to the known pubsub topic.
+1. When the client node detects a subscription change on the pubsub topic, it will send a `wut-CID` message to the server node.
+1. The server node will respond to the `wut-CID` message with the string representation of a CID the server node is providing.
+1. The client node will request the content for that CID via `heliaDagCbor.get(CID.parse(msg))`
+1. Once the content is received, the client will publish a `done` message to the pubsub topic.
+1. The server node will detect the `done` message, and respond with a `done-ACK` message.
+1. The client node will detect the `done-ACK` message, respond with a `done-ACK` message of it's own, and shutdown after a timeout (to allow for the message to be sent)
+1. The server node will detect the `done-ACK` message, and shutdown immediately.
+
+### Testing
+
+Both scripts should be able to be run in any order, and the flow should work as expected. You can run `npm run test` to check this. The test will fail if the test runs for more than 10 seconds, or errors, but this failure mode is dependent upon the `timeout` command currently.
+
+### Further exploration of this example
+
+Some things to try:
+
+1. See if you can get ping/pong messages working without the nodes shutting down
+1. Run the server node from one computer on your local network, and the client node from another computer on your local network
+1. Try removing the shutdown code from the scripts, and see if you can get multiple clients to connect
+1. See if you can get the server to respond with a list of CIDs in it's blockstore, and have the client choose which one to request
+1. See if you can connect to bootstrap nodes with one of your nodes, and use the other node as a LAN only node.
+
+
+_For more examples, please refer to the [Documentation](#documentation)_
+
+## Documentation
+
+- [IPFS Primer](https://dweb-primer.ipfs.io/)
+- [IPFS Docs](https://docs.ipfs.io/)
+- [Tutorials](https://proto.school)
+- [More examples](https://github.com/ipfs-examples/helia-examples)
+- [API - Helia](https://ipfs.github.io/helia/modules/helia.html)
+- [API - @helia/unixfs](https://ipfs.github.io/helia-unixfs/modules/helia.html)
+
+## Contributing
+
+Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are **greatly appreciated**.
+
+1. Fork the IPFS Project
+2. Create your Feature Branch (`git checkout -b feature/amazing-feature`)
+3. Commit your Changes (`git commit -a -m 'feat: add some amazing feature'`)
+4. Push to the Branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+## Want to hack on IPFS?
+
+[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
+
+The IPFS implementation in JavaScript needs your help! There are a few things you can do right now to help out:
+
+Read the [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md) and [JavaScript Contributing Guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md).
+
+- **Check out existing issues** The [issue list](https://github.com/ipfs/helia/issues) has many that are marked as ['help wanted'](https://github.com/ipfs/helia/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3A%22help+wanted%22) or ['difficulty:easy'](https://github.com/ipfs/helia/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Adifficulty%3Aeasy) which make great starting points for development, many of which can be tackled with no prior IPFS knowledge
+- **Look at the [Helia Roadmap](https://github.com/ipfs/helia/blob/main/ROADMAP.md)** This are the high priority items being worked on right now
+- **Perform code reviews** More eyes will help
+  a. speed the project along
+  b. ensure quality, and
+  c. reduce possible future bugs
+- **Add tests**. There can never be enough tests
+
+[cid]: https://docs.ipfs.tech/concepts/content-addressing  "Content Identifier"
+[Uint8Array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array
+[libp2p]: https://libp2p.io

examples/helia-lan-discovery/package.json

@@ -0,0 +1,39 @@
+{
+  "name": "helia-lan-discovery",
+  "version": "1.0.0",
+  "private": true,
+  "type": "module",
+  "description": "Get two helia nodes to connect on LAN without supplying address",
+  "scripts": {
+    "server": "cross-env node src/server.js",
+    "client": "cross-env node src/client.js",
+    "test": "npm run test1 && npm run test2",
+    "test1": "timeout 10 npm run server & npm run client",
+    "test2": "timeout 10 npm run client & npm run server"
+  },
+  "dependencies": {
+    "@chainsafe/libp2p-gossipsub": "^11.0.1",
+    "@chainsafe/libp2p-noise": "^14.1.0",
+    "@chainsafe/libp2p-yamux": "^6.0.1",
+    "@helia/dag-cbor": "^1.0.3",
+    "@libp2p/autonat": "^1.0.5",
+    "@libp2p/circuit-relay-v2": "^1.0.7",
+    "@libp2p/identify": "^1.0.6",
+    "@libp2p/kad-dht": "^11.0.7",
+    "@libp2p/mdns": "^10.0.7",
+    "@libp2p/mplex": "^10.0.7",
+    "@libp2p/ping": "^1.0.6",
+    "@libp2p/tcp": "^9.0.7",
+    "@libp2p/webrtc": "^4.0.10",
+    "@libp2p/websockets": "^8.0.7",
+    "@libp2p/webtransport": "^4.0.10",
+    "helia": "^2.1.0",
+    "libp2p": "^1.0.10"
+  },
+  "devDependencies": {
+    "@types/jest": "^29.5.11",
+    "cross-env": "^7.0.3",
+    "jest": "^29.7.0",
+    "ts-jest": "^29.1.1"
+  }
+}

examples/helia-lan-discovery/src/client.js

@@ -0,0 +1,33 @@
+/* eslint-disable no-console */
+// client.js
+import { dagCbor } from '@helia/dag-cbor'
+import { CID } from 'multiformats/cid'
+import { comms, getHelia, pubSubTopic } from './utils.js'
+
+const helia = await getHelia('client')
+const heliaDagCbor = dagCbor(helia)
+
+helia.libp2p.services.pubsub.addEventListener('subscription-change', (evt) => {
+  if (helia.libp2p.services.pubsub.getSubscribers(pubSubTopic).length !== 0) {
+    // we're subscribed, and so is another node, so request the CID
+    helia.libp2p.services.pubsub?.publish(pubSubTopic, new TextEncoder().encode('wut-CID'))
+  }
+})
+
+await comms(helia, pubSubTopic, 'client', async (msg) => {
+  if (msg === 'pong') {
+    // helia.libp2p.services.pubsub?.publish(pubSubTopic, new TextEncoder().encode('done'))
+  } else if (msg === 'done-ACK') {
+    helia.libp2p.services.pubsub?.publish(pubSubTopic, new TextEncoder().encode('done-ACK'))
+    setTimeout(async () => {
+      await helia.stop()
+      process.exit(0)
+    }, 500)
+  } else {
+    // msg is a CID (response to wut-CID request), so fetch it
+    console.log('requesting CID: %s', msg)
+    const data = await heliaDagCbor.get(CID.parse(msg))
+    console.log('got CID data: ', data)
+    helia.libp2p.services.pubsub?.publish(pubSubTopic, new TextEncoder().encode('done'))
+  }
+})

examples/helia-lan-discovery/src/server.js

@@ -0,0 +1,27 @@
+/* eslint-disable no-console */
+// server.js
+import { dagCbor } from '@helia/dag-cbor'
+import { comms, getHelia, pubSubTopic } from './utils.js'
+
+const helia = await getHelia('server')
+
+const heliaDagCbor = dagCbor(helia)
+const uuid = `${new Date().toLocaleString()}: My test string that you only know if you're in the same pubsub channel as me and request my CID`
+const cid = await heliaDagCbor.add(uuid)
+const cidString = cid.toString()
+
+console.log('CID: %s', cidString)
+
+await comms(helia, pubSubTopic, 'server', async (msg) => {
+  if (msg === 'done') {
+    helia.libp2p.services.pubsub?.publish(pubSubTopic, new TextEncoder().encode('done-ACK'))
+  } else if (msg === 'ping') {
+    helia.libp2p.services.pubsub?.publish(pubSubTopic, new TextEncoder().encode('pong'))
+  } else if (msg === 'done-ACK') {
+    // other node sent done-ack and should have shut down, now it's our turn to shut down.
+    await helia.stop()
+    process.exit(0)
+  } else if (msg === 'wut-CID') {
+    helia.libp2p.services.pubsub?.publish(pubSubTopic, new TextEncoder().encode(cidString))
+  }
+})

examples/helia-lan-discovery/src/utils.js

@@ -0,0 +1,107 @@
+/* eslint-disable no-console */
+import { gossipsub } from '@chainsafe/libp2p-gossipsub'
+import { noise } from '@chainsafe/libp2p-noise'
+import { yamux } from '@chainsafe/libp2p-yamux'
+import { autoNAT as autoNATService } from '@libp2p/autonat'
+import { identify as identifyService } from '@libp2p/identify'
+import { kadDHT } from '@libp2p/kad-dht'
+import { mdns } from '@libp2p/mdns'
+import { ping as pingService } from '@libp2p/ping'
+import { tcp } from '@libp2p/tcp'
+import { webRTC, webRTCDirect } from '@libp2p/webrtc'
+import { webSockets } from '@libp2p/websockets'
+import { MemoryDatastore } from 'datastore-core'
+import { createHelia } from 'helia'
+import { bitswap } from 'helia/block-brokers'
+import { createLibp2p } from 'libp2p'
+
+// @ts-check
+
+export async function getHelia (clientName) {
+  const datastore = new MemoryDatastore()
+
+  const libp2p = await createLibp2p({
+    datastore,
+    connectionManager: {
+      minConnections: 1
+    },
+    addresses: {
+      /**
+       * you have to make sure that listening multiaddrs are announced for mdns to work
+       *
+       * @see https://github.com/libp2p/js-libp2p/blob/742915567749072aa784cf179ce9810f66ac6c6e/packages/peer-discovery-mdns/src/query.ts#L87-L89
+       * @see https://github.com/libp2p/js-libp2p/blob/742915567749072aa784cf179ce9810f66ac6c6e/packages/peer-discovery-mdns/src/mdns.ts#L92-L101
+       */
+      listen: [
+        '/ip4/0.0.0.0/webrtc',
+        '/ip4/0.0.0.0/ws',
+        '/ip4/0.0.0.0/tcp/0'
+      ]
+    },
+    transports: [
+      webRTC(),
+      webRTCDirect(),
+      webSockets(),
+      tcp()
+    ],
+    connectionEncryption: [
+      noise()
+    ],
+    streamMuxers: [
+      // mplex(),
+      yamux()
+    ],
+    peerDiscovery: [
+      mdns({
+        // broadcast: mdnsBroadcast
+      })
+    ],
+    services: {
+      identify: identifyService(),
+      ping: pingService({
+        protocolPrefix: 'ipfs'
+      }),
+      dht: kadDHT(),
+      pubsub: gossipsub(),
+      nat: autoNATService({
+        enabled: true
+      })
+    }
+  })
+
+  const helia = await createHelia({
+    blockBrokers: [
+      bitswap()
+    ],
+    datastore,
+    libp2p
+  })
+
+  helia.libp2p.addEventListener('peer:discovery', async (connection) => {
+    const peer = connection.detail.id
+    console.log('%s discovered peer: ', clientName, peer.toString())
+  })
+  helia.libp2p.addEventListener('peer:connect', async (connection) => {
+    console.log('%s connected to peer: ', clientName, connection.detail.toString())
+  })
+  return helia
+}
+
+export async function comms (helia, topic, prefix, onMessage) {
+  if (helia.libp2p.services.pubsub == null) {
+    return
+  }
+  if (onMessage == null) {
+    throw new Error('onMessage is required')
+  }
+  console.log('%s helia.libp2p.peerId %s subscribing to topic %s', prefix, helia.libp2p.peerId.toString(), topic)
+
+  helia.libp2p.services.pubsub?.subscribe(topic)
+  await helia.libp2p.services.pubsub?.addEventListener('message', async (evt) => {
+    const messageString = new TextDecoder().decode(evt.detail.data)
+    console.log('%s gossipsub:message received: %s', prefix, messageString)
+    await onMessage(messageString)
+  })
+}
+
+export const pubSubTopic = 'helia-lan-discovery'