Unable to load data on ipfs gateway

[FatNerdPeng]

Hello team,I successfully stored the data "hello world 101" on the helia node according to the example, but I am unable to display the data on the IPFS gateway as stated in the document. I am using a Brave browser, and my country has a firewall. To access the IPFS gateway, I need to use a VPN. If I do, I can successfully access the official website of IPFS, but I cannot obtain the "hello world 101" I uploaded on the gateway. It always displays 504. Is this related to the firewall or network?

[sfemat]

Same here. I tried latest chrome, brave, edge, firefox on windows 11 and firefox on ubuntu, a couple of vpns, and also added webtransport and webrtc. Enabling debug messages I can see some communication happening but no luck trying to get files from other nodes or gateways.

[achingbrain]

In order for the gateway to show data from your node it must be dialable from the public internet.

Are you able to use the https://ipfs-check.on.fleek.co/ website to confirm your node is routable from outside of your network?

[PedroBarcha]

@FatNerdPeng @sfemat were you able to solve it? I tried this example, and ran into the same issue as you: I get 504 when trying to access the content.

[achingbrain]

504 means the gateway IPFS node timed out before it could resolve your content.

This normally means your node is not dialable from public networks, or it could mean it's not published a provider record or it could be something else.

You can use the IPFS Content Diagnostic section of https://pl-diagnose.on.fleek.co/ to find out what has gone wrong.

[PedroBarcha]

@achingbrain thanks for the fast reply :)

I used IPFS check as you suggested and got this:

It looks like my Helia node isn't connecting to the IPFS network. Maybe I am not configuring it correctly? As far as I understood I don't need to run a kubo instance or smt like that, since the node app will take care of that for me, correct?

[achingbrain]

It looks like my Helia node isn't connecting to the IPFS network.

You can tell if you have network peers via:

helia.libp2p.getPeers()

If you have peers you are connected to the IPFS network in the sense that you can make DHT queries etc, but you need an externally dialable address if you want other peers to be able to contact you (e.g. to retrieve content from your node).

Do you see any externally routable address in the output of:

helia.libp2p.getMultiaddrs()

[PedroBarcha]

@achingbrain funny thing, getPeers() returns empty but getMultiaddrs() doesn't...

getPeers:  []
getMultiaddrs: [
  Multiaddr(/ip4/127.0.0.1/tcp/41153/p2p/12D3KooWCALnZevqFAh6Y7UkhjVJqK9iDxoGcbi7raRKDqfWuk64)
]

I am sorry but I am not sure how to proceed, this is pretty new ground for me .

[achingbrain]

The listen address needs to be 0.0.0.0 if you want other hosts to be able to connect to you (this will bind to every available interface, 127.0.0.1 only allows connections from the local host):

listen: ['/ip4/0.0.0.0/tcp/0']

If you are on a home network and your router supports UPnP you should also configure UPnP hole punching - once the external address appears in your multiaddrs list, this will allow external peers to dial your node:

import { uPnPNATService } from 'libp2p/upnp-nat'

const libp2p = await createLibp2p({
  // other stuff
  services: {
    // other stuff
    nat: uPnPNATService()
  }
})

[conanoc]

I have the same issue and uPnPNATService doesn't help.

peers:  []
addr:  [
  Multiaddr(/ip4/127.0.0.1/tcp/4001/p2p/12D3KooWMNemB5VpcBWrPWfJTqgZui558E7UmvG1z1PM9SwYXT7g),
  Multiaddr(/ip4/192.168.0.41/tcp/4001/p2p/12D3KooWMNemB5VpcBWrPWfJTqgZui558E7UmvG1z1PM9SwYXT7g),
  Multiaddr(/ip4/10.25.167.110/tcp/4001/p2p/12D3KooWMNemB5VpcBWrPWfJTqgZui558E7UmvG1z1PM9SwYXT7g)
]

getPeers() prints empty list on startup but after a while I get this:

{"peers":["QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt","QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN","QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ","QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa","QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"]}

https://ipfs.io/ipfs/CID still gets 504 Gateway timeout.
I'm running this with nodejs v20.5.0 on my laptop.
I think the peers should increase over time but it remains only the bootstrap peers. Any idea? @achingbrain

[juliangruber]

The listen address needs to be 0.0.0.0 if you want other hosts to be able to connect to you (this will bind to every available interface, 127.0.0.1 only allows connections from the local host):

I have changed the listening address to 0.0.0.0 as advised, and am running the script on a public server, however neither the ipfs nor the saturn gateway can load the data.

Full script:

import { noise } from '@chainsafe/libp2p-noise'
import { yamux } from '@chainsafe/libp2p-yamux'
import { unixfs } from '@helia/unixfs'
import { bootstrap } from '@libp2p/bootstrap'
import { tcp } from '@libp2p/tcp'
import { MemoryBlockstore } from 'blockstore-core'
import { MemoryDatastore } from 'datastore-core'
import { createHelia } from 'helia'
import { createLibp2p } from 'libp2p'
import { identifyService } from 'libp2p/identify'

async function createNode () {
  // the blockstore is where we store the blocks that make up files
  const blockstore = new MemoryBlockstore()

  // application-specific data lives in the datastore
  const datastore = new MemoryDatastore()

  // libp2p is the networking layer that underpins Helia
  const libp2p = await createLibp2p({
    datastore,
    addresses: {
      listen: [
        '/ip4/0.0.0.0/tcp/0'
      ]
    },
    transports: [
      tcp()
    ],
    connectionEncryption: [
      noise()
    ],
    streamMuxers: [
      yamux()
    ],
    peerDiscovery: [
      bootstrap({
        list: [
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN',
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa',
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb',
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt'
        ]
      })
    ],
    services: {
      identify: identifyService()
    }
  })

  return await createHelia({
    datastore,
    blockstore,
    libp2p
  })
}

// create two helia nodes
const node1 = await createNode()
const node2 = await createNode()

// connect them together
const multiaddrs = node2.libp2p.getMultiaddrs()
await node1.libp2p.dial(multiaddrs[0])

// create a filesystem on top of Helia, in this case it's UnixFS
const fs = unixfs(node1)

// we will use this TextEncoder to turn strings into Uint8Arrays
const encoder = new TextEncoder()

// add the bytes to your node and receive a unique content identifier
const cid = await fs.addBytes(encoder.encode('Hello World 307'))

console.log('Added file:', cid.toString())

// create a filesystem on top of the second Helia node
const fs2 = unixfs(node2)

// this decoder will turn Uint8Arrays into strings
const decoder = new TextDecoder()
let text = ''

// use the second Helia node to fetch the file from the first Helia node
for await (const chunk of fs2.cat(cid)) {
  text += decoder.decode(chunk, {
    stream: true
  })
}

console.log('Fetched file contents:', text)

[hsjo12]

Does anyone solve the problem ?.. I am stuck with this too.

[achingbrain]

The configuration here is not enough to publish content because it's missing a content-routing implementation - for publishing content this normally means the kad-dht implementation.

When you publish a bit of content the following needs to happen:

1. Your node needs a publicly dialable multiaddr
2. Add the content to your blockstore
3. Publish a provider record for your content (e.g. await helia.routing.provide(cid)) - this needs a content routing implementation like kad-dht module to work
4. Keep your node online, and listening on the same publicly dialable multiaddr

At this point, take the CID that you're trying to make retrievable and use a tool like https://helia-find-provs.on.fleek.co or https://pl-diagnose.on.fleek.co to verify you can find providers for it.

If you cannot, one of 1-4 above has not happened.

When the content is resolved, the following will happen:

1. Resolve the provider record for the CID (e.g. search the content routing - kad-dht or http delegates or something else)
2. Dial the multiaddrs from the provider record
3. Use bitswap to fetch the data