Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: check Graphsync retrievals too #56

Merged
merged 3 commits into from
Mar 8, 2024
Merged

Conversation

bajtos
Copy link
Member

@bajtos bajtos commented Mar 7, 2024

When the storage provider does not advertise HTTP retrieval, but advertises Grapsync retrievals, then let's perform the retrieval check.

This was my original intention when I implemented IPNI querying.

I discovered the problem by accident while working on #55. Unfortunately, there is no easy way how to write a reliable automated test for this, as we would need a CID that's guaranteed to have only Graphsync advertised to IPNI.

FWIW, we want to move to testing HTTP retrievals only and drop support for Graphsync (see #46). However, since we are not there yet, I feel this bug is still worth fixing.

Here is a script I used to verify the fix manually. Please not you need to include changes from #55 to be able to reproduce the problem.

import Spark from './lib/spark.js'

const spark = new Spark()
const stats = {
  byteLength: 0
}

await spark.executeRetrievalCheck({
  cid: 'bafybeidkqbga5lsv4b7uhp2lyvuqsx7topvawtsn7ypej3lz44abs55wfu',
  minerId: 'f02055540'
}, stats)

console.log('\nSTATS\n', stats)

When the storage provider does not advertise HTTP retrieval, but
advertises Grapsync retrievals, then let's perform the retrieval check.

This was my original intention when I implemented IPNI querying.

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>
@bajtos bajtos requested a review from juliangruber March 7, 2024 15:51
lib/spark.js Outdated
@@ -43,7 +43,7 @@ export default class Spark {
const { indexerResult, provider } = await queryTheIndex(retrieval.cid)
stats.indexerResult = indexerResult

if (indexerResult !== 'OK') return
if (indexerResult !== 'OK' && indexerResult !== 'HTTP_NOT_ADVERTISED') return
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (indexerResult !== 'OK' && indexerResult !== 'HTTP_NOT_ADVERTISED') return
if (!(indexerResult === 'OK' || indexerResult === 'HTTP_NOT_ADVERTISED')) return

I find this more understandable: Skip if the indexer result isn't ok or http_not_advertised. Alternatively:

Suggested change
if (indexerResult !== 'OK' && indexerResult !== 'HTTP_NOT_ADVERTISED') return
if (!['OK', 'HTTP_NOT_ADVERTISED'].includes(indexerResult)) return

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I decided to extract a named variable to make this even easier to understand.

    const providerFound = indexerResult === 'OK' || indexerResult === 'HTTP_NOT_ADVERTISED'
    if (!providerFound) return

Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>
@bajtos bajtos dismissed juliangruber’s stale review March 8, 2024 08:16

I addressed your comment

@bajtos bajtos merged commit ecba43d into main Mar 8, 2024
1 check passed
@bajtos bajtos deleted the test-graphsync-retrievals branch March 8, 2024 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ✅ done
Development

Successfully merging this pull request may close these issues.

2 participants