Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable ipnisync to be served over libp2p. #400

Merged
merged 13 commits into from
Sep 1, 2023
Merged

Conversation

gammazero
Copy link
Collaborator

@gammazero gammazero commented Aug 19, 2023

This allows all index-providers to upgrade without any changes to their configuration or any changes to infrastructure.

Changes

After upgrade, index-providers will start serving ipnisync and stop using data-transfer/graphsync to serve advertisements.

  • For providers serving over data-transfer/graphsync, they will switch to serving HTTP over their existing libp2p servers. They will also stop serving the old 'head' protocol over libp2p, and switch to serving the new HTTP head protocol.
  • For providers that are using plain HTTP, they will continue to serve over plain HTTP as the new ipnisync publisher supports that, using the existing configuration. They will also automatically serve HTTP over libp2p if a libp2p host is available.
  • Providers can be configured to serve only plain HTTP or only HTTP over libp2p.
  • Providers can be configured to serve over the discontinued data-transfer/graphsync as a temporary backup.
  • Mirror can serve over both HTTP and libp2p. If m.httpListenAddr has a value, then mirror will serve over HTTP on that address. If there is a libp2p Host, then the mirror will serve HTTP over libp2p using that host.

New Engine Options and Config

  • New engine option WithHttpNoLibp2p: If true, disables serving HTTP over libp2p and only serves plain HTTP.
  • New config item: config.Ingest.HttpPublisher.NoLibp2p: NoLibp2p disables serving HTTP over libp2p if true. Set this to true to publish over plain HTTP only. This configures the option WithHttpNoLibp2p
  • If config.Ingest.PublisherKind is set to "dtsync", a warning is logged.

@codecov-commenter
Copy link

codecov-commenter commented Aug 19, 2023

Codecov Report

Patch coverage: 40.98% and project coverage change: -0.16% ⚠️

Comparison is base (929faef) 50.15% compared to head (7c16315) 50.00%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #400      +/-   ##
==========================================
- Coverage   50.15%   50.00%   -0.16%     
==========================================
  Files          61       61              
  Lines        5022     5062      +40     
==========================================
+ Hits         2519     2531      +12     
- Misses       2185     2214      +29     
+ Partials      318      317       -1     
Files Changed Coverage Δ
cmd/provider/init.go 0.00% <0.00%> (ø)
cmd/provider/internal/config/httppublisher.go 20.00% <0.00%> (-3.53%) ⬇️
engine/options.go 62.28% <ø> (ø)
engine/engine.go 52.52% <44.44%> (+0.17%) ⬆️
cmd/provider/internal/config/ingest.go 52.63% <100.00%> (ø)
mirror/mirror.go 54.82% <100.00%> (-0.58%) ⬇️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gammazero gammazero force-pushed the ipnisync-with-libp2phttp branch 2 times, most recently from 581fccd to e916156 Compare August 24, 2023 15:33
@gammazero gammazero requested a review from masih September 1, 2023 04:55
@gammazero gammazero merged commit 7ed5da5 into main Sep 1, 2023
10 checks passed
@gammazero gammazero deleted the ipnisync-with-libp2phttp branch September 1, 2023 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants