Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

routing/http: feat: add streaming support #18

Merged
merged 10 commits into from
May 10, 2023
Merged

Conversation

guseggert
Copy link
Contributor

@guseggert guseggert commented Dec 21, 2022

This adds streaming support to the routing/v1 client and server by changing the interfaces to use iterators instead of slices, and adding content type negotation to the client and server with application/x-ndjson support.

routing/http/client/client.go Outdated Show resolved Hide resolved
routing/http/types/iter/iter.go Outdated Show resolved Hide resolved
routing/http/client/client.go Outdated Show resolved Hide resolved
routing/http/server/server.go Outdated Show resolved Hide resolved
routing/http/server/server.go Outdated Show resolved Hide resolved
routing/http/server/server.go Outdated Show resolved Hide resolved
routing/http/types/iter/iter.go Outdated Show resolved Hide resolved
@guseggert guseggert self-assigned this Jan 3, 2023
@guseggert guseggert force-pushed the feat/routing-http-iterator branch 2 times, most recently from 013e55a to b24943a Compare January 16, 2023 23:08
@codecov
Copy link

codecov bot commented Jan 26, 2023

Codecov Report

Merging #18 (12c75c2) into main (8059f18) will increase coverage by 0.24%.
The diff coverage is 74.57%.

❗ Current head 12c75c2 differs from pull request most recent head d21f0c8. Consider uploading reports for the commit d21f0c8 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #18      +/-   ##
==========================================
+ Coverage   47.79%   48.03%   +0.24%     
==========================================
  Files         274      279       +5     
  Lines       33237    33439     +202     
==========================================
+ Hits        15885    16063     +178     
- Misses      15673    15694      +21     
- Partials     1679     1682       +3     
Impacted Files Coverage Δ
routing/http/types/json/provider.go 49.33% <49.33%> (ø)
routing/http/types/ndjson/provider.go 52.38% <52.38%> (ø)
routing/http/contentrouter/contentrouter.go 51.19% <54.54%> (+1.19%) ⬆️
routing/http/server/server.go 68.25% <75.60%> (+5.32%) ⬆️
routing/http/types/iter/map.go 84.21% <84.21%> (ø)
routing/http/client/client.go 77.83% <86.56%> (+3.78%) ⬆️
routing/http/client/measures.go 54.34% <100.00%> (-8.16%) ⬇️
routing/http/types/iter/iter.go 100.00% <100.00%> (ø)
routing/http/types/iter/json.go 100.00% <100.00%> (ø)
routing/http/types/iter/slice.go 100.00% <100.00%> (ø)

... and 7 files with indirect coverage changes

@guseggert guseggert force-pushed the feat/routing-http-iterator branch 4 times, most recently from 498cfce to f21e4b7 Compare February 10, 2023 19:28
@BigLep
Copy link
Contributor

BigLep commented Feb 16, 2023

@guseggert : what's remaining so this can move out of draft?

@BigLep
Copy link
Contributor

BigLep commented May 9, 2023

2023-05-09 maintainer conversation:

This adds streaming support to the routing/v1 client and server by
changing the interfaces to use iterators instead of slices, and adding
content type negotation to the client and server.
Errors can just be packed inside of a struct, it over-complicates the
interface to require errors as separate return values everywhere.
This is always needed, so we can simplify things by just including it
in the main iterator interface.
These are practically the same, but cid.contact has already started
using ndjson so we're just going to use that instead.
@guseggert guseggert force-pushed the feat/routing-http-iterator branch from 93b5f90 to 296fcef Compare May 9, 2023 18:53
@guseggert
Copy link
Contributor Author

Re: testing with cid.contact, I talked with @masih and I was wrong--this is not implemented in cid.contact yet. It is for IPNI APIs but not for routing/v1. Confirmed by wiring into Kubo, req Accept=application/x-ndjson,application/json but resp Content-Type=application/json. Tracking issue for this on their side: ipni/indexstar#92

I think the best option is to just go ahead and ship this in Kubo and let cid.contact own the integration testing when they are ready to deploy.

Copy link
Member

@hacdias hacdias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! I also went over previous comments to check if everything was addressed. Let's finally merge this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

7 participants