Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Treat asyncio resume_reading error as a closed connection #201

Merged
merged 9 commits into from
Oct 6, 2020

Conversation

cdeler
Copy link
Member

@cdeler cdeler commented Oct 1, 2020

Closes encode/httpx#1213

There is a gist in the issue, which can help you reproduce the error: link . You can also switch the async backend and check, that the error appears only when we asyncio backend.

The introduced fix mends the error, so that the error message caused by the exception looks like:

httpcore.RemoteProtocolError: peer closed connection without sending complete message body (received 917504 bytes, expected 1000000)

instead of the original stacktrace from the issue

There is a problem:
I cannot figure out how to make a unit test for this error

Copy link
Member

@florimondmanca florimondmanca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, okay, sounds like a sensible ugly fix to an asyncio weakness.

Note: have we tried to see if this reproduces on anyio (rubbing in asyncio)?

httpcore/_backends/asyncio.py Outdated Show resolved Hide resolved
@cdeler
Copy link
Member Author

cdeler commented Oct 2, 2020

@florimondmanca
Having updated the gist, I tried to reproduce the error versus "anyio" backend, but couldn't. I'll try to figure out if we can do something with it no way, from my point of view the problem is inside asyncio.sslproto. Technically we can try to replace TLS implementation for asyncio backend

PS
Thats miracle, thank you @agronholm for "anyio"

@cdeler cdeler force-pushed the fix-asyncio-ssl-connection-issue branch 3 times, most recently from e23e01d to 513b4b8 Compare October 2, 2020 10:03
Copy link
Member

@florimondmanca florimondmanca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello! I've got some nitpicks on the code comment… Other than that, I'm not sure how confident we are merging this change? Can you point here to a comment with a test setup that allows reproducing this 100% of the time, to verify this is fixed with this PR?

Comment on lines 140 to 146
# We have to return empty string there,
# due to one ugly bug in asyncio
# More information you can find in this issue:
# https://github.com/encode/httpx/issues/1213
#
# The empty byte-string returned from there caused
# httpcore.RemoteProtocolError at the top level
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Nit) Tweaking the leading spaces here…

Suggested change
# We have to return empty string there,
# due to one ugly bug in asyncio
# More information you can find in this issue:
# https://github.com/encode/httpx/issues/1213
#
# The empty byte-string returned from there caused
# httpcore.RemoteProtocolError at the top level
# We have to return empty string there,
# due to one ugly bug in asyncio
# More information you can find in this issue:
# https://github.com/encode/httpx/issues/1213
#
# The empty byte-string returned from there caused
# httpcore.RemoteProtocolError at the top level

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

httpcore/_backends/asyncio.py Outdated Show resolved Hide resolved
@cdeler
Copy link
Member Author

cdeler commented Oct 3, 2020

@florimondmanca

Can you point here to a comment with a test setup that allows reproducing this 100% of the time, to verify this is fixed with this PR?

I created a gist with the test file

https://gist.github.com/cdeler/86d99a42c5bfb522bc15cac8d27824aa

Just run the script using python3, also you can check that the bug is not reproducible under other backends (just run path_to_script --help to check other switches)

cdeler and others added 3 commits October 3, 2020 20:53
Co-authored-by: Florimond Manca <florimond.manca@gmail.com>

Co-authored-by: florimondmanca <florimond.manca@gmail.com>
Co-authored-by: Florimond Manca <florimond.manca@gmail.com>
@cdeler cdeler force-pushed the fix-asyncio-ssl-connection-issue branch from 7c84da4 to bdd8570 Compare October 3, 2020 17:55
@cdeler
Copy link
Member Author

cdeler commented Oct 5, 2020

@florimondmanca

Probably I can add the test from the gist to our CI

Copy link
Member

@tomchristie tomchristie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lovely, thanks!

@tomchristie tomchristie merged commit d93b6ff into encode:master Oct 6, 2020
@tomchristie tomchristie mentioned this pull request Oct 6, 2020
@cdeler cdeler deleted the fix-asyncio-ssl-connection-issue branch October 6, 2020 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

asyncio resume_reading bug exposed while streaming response.
3 participants