bpo-36819: Fix crashes in built-in encoders with weird error handlers #28593
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If the error handler returns position less or equal than the starting position of non-encodable characters, most of built-in encoders didn't properly re-size the output buffer. This led to out-of-bounds writes, and segfaults.
serhiy-storchaka
added
type-bug
|
I need to add more tests for exceptions raised when an error handler returns invalid result, and analyze decoders. |
vstinner
reviewed
Sep 29, 2021
There was a problem hiding this comment.
Honestly, I'm not sure that it's worth it to handle this weird case: https://bugs.python.org/issue36819#msg402826
|
There are existing tests for position out of range (even negative values). They just are not harsh enough to cause a crash. The code needs to be changed even if we restrict returned values for error handlers. |
|
I have added more tests and it is now ready for review. |
|
Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10. |
|
GH-92136 is a backport of this pull request to the 3.10 branch. |
|
GH-92137 is a backport of this pull request to the 3.9 branch. |
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
May 2, 2022
…pythonGH-28593) If the error handler returns position less or equal than the starting position of non-encodable characters, most of built-in encoders didn't properly re-size the output buffer. This led to out-of-bounds writes, and segfaults. (cherry picked from commit 18b07d7) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
miss-islington
added a commit
that referenced
this pull request
May 2, 2022
…GH-28593) If the error handler returns position less or equal than the starting position of non-encodable characters, most of built-in encoders didn't properly re-size the output buffer. This led to out-of-bounds writes, and segfaults. (cherry picked from commit 18b07d7) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
miss-islington
added a commit
that referenced
this pull request
May 2, 2022
…GH-28593) If the error handler returns position less or equal than the starting position of non-encodable characters, most of built-in encoders didn't properly re-size the output buffer. This led to out-of-bounds writes, and segfaults. (cherry picked from commit 18b07d7) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
hello-adam
pushed a commit
to hello-adam/cpython
that referenced
this pull request
Jun 2, 2022
…pythonGH-28593) If the error handler returns position less or equal than the starting position of non-encodable characters, most of built-in encoders didn't properly re-size the output buffer. This led to out-of-bounds writes, and segfaults. (cherry picked from commit 18b07d7) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If the error handler returns position less or equal than the starting
position of non-encodable characters, most of built-in encoders didn't
properly re-size the output buffer. This led to out-of-bounds writes,
and segfaults.
https://bugs.python.org/issue36819
#81000