Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

process: fix two overflow cases in SourceMap VLQ decoding #31490

Closed
wants to merge 3 commits into from

Conversation

jridgewell
Copy link
Contributor

@jridgewell jridgewell commented Jan 24, 2020

These both have to do with extremely large numbers, so it's unlikely to cause a problem in practice. Still, correctness.

First, encoding -2147483648 in VLQ returns the value "B". When decoding, we get the value 1 after reading the base64. We then check if the first bit is set (it is) to see if we should negate it, then we shift all bits right once. Now, value will be 0 and negate will be true. So, we'd return -0. Which is a bug! -0 isn't -2147483648, and we've broken a round trip.

Second, encoding any number with the 31st bit set, we'd return the opposite sign. Let's use 1073741824. Encoding, we get "ggggggC". When decoding, we get the value -2147483648 after reading the base64. Notice, it's already negative (the 32nd bit is set, because the 31st was set and we shifted everything left once). We'd then check the first bit (it's not) and shift right. But we used >>, which does not shift the sign bit. We actually wanted >>>, which will. Because of that bug, we get back -1073741824 instead of the positive 1073741824. It's even worse if the 32nd and 31st bits are set, -1610612736 becomes 536870912 after a round trip.

I recently fixed the same two bugs in Closure Compiler: google/closure-compiler@584418eb

/cc @bcoe

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

These both have to do with extremely large numbers, so it's unlikely to
cause a problem in practice. Still, correctness.

First, encoding `-2147483648` in VLQ returns the value `"B"`. When
decoding, we get the value `1` after reading the base64. We then check
if the first bit is set (it is) to see if we should negate it, then we
shift all bits right once. Now, `value` will be `0` and `negate` will
be `true`. So, we'd return `-0`. Which is a bug! `-0` isn't
`-2147483648`, and we've broken a round trip.

Second, encoding any number with the 31st bit set, we'd return the
opposite sign. Let's use `1073741824`. Encoding, we get `"ggggggC"`.
When decoding, we get the value `-2147483648` after reading the base64.
Notice, it's already negative (the 32nd bit is set, because the 31st was
set and we shifted everything left once). We'd then check the first bit
(it's not) and shift right. But we used `>>`, which does not shift the
sign bit. We actually wanted `>>>`, which will. Because of that bug, we
get back `-1073741824` instead of the positive `1073741824`. It's even
worse if the 32nd and 31st bits are set, `-1610612736` becomes
`536870912` after a round trip.

I recently fixed the same two bugs in Closure Compiler:
google/closure-compiler@584418eb
@jridgewell jridgewell requested a review from bcoe January 24, 2020 08:00
@mscdex
Copy link
Contributor

mscdex commented Jan 24, 2020

Can you include some tests?

@jridgewell
Copy link
Contributor Author

Sure. Is it ok to expose the decodeVLQ fn directly? Or do I have to test through SourceMap?

@mscdex
Copy link
Contributor

mscdex commented Jan 24, 2020

It's probably better to test via SourceMap as that's the public interface. Adding to the existing tests in test/parallel/test-source-map-api.js should be fine.

@jridgewell
Copy link
Contributor Author

Tests added.

@nodejs-github-bot

This comment has been minimized.

Copy link
Contributor

@bcoe bcoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the tests, It's great to have an extra set of eyes on the implementation.

@bcoe bcoe added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. source maps Issues and PRs related to source map support. labels Jan 25, 2020
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@Trott
Copy link
Member

Trott commented Jan 27, 2020

Landed in 0214b90

@Trott Trott closed this Jan 27, 2020
Trott pushed a commit that referenced this pull request Jan 27, 2020
These both have to do with extremely large numbers, so it's unlikely to
cause a problem in practice. Still, correctness.

First, encoding `-2147483648` in VLQ returns the value `"B"`. When
decoding, we get the value `1` after reading the base64. We then check
if the first bit is set (it is) to see if we should negate it, then we
shift all bits right once. Now, `value` will be `0` and `negate` will
be `true`. So, we'd return `-0`. Which is a bug! `-0` isn't
`-2147483648`, and we've broken a round trip.

Second, encoding any number with the 31st bit set, we'd return the
opposite sign. Let's use `1073741824`. Encoding, we get `"ggggggC"`.
When decoding, we get the value `-2147483648` after reading the base64.
Notice, it's already negative (the 32nd bit is set, because the 31st was
set and we shifted everything left once). We'd then check the first bit
(it's not) and shift right. But we used `>>`, which does not shift the
sign bit. We actually wanted `>>>`, which will. Because of that bug, we
get back `-1073741824` instead of the positive `1073741824`. It's even
worse if the 32nd and 31st bits are set, `-1610612736` becomes
`536870912` after a round trip.

I recently fixed the same two bugs in Closure Compiler:
google/closure-compiler@584418eb

PR-URL: #31490
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Coe <bencoe@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
@jridgewell jridgewell deleted the vlq branch January 27, 2020 01:37
codebytere pushed a commit that referenced this pull request Feb 17, 2020
These both have to do with extremely large numbers, so it's unlikely to
cause a problem in practice. Still, correctness.

First, encoding `-2147483648` in VLQ returns the value `"B"`. When
decoding, we get the value `1` after reading the base64. We then check
if the first bit is set (it is) to see if we should negate it, then we
shift all bits right once. Now, `value` will be `0` and `negate` will
be `true`. So, we'd return `-0`. Which is a bug! `-0` isn't
`-2147483648`, and we've broken a round trip.

Second, encoding any number with the 31st bit set, we'd return the
opposite sign. Let's use `1073741824`. Encoding, we get `"ggggggC"`.
When decoding, we get the value `-2147483648` after reading the base64.
Notice, it's already negative (the 32nd bit is set, because the 31st was
set and we shifted everything left once). We'd then check the first bit
(it's not) and shift right. But we used `>>`, which does not shift the
sign bit. We actually wanted `>>>`, which will. Because of that bug, we
get back `-1073741824` instead of the positive `1073741824`. It's even
worse if the 32nd and 31st bits are set, `-1610612736` becomes
`536870912` after a round trip.

I recently fixed the same two bugs in Closure Compiler:
google/closure-compiler@584418eb

PR-URL: #31490
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Coe <bencoe@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
@codebytere codebytere mentioned this pull request Feb 17, 2020
@targos
Copy link
Member

targos commented Apr 18, 2020

Depends on #31132 to land on v12.x

@targos targos removed author ready PRs that have at least one approval, no pending requests for changes, and a CI started. backport-blocked-v12.x labels Apr 25, 2020
targos pushed a commit to targos/node that referenced this pull request Apr 25, 2020
These both have to do with extremely large numbers, so it's unlikely to
cause a problem in practice. Still, correctness.

First, encoding `-2147483648` in VLQ returns the value `"B"`. When
decoding, we get the value `1` after reading the base64. We then check
if the first bit is set (it is) to see if we should negate it, then we
shift all bits right once. Now, `value` will be `0` and `negate` will
be `true`. So, we'd return `-0`. Which is a bug! `-0` isn't
`-2147483648`, and we've broken a round trip.

Second, encoding any number with the 31st bit set, we'd return the
opposite sign. Let's use `1073741824`. Encoding, we get `"ggggggC"`.
When decoding, we get the value `-2147483648` after reading the base64.
Notice, it's already negative (the 32nd bit is set, because the 31st was
set and we shifted everything left once). We'd then check the first bit
(it's not) and shift right. But we used `>>`, which does not shift the
sign bit. We actually wanted `>>>`, which will. Because of that bug, we
get back `-1073741824` instead of the positive `1073741824`. It's even
worse if the 32nd and 31st bits are set, `-1610612736` becomes
`536870912` after a round trip.

I recently fixed the same two bugs in Closure Compiler:
google/closure-compiler@584418eb

PR-URL: nodejs#31490
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Coe <bencoe@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
targos pushed a commit that referenced this pull request Apr 28, 2020
These both have to do with extremely large numbers, so it's unlikely to
cause a problem in practice. Still, correctness.

First, encoding `-2147483648` in VLQ returns the value `"B"`. When
decoding, we get the value `1` after reading the base64. We then check
if the first bit is set (it is) to see if we should negate it, then we
shift all bits right once. Now, `value` will be `0` and `negate` will
be `true`. So, we'd return `-0`. Which is a bug! `-0` isn't
`-2147483648`, and we've broken a round trip.

Second, encoding any number with the 31st bit set, we'd return the
opposite sign. Let's use `1073741824`. Encoding, we get `"ggggggC"`.
When decoding, we get the value `-2147483648` after reading the base64.
Notice, it's already negative (the 32nd bit is set, because the 31st was
set and we shifted everything left once). We'd then check the first bit
(it's not) and shift right. But we used `>>`, which does not shift the
sign bit. We actually wanted `>>>`, which will. Because of that bug, we
get back `-1073741824` instead of the positive `1073741824`. It's even
worse if the 32nd and 31st bits are set, `-1610612736` becomes
`536870912` after a round trip.

I recently fixed the same two bugs in Closure Compiler:
google/closure-compiler@584418eb

PR-URL: #31490
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Coe <bencoe@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
source maps Issues and PRs related to source map support.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants