Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling of extraneous data in Unmarshal() & Valid() #380

Merged
merged 1 commit into from
Jan 1, 2023

Conversation

fxamacker
Copy link
Owner

@fxamacker fxamacker commented Jan 1, 2023

Previously, Unmarshal() and Valid() ignored trailing bytes (if any) after the current CBOR data item. This matched the behavior of Decoder.Decode().

However, Unmarshal() and Valid() are typically used for single CBOR data item, so trailing bytes (if present) should return ExtraneousDataError given what RFC 8949 says:

A well-formed data item uses the initial bytes and the byte strings and/or data items that are implied by their values as defined in CBOR and does not include following extraneous data. CBOR decoders by definition only return contents from well-formed data items.

This is a bug fix rather than a new non-default option because the previous behavior of ignoring extra data didn't exactly match RFC 8949 and was also different from how Marshal() in encoding/json handles extraneous data.

Special thanks to @zensh for reporting this issue and proposing a solution.

Closes #359

How to handle extraneous data as non-errors

Decoder.Decode() and UnmarshalFirst() can handle extraneous data as non-errors.

  • Decoder.Decode() uses io.Reader and ignores remaining bytes after reading a single CBOR data item. By design, it does not treat extraneous data as an error.

  • 🆕 UnmarshalFirst() will decode first CBOR data item and return trailing bytes (if any). By design, it will not treat extraneous data as an error.

UnmarshalFirst(data []byte, v interface{}) (rest []byte, err error)

Decoder.Decode() and UnmarshalFirst() are also useful for decoding CBOR Sequences (RFC 8742).

Previously, Unmarshal() and Valid() ignored extra
data (if any) after the current CBOR data item. This
matched the behavior of Decoder.Decode().

However, Unmarshal() and Valid() are typically used for
single CBOR data item, so extra data (if present)
should return ExtraneousDataError.

This is a bug fix because the previous behavior
of ignoring extra data didn't exactly match RFC 8949
regarding this and was also different from how
encoding/json handles this.

The behavior for Decoder.Decode() is unchanged because
extra data after reading a single CBOR data item
is expected for Decoder with io.Reader.

Special thanks to zensh for reporting this issue
and proposing a solution.
@fxamacker fxamacker added the bug Something isn't working label Jan 1, 2023
@fxamacker fxamacker added this to the v2.5.0 milestone Jan 1, 2023
@fxamacker fxamacker merged commit 25ddb46 into master Jan 1, 2023
@fxamacker fxamacker deleted the fxamacker/handle-extranenous-data branch January 2, 2023 00:06
@fxamacker fxamacker changed the title Fix handling of extra data in Unmarshal() & Valid() Fix handling of extraneous data in Unmarshal() & Valid() Aug 31, 2023
leif added a commit to katzenpost/katzenpost that referenced this pull request Feb 7, 2024
this requires that ping use this:
https://pkg.go.dev/github.com/fxamacker/cbor/v2#UnmarshalFirst
to unmarshal the padded payload and ignore the padding, due to this change in
the cbor library:
fxamacker/cbor#380
mixmasala added a commit to katzenpost/katzenpost that referenced this pull request Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: Unmarshal and Valid should check following extraneous data
2 participants