-
Notifications
You must be signed in to change notification settings - Fork 121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add default Huffman table if huffman table uninitialized #493
Add default Huffman table if huffman table uninitialized #493
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice fix! can we get some tests in for this behavior. I mentioned one specifically in the comments to prevent a pathological case.
pkg/frame/compressed.go
Outdated
func decodeMJPEG(frame []byte, width, height int) (image.Image, func(), error) { | ||
img, err := jpeg.Decode(bytes.NewReader(frame)) | ||
|
||
if err != nil && err.Error() == "invalid JPEG format: uninitialized Huffman table" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you return early if there is no error?
also, it would be better to use errors.As with https://cs.opensource.google/go/go/+/refs/tags/go1.20.3:src/image/jpeg/reader.go;l=18 to then check just the "uninitialized Huffman table" error so we can be a bit more specific
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also, it would be better to use errors.As with https://cs.opensource.google/go/go/+/refs/tags/go1.20.3:src/image/jpeg/reader.go;l=18 to then check just the "uninitialized Huffman table" error so we can be a bit more specific
Could you explain this a little more? We can check that its a FormatError
but still need to check the contents of the message to make sure it isn't a different type of FormatError
. Also, how can I extract only "uninitialized Huffman table", without the "invalid JPEG format: " prefix added in the function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we do but we can eliminate the first part of that string to check
pkg/frame/compressed.go
Outdated
@@ -6,7 +6,25 @@ import ( | |||
"image/jpeg" | |||
) | |||
|
|||
// Thank you to https://github.com/filiptc/gorbit/blob/fa87ff39b68a6706306f34c318e0b9a5a3c97110/image/overlay.go#L37-L40 for addMotionDht, dhtMarker, dht, and sosMarker |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
licensing checks out 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Strictly speaking, when OSS licensed code is copied, we should include the full original license statements according to
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
in case of MIT.
https://github.com/filiptc/gorbit/blob/fa87ff39b68a6706306f34c318e0b9a5a3c97110/LICENSE
pkg/frame/compressed.go
Outdated
|
||
func addMotionDht(frame []byte) []byte { | ||
jpegParts := bytes.Split(frame, sosMarker) | ||
return append(jpegParts[0], append(dhtMarker, append(dht, append(sosMarker, jpegParts[1]...)...)...)...) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should check that the split worked. otherwise a malicious input could crash the caller. this would be a good test
pkg/frame/compressed.go
Outdated
|
||
func addMotionDht(frame []byte) []byte { | ||
jpegParts := bytes.Split(frame, sosMarker) | ||
return append(jpegParts[0], append(dhtMarker, append(dht, append(sosMarker, jpegParts[1]...)...)...)...) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is copying bytes in a potentially expensive way. can you make the slice in advance with the new size and then copy in order. it will also make these appends a tad easier to read!
pkg/frame/compressed.go
Outdated
|
||
var ( | ||
dhtMarker = []byte{255, 196} | ||
dht = []byte{1, 162, 0, 0, 1, 5, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1, 0, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 16, 0, 2, 1, 3, 3, 2, 4, 3, 5, 5, 4, 4, 0, 0, 1, 125, 1, 2, 3, 0, 4, 17, 5, 18, 33, 49, 65, 6, 19, 81, 97, 7, 34, 113, 20, 50, 129, 145, 161, 8, 35, 66, 177, 193, 21, 82, 209, 240, 36, 51, 98, 114, 130, 9, 10, 22, 23, 24, 25, 26, 37, 38, 39, 40, 41, 42, 52, 53, 54, 55, 56, 57, 58, 67, 68, 69, 70, 71, 72, 73, 74, 83, 84, 85, 86, 87, 88, 89, 90, 99, 100, 101, 102, 103, 104, 105, 106, 115, 116, 117, 118, 119, 120, 121, 122, 131, 132, 133, 134, 135, 136, 137, 138, 146, 147, 148, 149, 150, 151, 152, 153, 154, 162, 163, 164, 165, 166, 167, 168, 169, 170, 178, 179, 180, 181, 182, 183, 184, 185, 186, 194, 195, 196, 197, 198, 199, 200, 201, 202, 210, 211, 212, 213, 214, 215, 216, 217, 218, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 17, 0, 2, 1, 2, 4, 4, 3, 4, 7, 5, 4, 4, 0, 1, 2, 119, 0, 1, 2, 3, 17, 4, 5, 33, 49, 6, 18, 65, 81, 7, 97, 113, 19, 34, 50, 129, 8, 20, 66, 145, 161, 177, 193, 9, 35, 51, 82, 240, 21, 98, 114, 209, 10, 22, 36, 52, 225, 37, 241, 23, 24, 25, 26, 38, 39, 40, 41, 42, 53, 54, 55, 56, 57, 58, 67, 68, 69, 70, 71, 72, 73, 74, 83, 84, 85, 86, 87, 88, 89, 90, 99, 100, 101, 102, 103, 104, 105, 106, 115, 116, 117, 118, 119, 120, 121, 122, 130, 131, 132, 133, 134, 135, 136, 137, 138, 146, 147, 148, 149, 150, 151, 152, 153, 154, 162, 163, 164, 165, 166, 167, 168, 169, 170, 178, 179, 180, 181, 182, 183, 184, 185, 186, 194, 195, 196, 197, 198, 199, 200, 201, 202, 210, 211, 212, 213, 214, 215, 216, 217, 218, 226, 227, 228, 229, 230, 231, 232, 233, 234, 242, 243, 244, 245, 246, 247, 248, 249, 250} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this data standardized or specific to some device of some vendor?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should work across devices in the case that the Huffman encoding info has been omitted for bandwidth optimization
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you point where this comes from though?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The JPEG wiki lists dhtMarker
& sosMarker
as dht & sos respectively.
This RFC discusses a "Standard Huffman Table" and location of the citation is behind a paywall. This thread is where I got the table from. It works on cameras I have tested with and has been reported to work on other cameras as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@at-wat is that cool with you?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This RFC discusses a "Standard Huffman Table" and location of the citation is behind a paywall.
https://www.w3.org/Graphics/JPEG/itu-t81.pdf
W3C seems putting it in public.
pkg/frame/compressed.go
Outdated
/* addMotionDht, dhtMarker, dht, and sosMarker is protected under the following license: | ||
|
||
The MIT License (MIT) | ||
|
||
Copyright (c) 2016 Philip Thomas Casado | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. | ||
|
||
*/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@at-wat like this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe better to move the copied part to separated file (if code inside decodeMJPEG
and addMotionDht
is also copied, current structure is fine)
Could you add a URL of the original repo as well?
Other things looks good to me!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Link is added! parts of the functions you mentioned are taken from the repo
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done with another pass. Thanks for adding tests!
pkg/frame/compressed.go
Outdated
return img, func() {}, err | ||
} | ||
|
||
var target jpeg.FormatError = jpeg.FormatError("uninitialized Huffman table") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this can be pulled outside of this function so it's not recomptued
var target jpeg.FormatError = jpeg.FormatError("uninitialized Huffman table") | ||
if errors.As(err, &target) { | ||
if err.Error() == target.Error() { | ||
img, err = jpeg.Decode(bytes.NewReader(addMotionDht(frame))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should check if the error is not nil and return the error. Otherwise it can return the image. It's a good practice to not return any potentially initialized values (img) in the case of failure cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could that break other things? The function did not originally ever return nil
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it would not
pkg/frame/compressed.go
Outdated
correctedFrame := make([]byte, len(jpegParts[0])+len(dhtMarker)+len(dht)+len(sosMarker)+len(jpegParts[1])) | ||
correctedFrameOffset := 0 | ||
|
||
for indx, item := range jpegParts[0] { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
indx -> idx for each of these
pkg/frame/compressed.go
Outdated
if len(jpegParts) != 2 { | ||
return frame | ||
} | ||
correctedFrame := make([]byte, len(jpegParts[0])+len(dhtMarker)+len(dht)+len(sosMarker)+len(jpegParts[1])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
len(dhtMarker)+len(dht)+len(sosMarker)
can be calculated once outside of this function which will simplify reading this
pkg/frame/compressed.go
Outdated
correctedFrameOffset += len(jpegParts[0]) | ||
|
||
for indx, item := range dhtMarker { | ||
correctedFrame[indx+correctedFrameOffset] = item |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
slightly more readable to have it be the offset + the idx for each of these
pkg/frame/compressed.go
Outdated
correctedFrameOffset := 0 | ||
|
||
for indx, item := range jpegParts[0] { | ||
correctedFrame[indx] = item |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
instead of these loops I think you can do copy(correctedFrame[offset:], jpegParts[0])
for each of these. that should make things smaller and simple to read!
pkg/frame/compressed_test.go
Outdated
// Decode fails with an uninitialized Huffman table error for sample input | ||
expectedErrorMessage := "invalid JPEG format: uninitialized Huffman table" | ||
if err.Error() != expectedErrorMessage { | ||
t.Errorf("Wrong decode error result,\nexpected:\n%+v\ngot:\n%+v", expectedErrorMessage, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe you need to use FailNow as well
correctedFrame := make([]byte, len(jpegParts[0])+huffmanTableInfoLength+len(jpegParts[1])) | ||
correctedFrameOffset := 0 | ||
|
||
copy(correctedFrame[correctedFrameOffset:], jpegParts[0]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so much better!
pkg/frame/compressed_test.go
Outdated
// Decode passes after adding default Huffman table to | ||
defaultHuffmanTableFrame, err := jpeg.Decode(bytes.NewReader(addMotionDht(UninitializedHuffmanTable))) | ||
if err != nil { | ||
t.Errorf("Expected decode function to pass after adding default Huffman table. Failed with %v\n", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all of these need the failnow too but you can just do Fatalf instead
func TestAddMotionDht(t *testing.T) { | ||
uninitializedHuffmanTableFrame, err := jpeg.Decode(bytes.NewReader(UninitializedHuffmanTable)) | ||
|
||
// Decode fails with an uninitialized Huffman table error for sample input | ||
expectedErrorMessage := "invalid JPEG format: uninitialized Huffman table" | ||
if err.Error() != expectedErrorMessage { | ||
t.Fatalf("Wrong decode error result,\nexpected:\n%+v\ngot:\n%+v", expectedErrorMessage, err) | ||
} | ||
|
||
// Decode passes after adding default Huffman table to | ||
defaultHuffmanTableFrame, err := jpeg.Decode(bytes.NewReader(addMotionDht(UninitializedHuffmanTable))) | ||
if err != nil { | ||
t.Fatalf("Expected decode function to pass after adding default Huffman table. Failed with %v\n", err) | ||
} | ||
|
||
// Adding default Huffman table to a valid frame without a Huffman table changes the table | ||
if uninitializedHuffmanTableFrame == defaultHuffmanTableFrame { | ||
t.Fatalf("Expected addMotionDht to update frame. Instead returned original frame") | ||
} | ||
|
||
// Check that an improperly constructed frame does not get updated by addMotionDht | ||
randomBytes := []byte{1, 2, 3, 4} | ||
frame1, err := jpeg.Decode(bytes.NewReader(randomBytes)) | ||
if err == nil { | ||
t.Fatalf("Expected decode function to fail with random bytes but passed.") | ||
} | ||
|
||
frame2, err := jpeg.Decode(bytes.NewReader(addMotionDht(randomBytes))) | ||
if err == nil { | ||
t.Fatalf("Expected decode function to fail with random bytes but passed.") | ||
} | ||
|
||
if frame1 != frame2 { | ||
t.Fatalf("addMotionDht updated the frame despite being improperly constructed") | ||
} | ||
} | ||
|
||
func TestDecodeMJPEG(t *testing.T) { | ||
_, _, err := decodeMJPEG(UninitializedHuffmanTable, 640, 480) | ||
if err != nil { | ||
t.Fatalf("Expected decode function to pass. Failed with %v\n", err) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@edaniels like this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Will wait on @at-wat for other approval
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #493 +/- ##
==========================================
+ Coverage 58.85% 59.11% +0.25%
==========================================
Files 62 62
Lines 3753 3784 +31
==========================================
+ Hits 2209 2237 +28
Misses 1416 1416
- Partials 128 131 +3
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
Description
Certain MJPEG frames do not include Huffman tables, this causes an
uninitialized Huffman table
error duringjpeg.Decode()
. These changes add a default Huffman Table in the case that one was not provide, so that the jpeg can be processed and transformed into an image. I tested these changes locally with cameras that provide and do not provide Huffman tables to ensure that the error is avoided, and for frames with initialized Huffman tables there is no change in behavior.Reference issue
Fixes an uninitialized Huffman error that is returned from
jpeg.Decode()
when a jpeg Frame omits the Huffman Table