-
Notifications
You must be signed in to change notification settings - Fork 198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
An example of XMP metadata that triggers the error "Error writing PNG: XMP metadata contains an invalid null character" in avifpng.c #1333
Comments
Running avifdec in the debugger, I found that the null character is at the end of the |
The XMP specification says "When XMP is encoded as UTF-8, there are no zero bytes in the XMP Packet." It seems that the XMP metadata in moon.jpg has an XMP packet that is incorrectly null-terminated. |
comes from Lines 481 to 485 in d48836b
I assume the issue is that
Opinions? |
IMHO, it makes sense to try and sanitize the XMP chunk at the point of import, from any format (this is not tied to PNG spec only, this is in the XMP spec), so you don't even produce an "invalid" AVIF in the first place. I think this is the approach we took in exiv2 as well as libexpat was throwing a similar error and refused to parse it from a sample JPEG... |
@kmilos Thanks for the info. Do you have any pointer to exiv2 or libexpat code implementing what you described? |
Fond it - take a look at Exiv2/exiv2#2133 and the related issue report. libexpat just assumes its input is not terminated and simply barfs if it is, it has no workarounds for it. I remember even looking into to the libexpat code to convince myself that the nul character was explicitly in the list of invalid XML characters they check for... |
We need to first make sure the terminating null byte comes from moon.jpg rather than our code in Once we know We can leave the check during PNG export unchanged. |
This has nothing to do w/ libjpeg per se, who knows how that original file was created.... |
As noted,
|
Yep, the fix is in exiv2 0.27.6 Btw,
|
I confirm that at the following point: libavif/apps/shared/avifjpeg.c Line 532 in d48836b
|
Miloš, Yannis: Thank you very much for confirming our code is correct and the error is in the moon.jpg file. Should we report an error if an XMP packet is not valid UTF-8? Or just check if the XMP packet contains a zero byte (similar to the check in avifpng.c)? Our error message can point to tools that can be used to inspect and correct the errors in the XMP packet. |
I found that sentence only in XMP spec Part 3 section
I started writing a change involving a new |
I feel you should do it at a single/common entry point (e.g.
It is forbidden in UTF-8 XML period, nothing to do w/ images. As I said, libexpat just flatly refuses to parse null-terminated XML, no matter where it comes from. From https://en.wikipedia.org/wiki/Valid_characters_in_XML:
|
I considered that, but wondered whether XMP chunks provided by users through the API should be altered. For example, someone setting 100 bytes of XMP, then accessing
I agree. Although it is just a few formats in libavif (for now...).
Great, this simplifies the fix. Thanks for the reference. |
Yannis: Since I filed this issue, I wanted to clarify that after confirming our code is correct and the error is in the moon.jpg file, I think our current policy is fine (importing metadata faithfully, and only checking for errors that break the output format, such as zero bytes for a PNG iTXt chunk). As the bug reporter, I think it is okay to close this issue. If you'd like to remove trailing null characters on import, I suggest we remove only one (to handle the off-by-one bug of including the terminating null character of a null-terminated string). Fixing up errors will hide them and make it harder to track down the source of the errors, so if an extra terminating null character is not a common error, we don't really need to fix it up. |
Well, then the bug is that you only check/warn for PNG, while it should be for all output formats. |
I think an In parallel, removing a single trailing null character on any import seems acceptable. |
As an alternative, you could just not fail at all on either Or add an |
I found this topic very interesting! I encountered the same issue when I discovered that the pre-installed files IMG_0001.jpg and IMG_0002.jpg in the iOS simulator also have the illegal NUL terminator. Initially, I considered whether the issue should be addressed in https://github.com/Ashampoo/kim or not. However, I realized that even the command "exiftool -xmp -b IMG_0001.JPG > out.xmp" exports the NUL character at the end. Therefore, it seems correct that the XMP string is exported as is - with all errors. In the end, I made the decision that the proper approach would be to handle this issue within the XMP parser. I fixed it in my case in https://github.com/Ashampoo/xmpcore/pull/4/files. I want to express my gratitude for all the insights shared during this discussion. Thank you! |
Not quite. Note that the |
Yes, that's correct. If ExifTool parses the actual XMP and writes new it looks very different. In my case I have two libraries: At first I was unsure where I should fix the problem and wanted Kim to output a trimmed XMP string. But after seeing that ExifTool extracts the untouched original I decided to do the same. The XMP parser must be the one resilient to this problems. ExifTool ignores the invalid char if parsing the data, but it exports the RAW data exactly as it is. I think this is the correct behavior. |
I just noticed that also the XMP of the embedded preview JPG of my RAF files produced by my X-T4 have NUL characters at the end. In that case two. |
Yannis: Please take a look at the XMP metadata in this image and see if avifpng.c is rejecting it correctly.
Steps to reproduce:
The avifdec command fails with the error message:
The text was updated successfully, but these errors were encountered: