Signed Integer Overflow in calculateNumTiles (73181093)

[Google-Autofuzz]

Hello OpenEXR team,

As part of our fuzzing efforts at Google, we have identified an issue affecting
OpenEXR (tested with revision * develop 165dcea).

To reproduce, we are attaching a Dockerfile which compiles the project with
LLVM, taking advantage of the sanitizers that it offers. More information about
how to use the attached Dockerfile can be found here:
https://docs.docker.com/engine/reference/builder/

TL;DR instructions:

• mkdir project
• cp Dockerfile.OpenEXR /path/to/project/Dockerfile
• docker build --no-cache /path/to/project
• docker run -it image_id_from_docker_build

From another terminal, outside the container:
docker cp /path/to/attached/reproducer running_container_hostname:/fuzzing/reproducer
(reference: https://docs.docker.com/engine/reference/commandline/cp/)

And, back inside the container:
/fuzzing/repro.sh /fuzzing/reproducer

Alternatively, and depending on the bug, you could use gcc, valgrind or other
instrumentation tools to aid in the investigation. The sanitizer errors that we
encountered were:

ImfTiledMisc.cpp:304:48: runtime error: signed integer overflow: 64 + 2147483628 cannot be represented in type 'int'
    #0 0x5cb7c1 in Imf_2_2::(anonymous namespace)::calculateNumTiles(int*, int, int, int, int, Imf_2_2::LevelRoundingMode) /fuzzing/openexr/OpenEXR/IlmImf/ImfTiledMisc.cpp:304:48
    #1 0x5caebb in Imf_2_2::precalculateTileInfo(Imf_2_2::TileDescription const&, int, int, int, int, int*&, int*&, int&, int&) /fuzzing/openexr/OpenEXR/IlmImf/ImfTiledMisc.cpp:324:5
    #2 0x66b134 in Imf_2_2::TiledInputFile::initialize() /fuzzing/openexr/OpenEXR/IlmImf/ImfTiledInputFile.cpp:934:5
    #3 0x66c387 in Imf_2_2::TiledInputFile::TiledInputFile(Imf_2_2::Header const&, Imf_2_2::IStream*, int, int) /fuzzing/openexr/OpenEXR/IlmImf/ImfTiledInputFile.cpp:837:5
    #4 0x630f64 in Imf_2_2::InputFile::initialize() /fuzzing/openexr/OpenEXR/IlmImf/ImfInputFile.cpp:545:32
    #5 0x62feb4 in Imf_2_2::InputFile::InputFile(char const*, int) /fuzzing/openexr/OpenEXR/IlmImf/ImfInputFile.cpp:382:13
    #6 0x55cb1a in Imf_2_2::RgbaInputFile::RgbaInputFile(char const*, int) /fuzzing/openexr/OpenEXR/IlmImf/ImfRgbaFile.cpp:1166:21
    #7 0x524984 in (anonymous namespace)::readSingle(char const*) /fuzzing/security-research-pocs/autofuzz/openexr_fuzz_scanlines.cc:20:14
    #8 0x524805 in LLVMFuzzerTestOneInput /fuzzing/security-research-pocs/autofuzz/openexr_fuzz_scanlines.cc:113:3
    #9 0x516d9c in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/fuzzing/openexr_fuzz_scanlines+0x516d9c)
    #10 0x51655e in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long) (/fuzzing/openexr_fuzz_scanlines+0x51655e)
    #11 0x5103bd in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*) (/fuzzing/openexr_fuzz_scanlines+0x5103bd)
    #12 0x51188f in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/fuzzing/openexr_fuzz_scanlines+0x51188f)
    #13 0x51026c in main (/fuzzing/openexr_fuzz_scanlines+0x51026c)
    #14 0x7f5f59cd62b0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202b0)
    #15 0x425979 in _start (/fuzzing/openexr_fuzz_scanlines+0x425979)

SUMMARY: AddressSanitizer: undefined-behavior ImfTiledMisc.cpp:304:48 

We will gladly work with you so you can successfully confirm and reproduce this
issue. Do let us know if you have any feedback surrounding the documentation.

Once you have reproduced the issue, we'd appreciate to learn your expected
timeline for an update to be released. With any fix, please attribute the report
to "Google Autofuzz project".

We are also pleased to inform you that your project is eligible for inclusion to
the OSS-Fuzz project, which can provide additional continuous fuzzing, and
encourage you to investigate integration options.

Don't hesitate to let us know if you have any questions!

Google AutoFuzz Team
artifacts_73181093.zip

[Google-Autofuzz]

Additional stack trace that triggers the bug from a different code path:

ImfTiledMisc.cpp:304:48: runtime error: signed integer overflow: 1626642713 + 1073741824 cannot be represented in type 'int'
    #0 0x7f91d8ffa497 in Imf_2_2::(anonymous namespace)::calculateNumTiles(int*, int, int, int, int, Imf_2_2::LevelRoundingMode) /fuzzing/openexr/OpenEXR/IlmImf/ImfTiledMisc.cpp:304:48
    #1 0x7f91d8ffa61d in Imf_2_2::getTiledChunkOffsetTableSize(Imf_2_2::Header const&) /fuzzing/openexr/OpenEXR/IlmImf/ImfTiledMisc.cpp:355:5
    #2 0x7f91d9043c28 in Imf_2_2::MultiPartInputFile::Data::readChunkOffsetTables(bool) /fuzzing/openexr/OpenEXR/IlmImf/ImfMultiPartInputFile.cpp:738:36
    #3 0x7f91d9042a0f in Imf_2_2::MultiPartInputFile::initialize() /fuzzing/openexr/OpenEXR/IlmImf/ImfMultiPartInputFile.cpp:429:12
    #4 0x7f91d9041e0b in Imf_2_2::MultiPartInputFile::MultiPartInputFile(char const*, int, bool) /fuzzing/openexr/OpenEXR/IlmImf/ImfMultiPartInputFile.cpp:136:9
    #5 0x50f897 in printInfo(char const*) /fuzzing/openexr/OpenEXR/exrheader/main.cpp:290:24
    #6 0x5113f8 in main /fuzzing/openexr/OpenEXR/exrheader/main.cpp:562:13
    #7 0x7f91d79092b0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202b0)
    #8 0x424719 in _start (/fuzzing/openexr/OpenEXR/exrheader/.libs/exrheader+0x424719)

SUMMARY: AddressSanitizer: undefined-behavior ImfTiledMisc.cpp:304:48 in 

Attached you will find another an additional sample as well as a patch that fixes the issue.
artifacts_73583817.zip

[Google-Autofuzz]

Attached the patch.

73583817.txt

[peterhillman]

Fixed by enforcing a limit to maximum number of tiles