Better error message when comparing images

[Yay295]

When comparing images with assert_image_equal(), assert their bytes instead of asserting False. This gives a better error message:

import pytest

def test_assert_false():
    assert False, "message"

def test_assert_compare():
    assert b"abc" == b"abb", "message"

================ FAILURES =================
____________ test_assert_false ____________

    def test_assert_false():
>       assert False, "message"
E       AssertionError: message
E       assert False

Tests/test_assert.py:4: AssertionError
___________ test_assert_compare ___________

    def test_assert_compare():
>       assert b"abc" == b"abb", "message"
E       AssertionError: message
E       assert b'abc' == b'abb'
E         At index 2 diff: b'c' != b'b'
E         Use -v to get more diff

Tests/test_assert.py:7: AssertionError

However, if the verbosity is too high, it will show a diff of the full image bytes. We can avoid this by temporarily changing the verbosity.

[radarhere]

This was previously discussed in #6653 (comment)

[radarhere]

Before

self = <Tests.test_image.TestImage object at 0x1081b8fa0>

    def test_p_from_rgb_rgba(self):
        for mode, color in [
            ("RGB", "#DDEEFF"),
            ("RGB", (221, 238, 255)),
            ("RGBA", (221, 238, 255, 255)),
        ]:
            im = Image.new("P", (100, 100), color)
            expected = Image.new(mode, (100, 100), color).rotate(45)
>           assert_image_equal(im.convert(mode), expected)

Tests/test_image.py:681: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

a = <PIL.Image.Image image mode=RGB size=100x100 at 0x10818C0D0>, b = <PIL.Image.Image image mode=RGB size=100x100 at 0x10818C430>, msg = None

    def assert_image_equal(a, b, msg=None):
        assert a.mode == b.mode, msg or f"got mode {repr(a.mode)}, expected {repr(b.mode)}"
        assert a.size == b.size, msg or f"got size {repr(a.size)}, expected {repr(b.size)}"
        if a.tobytes() != b.tobytes():
            if HAS_UPLOADER:
                try:
                    url = test_image_results.upload(a, b)
                    logger.error(f"Url for test images: {url}")
                except Exception:
                    pass
    
>           assert False, msg or "got different content"
E           AssertionError: got different content
E           assert False

Tests/helper.py:98: AssertionError
=============================================================================================================== short test summary info ================================================================================================================
FAILED Tests/test_image.py::TestImage::test_p_from_rgb_rgba - AssertionError: got different content

After

self = <Tests.test_image.TestImage object at 0x103793c70>

    def test_p_from_rgb_rgba(self):
        for mode, color in [
            ("RGB", "#DDEEFF"),
            ("RGB", (221, 238, 255)),
            ("RGBA", (221, 238, 255, 255)),
        ]:
            im = Image.new("P", (100, 100), color)
            expected = Image.new(mode, (100, 100), color).rotate(45)
>           assert_image_equal(im.convert(mode), expected)

Tests/test_image.py:681: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

a = <PIL.Image.Image image mode=RGB size=100x100 at 0x103997400>, b = <PIL.Image.Image image mode=RGB size=100x100 at 0x103997A00>, msg = None

    def assert_image_equal(a, b, msg=None):
        assert a.mode == b.mode, msg or f"got mode {repr(a.mode)}, expected {repr(b.mode)}"
        assert a.size == b.size, msg or f"got size {repr(a.size)}, expected {repr(b.size)}"
    
        original_verbosity = pytestconfig.option.verbose
        try:
            # set pytest's verbosity to 0 so that it doesn't show the full bytes diff
            pytestconfig.option.verbose = 0
>           assert a.tobytes() == b.tobytes(), msg or "got different content"
E           AssertionError: got different content
E           assert b'\xdd\xee\xf...f\xdd\xee\xff' == b'\x00\x00\x0...0\x00\x00\x00'
E             At index 0 diff: b'\xdd' != b'\x00'
E             Use -v to get more diff

Tests/helper.py:98: AssertionError
=============================================================================================================== short test summary info ================================================================================================================
FAILED Tests/test_image.py::TestImage::test_p_from_rgb_rgba - AssertionError: got different content

This might just be personal preference, but "got different content" grabs my attention more in the before version.

[Yay295]

It's still there. This change just also tells you what the different content is.

[radarhere]

Yes, but I think in 99% of cases, the difference in bytes is not going to be meaningful to me as a human - it is highly likely that I'd want to visually compare the images. I'm saying that I think placing 'got different content' more prominently is more helpful.

[hugovk]

Agreed, we don't need to print out bytes to the screen.

If someone does want to see them, they temporarily can add prints to the test.