PyCharm diff comparison failure doesn't work with custom classes

[lindenwells]

Describe the bug

When comparing a failed snapshot test on an object without a custom __repr__ (i.e. relies on syrupy's serializer), PyCharm incorrectly displays the actual output as the default __repr__ (e.g. ...Mansion object at 0xBEEF2020) instead of what syrupy serialises it to (e.g. Mansion(rooms=[Room(...)])).

This might be a PyCharm bug, or Syrupy or somehow both. Just thought I'd make this issue as it tripped me up at work for over an hour and I imagine a fair few people use PyCharm and syrupy together.

To reproduce

1. checkout this repo I made
2. create a virtualenv within and pip install syrupy==3.0.2
3. find the singular test within the run configurations and notice that it fails.
4. in the failed test output, scroll to the clickable '<Click to see difference>' text:
5. click that text and you should get a Comparison Failure interface like this:

Expected behavior

This is roughly what I should see in the Comparison Failure window.

I know this is wrong because when I run pytest --snapshot-update, it updates the snapshot like this.

Screenshots
added throughout where appropriate

Environment:

• OS: Windows 11 Pro 21H2
• Syrupy Version: 3.0.2
• Python Version: 3.9.13
• PyCharm Version: 2022.3 (Community Edition), Build #PC-223.7571.203, built on December 1, 2022, Runtime version: 17.0.5+1-b653.14 amd64, VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.

Additional context

This is similar to #572 but not the same.

[Crown0815]

Similar issue here with basic strings:

Create snapshot from:

def test_test(snapshot):
    test = """Hello to
the world"""
    assert test == snapshot

pytest --snapshot-update

Modify the string (e.g., add ! to world)

def test_test(snapshot):
    test = """Hello to
the world!"""                   # Added the `!`
    assert test == snapshot

Yields non-helpful diff view:

[UltimateLobster]

I was getting the same issue and I was looking into this.
I found the built-in script that pycharm uses in order to compute the differences and managed to "patch" it from my own code in order to make this work:

@pytest.fixture(scope='session', autouse=True)
def _patch_pycharm_diff_viewer_for_snapshots():
    try:
        from teamcity.diff_tools import EqualsAssertionError
    except ImportError:
        yield
        return

    old_init = EqualsAssertionError.__init__
    old_init_signature = signature(old_init)

    @wraps(old_init)
    def new_init(self, *args, **kwargs):

        # Extract the __init__ arguments as originally passed in order to
        # process them later
        parameters = old_init_signature.bind(self, *args, **kwargs)
        parameters.apply_defaults()
        expected = parameters.arguments["expected"]
        actual = parameters.arguments["actual"]
        real_exception = parameters.arguments["real_exception"]

        if isinstance(expected, SnapshotAssertion):
            snapshot = expected
        elif isinstance(actual, SnapshotAssertion):
            snapshot = actual
        else:
            snapshot = None

        old_init(self, *args, **kwargs)

        # No snapshot was involved in the assertion. Let the old logic do its
        # thing.
        if snapshot is None:
            return

        # Although a snapshot was involved in the assertion, it seems the error
        # was a result of a non-assertion exception (Ex. `assert 1/0`).
        # Therefore, We will not do anything here either.
        if real_exception is not None:
            return

        assertion_result = snapshot.executions[snapshot.num_executions - 1]
        if assertion_result.exception is not None:
            return

        self.expected = str(assertion_result.recalled_data)
        self.actual = str(assertion_result.asserted_data)

    EqualsAssertionError.__init__ = new_init
    yield
    EqualsAssertionError.__init__ = old_init

This fixture makes it so we patch what pycharm will believe the "expected" and "actual" arguments are when a test fails. It only does it when the either the "expected" or the "actual" are snapshots, and only if the exception was an assertion error (and not an unrelated exception that may have been raised during the evaluation time).

I'll open a PR to add this (behind an opt-in flag?), but I have no idea whether one of the maintainers would actually accept such an ugly patch.

[noahnu]

🎉 This issue has been resolved in version 4.7.0 🎉

The release is available on:

• v4.7.0
• GitHub release

Your semantic-release bot 📦🚀