Unused snapshots when running on filename (test_a.py) that's the prefix of another (test_abc.py)

[huonw]

Describe the bug

I have a project that has filenames where the non-.py part happens to be a prefix of another (e.g. test_a.py vs. test_abc.py). When running test_a.py in isolation, syrupy picks up the snapshots from test_abc.py and either warns/errors about them or deletes them (depending on if --snapshot-update) is specified.

(Thanks for syrupy!)

To reproduce

# test_a.py
def test_foo(snapshot): assert snapshot == "a"

# test_abc.py
def test_bar(snapshot): assert snapshot == "abc"

# requirements.txt
attrs==21.4.0
colored==1.4.3
iniconfig==1.1.1
packaging==21.3
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.8
pytest==7.1.1
syrupy==1.7.4
tomli==2.0.1

With the above code, if I run pytest test_a.py, the snapshot from test_abc.py is flagged as unused:

================================= test session starts =================================
platform darwin -- Python 3.9.10, pytest-7.1.1, pluggy-1.0.0
rootdir: .../syrupy-file-prefix
plugins: syrupy-1.7.4
collected 1 item                                                                      

test_a.py .                                                                     [100%]

------------------------------- snapshot report summary -------------------------------
1 snapshot passed. 1 snapshot unused.

Unused test_bar (__snapshots__/test_abc.ambr)
Re-run pytest with --snapshot-update to delete unused snapshots.
================================== 1 passed in 0.01s ==================================

Fully packaged:

# set up virtualenv/deps/files
python --version  # Python 3.9.10
python -m venv venv
. venv/bin/activate
pip install attrs==21.4.0 colored==1.4.3 iniconfig==1.1.1 packaging==21.3 pluggy==1.0.0 py==1.11.0 pyparsing==3.0.8 pytest==7.1.1 syrupy==1.7.4 tomli==2.0.1

echo 'def test_foo(snapshot): assert snapshot == "a"' > test_a.py
echo 'def test_bar(snapshot): assert snapshot == "abc"' > test_abc.py

# create snapshots
pytest --snapshot-update  # 2 snapshots generated.
pytest  # 2 snapshots passed.

# running test_abc.py: all okay
pytest test_abc.py --snapshot-details  # 1 snapshot passed.

# running test_a.py: unused snapshots (BUG)
pytest test_a.py --snapshot-details  # Unused test_bar (__snapshots__/test_abc.ambr)
pytest test_a.py --snapshot-details --snapshot-update  # Deleted test_bar (__snapshots__/test_abc.ambr)

Expected behavior

When running pytest against a single/subset of files, snapshots from other files shouldn't be considered unused.

Screenshots

Environment (please complete the following information):

• OS: macOS 12.3
• Syrupy Version: 1.7.4
• Python Version: 3.9.10

Additional context

N/A

[noahnu]

Odd. This sounds like a regression of an old issue. We even have explicit tests around the prefix case. I'll have to dig into this.

[noahnu]

You can see the existing coverage we have for this here: https://github.com/tophat/syrupy/blob/master/tests/integration/test_snapshot_similar_names_default.py and related issue: #529

[huonw]

Ah yeah, I can see the similarities 👍 It looks like that one is more about the names of the test functions themselves whereas this one is about the file names?

[noahnu]

The bit of logic responsible for all this is here: https://github.com/tophat/syrupy/blob/f189e65a77923090154c93bfe9883e5854dd84b1/src/syrupy/report.py#L183.

I do have a feeling this was a relatively recent regression but will have to dig into it some more when I have some time later this week. If you want to try take a stab at it yourself, feel free to do so. I'm available to answer questions.

[huonw]

Thanks for the tip, that was helpful. I've made an attempt in #607.

[tophat-opensource-bot]

🎉 This issue has been resolved in version 2.3.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀