Using a different cwd and test path argument to pytest breaks test annotations

[Flamefire]

Relevant settings:

    "python.unitTest.cwd": "${workspaceFolder}/src/site-packages",
    "python.unitTest.pyTestArgs": [
        "${workspaceFolder}/test"
    ],

Debugging the extension I found that the tests path is converted relative to cwd but the test is relative to "${workspaceFolder}/test". This breaks the annotations.

[d3r3kk]

Hi @Flamefire, thanks for the report! However, there are some details I'll need before I can track this down properly.

• Please do fill out the default template for our issues with the information about your setup/reproduction steps.
• A sample for us to work with goes very far in allowing us to pinpoint the issue quickly, if you can, please do supply some sample code to reproduce the issue.

Thanks!

[DonJayamanne]

What's the path to the python interpreter?
Is it src/bin/python or similar?

I ask because of site_packages in your src folder.

[Flamefire]

@d3r3kk The "bug" is here: https://github.com/Microsoft/vscode-python/blob/fbd9c444b83216233f935687262b910a30210be2/src/client/unittests/pytest/services/parserService.ts#L104

To reproduce:

• Create a module in e.g. src/site-packages (or just src), e.g. fooPkg/__init__.py
• Create a pyTest test using that module in test, e.g. test_foo.py
• Use the module base path as cwd and the test path as first argument to pyTest (see config above)
• Observe that on discovering the tests on the above line the absolute path is resolved as src/site-packages/test_foo.py which is obviously wrong

I think the rest of the template is not relevant but I'll fill it out if you think otherwise

@DonJayamanne No. This was just something we tried: In a mixed C++/Python source base put python packages in that folder which could then be installed directly. This was changed since though. Interpreter is default python which is Python3 in this case but version does not matter for this bug.

[d3r3kk]

I see the issue, thanks for the extra info.

I believe what would be needed here is to add more robust test discovery to the extension to allow this dissociation between tests and the modules they perform tests on.

We are planning to add test-discovery functionality during the upcoming quarter, and we will cover scenarios such as this as we progress.

Until then, the workaround would be to remove the use of CWD as you have it now.

[Flamefire]

Note that a fix would be pretty easy. All that needs to be done is parse the line starting with "rootdir:" from pytest. All reported modules are relative to that. I'd probably be able to implement this but am unsure about the requirements for that (I guess at least appropriate test cases are required?)

[d3r3kk]

Note that a fix would be pretty easy.

Yes, that may be the case, and do feel free to contribute of course! However, as I mentioned we are actually refactoring test discovery in the upcoming releases, so a fix would be temporary, and I'd hate for you to spend a lot of time on anything that wouldn't be around for very long.

...at least appropriate test cases are required...

Yep! Unit tests and all that great stuff.

edit: test case comment added

[karthiknadig]

Should be addressed via #16769