Virtualenv not always found in case insensitive filesystem

[absassi]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10

• Poetry version: 1.0.5

• Link of a Gist with the contents of your pyproject.toml file: irrelevant

Issue

Poetry does not consider case sensitivity of the filesystem when computing the hash of the virtual environment name. This makes it to fail to find the environment in case insensitive filesystems depending on the case of letters in the path.

For example, in Windows, having a project named "test" in C:\test and accessing it from c:\test or C:\Test or any other case variation. This cannot be done from command prompt, as it always canonicalize the path, but it can be easily done from scripts (e.g. Jenkinsfile).

The hash is computed in EnvManager.generate_env_name in poetry.utils.env:

>>> from poetry.utils.env import EnvManager
>>> EnvManager.generate_env_name('test', 'C:\\test')
'test-bFaF-DbV'
>>> EnvManager.generate_env_name('test', 'c:\\test')
'test-7_Oi5G5Y'

I expected the hash to be computed always the same, when the filesystem is case-insensitive (or at least use os.path.normcase or similar, which is does the job in Windows).

Since I cannot control how the path case is in my build environment, the workaround I'm currently using is to set virtualenvs.in-project = true.

[dongho-jung]

I issued PR though, for getting a sense of contributing to poetry, I don't know how it could be a problem. Because, if you're in the case sensitive file system, it's the intended that two hashes are different. Otherwise, if you're in the case insensitive file system, you can't make two different entities with different case as far as I know.

[absassi]

@0xF4D3C0D3, thanks for the PR! You can't make, but you can easily enter a directory by many different paths (at least in Windows). See in the example below that depending on how was the case when you change the directory affects the output of os.getcwd (the same happens even if cwd is set outside Python):

>>> import os
>>> os.chdir('c:\\Temp')
>>> os.getcwd()
'c:\\Temp'
>>> os.chdir('..\\temp')
>>> os.getcwd()
'c:\\temp'
>>> os.chdir('C:\\')  # see note
>>> os.chdir('C:\\TEMP')
>>> os.getcwd()
'C:\\TEMP'

Note: this is needed to force the last chdir to do something, otherwise Python skips it, because it is the same path.

It's very annoying that Windows doesn't normalize the path case after changing cwd, but leave it to the applications... For instance, cmd normalizes it after calling cd, but PowerShell doesn't. So you can do the same as the example above in PS too (and also in others, like MSYS2/Git Bash):

PS C:\temp> cd C:\Temp
PS C:\Temp> cd C:\TEMP
PS C:\TEMP>

So, since the current working directory can have different case each time Poetry is invoked, it needs to compute the same hash regardless of the case in that system.

[dongho-jung]

@absassi Ooh I see, thanks for pointing it out that I can't make directories with different cases in the case insensitive file system but can enter with different cases. But I don't have Windows OS so I borrowed my sister's laptop and reproduced it with current poetry(1.0.5 / Windows 10) as follows:

PS C:\Users\****\Desktop\test> poetry new foobar
Created package foobar in foobar
PS C:\Users\****\Desktop\test> cd .\foobar\
PS C:\Users\****\Desktop\test\foobar> poetry env use python
Creating virtualenv foobar-mLxyB_lR-py3.8 in C:\Users\****\AppData\Local\pypoetry\Cache\virtualenvs
...
PS C:\Users\****\Desktop\test\foobar> cd ..
PS C:\Users\****\Desktop\test> cd  FooBar
PS C:\Users\****\Desktop\test\FooBar> poetry env use python
Creating virtualenv foobar-AmpZpXDm-py3.8 in C:\Users\****\AppData\Local\pypoetry\Cache\virtualenvs
...

As you can see both are the same path, but you can enter them with different cases (foobar, FooBar) and poetry generates different hashes for them and that's what we don't want.

The below is a test with poetry of my PR.

(poetry-3Lk8IOqn-py3.8) PS C:\Users\****\Desktop\test> poetry new foobar
Created package foobar in foobar
(poetry-3Lk8IOqn-py3.8) PS C:\Users\****\Desktop\test> cd foobar
(poetry-3Lk8IOqn-py3.8) PS C:\Users\****\Desktop\test\foobar> poetry env use python
Creating virtualenv foobar-81o3NTOh-py3.8 in C:\Users\****\AppData\Local\pypoetry\Cache\virtualenvs
...
(poetry-3Lk8IOqn-py3.8) PS C:\Users\****\Desktop\test\foobar> cd ..
(poetry-3Lk8IOqn-py3.8) PS C:\Users\****\Desktop\test> cd FooBar
(poetry-3Lk8IOqn-py3.8) PS C:\Users\****\Desktop\test\FooBar> poetry env use python
Using virtualenv: C:\Users\****\AppData\Local\pypoetry\Cache\virtualenvs\foobar-81o3NTOh-py3.8
(poetry-3Lk8IOqn-py3.8) PS C:\Users\****\Desktop\test\FooBar>

poetry doesn't make different hashes anymore. Review please thanks!

[radoering]

We encountered the same issue in a Jenkins use case with poetry 1.1.6 using poe. When executing a poetry command directly, a different virtualenv was used than when executing a task with poe. After some debugging, we encountered that poe resolves pathes while poetry does not.

A quickfix for our use case has been to replace the Path.cwd() in poetry/poetry/console/application.py by Path.cwd().resolve().

You can see the difference in a small Powershell example where the folder is actually named "Test" (with an upper case "T":

PS C:\Test> cd C:\test
PS C:\test> python -c "from pathlib import Path; print(Path.cwd())"
C:\test
PS C:\test> python -c "from pathlib import Path; print(Path.cwd().resolve())"
C:\Test

However, I think the solution from @0xF4D3C0D3 covers more use cases. We would appreciate his pull request being merged. Maybe, it should be considered if Path(cwd).resolve() should be used instead of os.path.normcase(cwd). I am not quite sure myself. If cwd cannot always be resolved (can it?), normcase is probably better. (resolve does not change the path if it cannot be resolved.) The drawback of using normcase is that the name of the virtualenv would change for many Windows users with pathes containing upper case letters so that the virtualenv has to be recreated once. Maybe, even a combination of both (Path(os.path.normcase(cwd)).resolve()) is the best solution, if pathes cannot always (but most of the times) be resolved.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.