Warn pipenv and black users when LC_ALL isn't set

[brettcannon]

See #1027

[MikhailArkhipov]

Probably part of #648

[DonJayamanne]

The underlying problem is #1254
We're not logging any errors, its just being swallowed up.

[brettcannon]

If we do #1338 and log the entire environment then this can help with this as well by providing people with those details (but it's also a lot of info, so we might want to still limit what we send to the Output window).

[brettcannon]

Talking with Christian at the PyCon US sprints, Python 3.7 is setting LC_ALL=C.UTF-8 by default if it isn't overridden. So we might as well do this as well in other versions of Python when the user has not set it to avoid this issue with Click (we should also double-check that LANG does not need to be set, otherwise it can also be C.UTF-8).

[brettcannon]

What we might need to do is also provide whatever shell details we can to help people realize why when they have set their LC_ALL appropriately why it isn't being picked up. Best solution is to simply experiment and figure out whether the shell being used is login and/or interactive and tell people that they need to make sure to use the appropriate config file (e.g. .bashrc is not used in the same instances as .bash_profile for bash users).

[brettcannon]

And if we can tell people which shell is being used then that might also help (e.g. if someone thinks they are using bash when in fact we are using just sh).

[brettcannon]

My guess is we're executing in a login, non-interactive shell and people are setting stuff in .bashrc and their .bash_profile is not sourcing their .bashrc (context).

[pzelnip]

Is there a workaround for this to set LC_ALL? As it stands, whenever I format code in VS Code using Black I end up with RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult http://click.pocoo.org/python3/for mitigation steps.. Running black on the command line in a separate terminal doesn't have this problem, so my understanding is that LC_ALL isn't set when the Python extension runs Black, but I'm not sure how I can manually set that.

[brettcannon]

@pzelnip manually setting it depends on your shell. If you're using Bash then try your .bash_profile or .bashrc file. For other shells I'm not sure.

[pzelnip]

@brettcannon My apologies, I misunderstood the conversation earlier, I assumed that from the comment "My guess is we're executing in a login, non-interactive shell and people are setting stuff in .bashrc and their .bash_profile is not sourcing their .bashrc" that changes to my bash_profile/bashrc wouldn't be reflected in the integrated VS Code terminal (which I'm assuming is where Black runs when spawned by VS Code). As it turns out, in my case the way I got it working was by setting $LC_ALL in my .bashrc as per your suggestion.

What's interesting to me at least is black was working in a Terminal window before this change (just not in VS Code). Digging into it, on my machine in a plain terminal I had the $LANG env var set to en_CA.UTF-8 but no $LC_ALL set. It appears as though click was using $LANG as if I unset $LANG in a regular terminal I got the same error about ASCII encoding.

I'm guessing that $LANG isn't set on login/non-interactive shell's, and as such isn't visible to Click when running Black in VS Code, but if it was then would work fine.

In any case, thanks for the workaround. 👍

[wcass77]

The workarounds above don't seem to work for me. I'm on High Sierra, and I just installed vs-code and the python extension (2018.8.0). I've tried adding LC_ALL to my bash_profile and bashrc files without any effect, using both export and the launchctl commands mentioned in other issues. I've uninstalled and reinstalled vs-code. I've tried launching it from finder and from the console. I've also added confirmed that in the console (both inside and outside vs-code), the environment variables are set correctly and pipenv --venv works as expected. Here is the error in full:

Workspace contains pipfile but attempt to run 'pipenv --venv' failed with 'Traceback (most recent call last): File "/Users/wcass/.local/bin/pipenv", line 11, in sys.exit(cli()) File "/Users/wcass/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in call return self.main(*args, **kwargs) File "/Users/wcass/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 676, in main _verify_python3_env() File "/Users/wcass/.local/lib/python3.6/site-packages/pipenv/vendor/click/_unicodefun.py", line 118, in _verify_python3_env 'for mitigation steps.' + extra) RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult http://click.pocoo.org/python3/for mitigation steps. This system lists a couple of UTF-8 supporting locales that you can pick from. The following suitable locales where discovered: af_ZA.UTF-8, am_ET.UTF-8, be_BY.UTF-8, bg_BG.UTF-8...

The pipenv is running python 3.6.5 installed via pyenv, and is having the same Click issue that has been mentioned elsewhere. Regardless of what I do, I can't seem to pass the enviroment variables on to the python extension so that it can execute pipenv. Any ideas how to get this to work?

[brettcannon]

@wcass77 I'll assume you didn't choose what your terminal is. That last option is .profile until we figure out how we want to solve this problem.

[wcass77]

Unfortunately, that didn't work for me either.

It's perhaps not a great idea, but I decided to just change the code inside click to set the locale to 'en_US.utf-8' (my actual locale) before checking what the locale is. This isn't for a production project, and I'm just changing it to what it will be when pipenv is called from the command line, so I don't think it should cause issues.

In the file /Users/wcass/.local/lib/python3.6/site-packages/pipenv/vendor/click/_unicodefun.py (path given in the traceback), I changed this at line 53:

try:
    import locale
    fs_enc = codecs.lookup(locale.getpreferredencoding()).name
except Exception:
    fs_enc = 'ascii'
if fs_enc != 'ascii':
    return

To this:

try:
    import locale
    # manually set locale to en_US.UTF8
    locale.setlocale(locale.LC_ALL, 'en_US.utf-8')
    #end my code
    fs_enc = codecs.lookup(locale.getpreferredencoding()).name
except Exception:
    fs_enc = 'ascii'
if fs_enc != 'ascii':
    return

There is probably a better way to change the locale (or at least a better place to put it), but this seemed like the most direct way to ensure that it was set to the correct value when it mattered, and changing the code inside pipenv/click is not a problem in my particular use case. I hope this helps.

[brettcannon]

@wcass77 If you can switch to Python 3.7 then that might also help as there was work to try and alleviate this kind of locale issue.

And sorry for the trouble. I'm hoping we can address this (and other shell-related issues) in the near future.

[wcass77]

No problem. I need to use 3.6 for this particular project, but I will definitely keep it in mind. Now that I've got everything working, I'm definitely enjoying vs code, and it's great to have it automatically run tools like pylint in the right environment. Thanks!

…

On Tue, Sep 11, 2018 at 2:36 PM Brett Cannon ***@***.***> wrote: @wcass77 <https://github.com/wcass77> If you can switch to Python 3.7 then that might also help as there was work to try and alleviate this kind of locale issue. And sorry for the trouble. I'm hoping we can address this (and other shell-related issues) in the near future. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1403 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABwjNdVfEVrRDLSanSwufrr1RHSSvGiJks5uaAK_gaJpZM4TXXZt> .

[DonJayamanne]

Closing as this hasn't been reported in the past few releases.