direnvrc: eval: <line-number>: syntax error near unexpected token `;'

[rbreslow]

Multiple users in our organization have reported sudden issues with direnv. Seems to be resolved by clearing the direnv cache rm -rf .direnv and reloading the folder.

User 1:

➜  Projects cd terraform
direnv: loading ~/Projects/terraform/.envrc
direnv: using flake
warning: Git tree '/Users/<username>/Projects/terraform' is dirty
evaluating derivation 'git+file:///Users/<username>/Projects/terraform#devShells.aarch64-darwin.default'direnv: ([/opt/homebrew/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
warning: Git tree '/Users/<username>/Projects/terraform' is dirty
direnv: nix-direnv: renewed cache
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: eval:1858: syntax error near unexpected token `;'
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: eval:1858: `            ;&'
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +DETERMINISTIC_BUILD +HOST_PATH +IN_NIX_SHELL +LD +LD_DYLD_PATH +MACOSX_DEPLOYMENT_TARGET +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_aarch64_apple_darwin +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_aarch64_apple_darwin +NIX_CFLAGS_COMPILE +NIX_DONT_SET_RPATH +NIX_DONT_SET_RPATH_FOR_BUILD +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_IGNORE_LD_THROUGH_GCC +NIX_LDFLAGS +NIX_NO_SELF_RPATH +NIX_STORE +NM +PATH_LOCALE +PYTHONHASHSEED +PYTHONNOUSERSITE +PYTHONPATH +RANLIB +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +XDG_DATA_DIRS +ZERO_AR_DATE +__darwinAllowLocalNetworking +__impureHostDeps +__propagatedImpureHostDeps +__propagatedSandboxProfile +__sandboxProfile +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH
➜  terraform git status
zsh: command not found: git

User 2:

<username>@MacBook-Pro-2 <company> % cd terraform/live/apollo/us-east-1/transfer/eff-sftp-out/
direnv: loading ~/src/freelancing/<company>/terraform/.envrc
direnv: using flake
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: line 211: local: -n: invalid option
local: usage: local name[=value] ...
direnv: nix-direnv: using cached dev shell
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: eval:1881: syntax error near unexpected token `;'
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: eval:1881: `            ;&'
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +DETERMINISTIC_BUILD +HOST_PATH +IN_NIX_SHELL +LD +LD_DYLD_PATH +MACOSX_DEPLOYMENT_TARGET +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_apple_darwin +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_apple_darwin +NIX_CFLAGS_COMPILE +NIX_DONT_SET_RPATH +NIX_DONT_SET_RPATH_FOR_BUILD +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_IGNORE_LD_THROUGH_GCC +NIX_LDFLAGS +NIX_NO_SELF_RPATH +NIX_STORE +NM +PATH_LOCALE +PYTHONHASHSEED +PYTHONNOUSERSITE +PYTHONPATH +RANLIB +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +ZERO_AR_DATE +__darwinAllowLocalNetworking +__impureHostDeps +__propagatedImpureHostDeps +__propagatedSandboxProfile +__sandboxProfile +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH

<username>@MacBook-Pro-2 eff-sftp-out % ps
zsh: command not found: ps

All of these users are on macOS and have used the following instructions to install Nix, direnv, and nix-direnv:

$ curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
$ brew install direnv
$ echo eval "$(direnv hook zsh)" >>~/.zshrc
$ nix profile install nixpkgs#nix-direnv
$ echo 'source $HOME/.nix-profile/share/nix-direnv/direnvrc' >>~/.config/direnv/direnvrc

It's never happened on my machines, but I have a different set up with a laptop running nix-darwin and a NixOS desktop.

[rbreslow]

#451 (comment)

cc: @simonzkl, just read your comment, and sounds possibly related?

[simonzkl]

I don't think this is related. It kinda sounds like zsh executing a bash script or something like that.

[rbreslow]

FWIW, this is the contents of our .envrc file:

$ cat ~/projects/terraform/.envrc   
use flake

[rbreslow]

I instructed one of our users to rm -rf .direnv to see if clearing the cache helps.

He now is seeing this error:

<username>@MacBook-Pro-2 ~ % cd src/freelancing/<company>/terraform/live/apollo/us-east-1/transfer/eff-sftp-out/
direnv: loading ~/src/freelancing/<company>/terraform/.envrc
direnv: using flake
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: line 211: local: -n: invalid option
local: usage: local name[=value] ...
warning: unknown setting 'upgrade-nix-store-path-url'
evaluating derivation 'git+file:///Users/<username>/src/freelancing/<company>/terraform#devShells.x86_64-darwin.default'direnv: ([/usr/local/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
warning: unknown setting 'upgrade-nix-store-path-url'
warning: unknown setting 'upgrade-nix-store-path-url'
warning: unknown setting 'upgrade-nix-store-path-url'
warning: unknown setting 'upgrade-nix-store-path-url'
warning: unknown setting 'upgrade-nix-store-path-url'
warning: unknown setting 'upgrade-nix-store-path-url'
direnv: nix-direnv: renewed cache
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: eval:1881: syntax error near unexpected token `;'
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: eval:1881: `            ;&'
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +DETERMINISTIC_BUILD +HOST_PATH +IN_NIX_SHELL +LD +LD_DYLD_PATH +MACOSX_DEPLOYMENT_TARGET +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_apple_darwin +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_apple_darwin +NIX_CFLAGS_COMPILE +NIX_DONT_SET_RPATH +NIX_DONT_SET_RPATH_FOR_BUILD +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_IGNORE_LD_THROUGH_GCC +NIX_LDFLAGS +NIX_NO_SELF_RPATH +NIX_STORE +NM +PATH_LOCALE +PYTHONHASHSEED +PYTHONNOUSERSITE +PYTHONPATH +RANLIB +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +ZERO_AR_DATE +__darwinAllowLocalNetworking +__impureHostDeps +__propagatedImpureHostDeps +__propagatedSandboxProfile +__sandboxProfile +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH

The warning: unknown setting 'upgrade-nix-store-path-url' stands out.

[Mic92]

What bash version are you using?
Try to use if you can reproduce the issue with bash installed with nix.

[rbreslow]

What bash version are you using? Try to use if you can reproduce the issue with bash installed with nix.

I'm instructing users to report their shell (I'm pretty sure we all use zsh) and shell version.

I'm also having folks try a bash shell with Nix via:

$ echo 'eval "$(direnv hook bash)"' >>~/.bashrc
$ nix run nixpkgs#bash
$ cd <our-project-directory> 
$ direnv allow

I will report back when I have the results.

[Mic92]

It's also fine to use zsh as the interactive. It's about the version of bash in $PATH (nix-shell -b bash)
Just let them verify that bash --version returns something like 5.x i.e 5.2.26

Even if you zsh as your shell, direnv will use bash internally to evaluate .envrc

[rocky-foreflight]

On my system bash --version returns:

$ bash --version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin23)
Copyright (C) 2007 Free Software Foundation, Inc.

Also:

[MacBook-Pro ~]$ nix shell nixpkgs#bash 
[MacBook-Pro ~]$ bash --version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin23)
Copyright (C) 2007 Free Software Foundation, Inc.

Since they're all macOS users, I suspect something similar.

I also have nix-darwin installed and installed direnv/nix-direnv with home-manager. So maybe that's why I'm not encountering any issues?

[Mic92]

There is actually a version check that should fail loudly if someone is using an too old bash version. Not sure why this check is not working properly. Otherwise you will need at least bash4. If you use install nix-direnv via home-manager that should already make sure that a new version of bash is put in place. Actually it's enough to install direnv via nix because nix in nixpkgs hardcode the version of bash used.

[Mic92]

https://github.com/NixOS/nixpkgs/blob/7858badcfc69157fae059a616e989ca055f2829a/pkgs/tools/misc/direnv/default.nix#L23

[rocky-foreflight]

From one of our users:

I’m using zsh.

<username>@MacBook-Pro-2 d2 % which zsh; zsh --version
/bin/zsh
zsh 5.9 (x86_64-apple-darwin22.0)

Running ventura 13.6.3, fwiw.

Tried running in a bash shell, same issue:

\[\]MacBook-Pro-2:terraform <username>$ \[\]\[\]direnv allow
direnv: loading ~/src/freelancing/<company>/terraform/.envrc
direnv: using flake
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: line 211: local: -n: invalid option
local: usage: local name[=value] ...
direnv: nix-direnv: using cached dev shell
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: eval:1881: syntax error near unexpected token `;'
/Users/<username>/.nix-profile/share/nix-direnv/direnvrc: eval:1881: `            ;&'
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +DETERMINISTIC_BUILD +HOST_PATH +IN_NIX_SHELL +LD +LD_DYLD_PATH +MACOSX_DEPLOYMENT_TARGET +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_apple_darwin +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_apple_darwin +NIX_CFLAGS_COMPILE +NIX_DONT_SET_RPATH +NIX_DONT_SET_RPATH_FOR_BUILD +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_IGNORE_LD_THROUGH_GCC +NIX_LDFLAGS +NIX_NO_SELF_RPATH +NIX_STORE +NM +PATH_LOCALE +PYTHONHASHSEED +PYTHONNOUSERSITE +PYTHONPATH +RANLIB +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +ZERO_AR_DATE +__darwinAllowLocalNetworking +__impureHostDeps +__propagatedImpureHostDeps +__propagatedSandboxProfile +__sandboxProfile +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system -PS2 ~PATH ~XDG_DATA_DIRS
\[\]MacBook-Pro-2:terraform <username>$ \[\]\[\]aws-vault exec apollow.serv^C
\[\]MacBook-Pro-2:terraform <username>$ \[\]\[\]aws-vault exec apollo.server-dev -- terragrunt plan
aws-vault: error: --prompt value must be one of terminal, got 'osascript', try --help
\[\]MacBook-Pro-2:terraform <username>$ \[\]\[\]ps
bash: ps: command not found

But! Switching from the homebrew to nixpkg version of direnv seems to resolve the issue.

[bbenne10]

So I think this means that our suggestion of installing direnv from Homebrew (in the README) is inappropriate and needs to be re-written? I think the new suggestion should simply be to install direnv from nixpkgs?

➜ nix shell nixpkgs#direnv

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
bash-3.2$ direnv exec . which -a bash
direnv: loading ~/code/personal/nix-direnv/.envrc
direnv: using flake
direnv: nix-direnv: using cached dev shell
/nix/store/4ia9i014ajl2l2cj75wzv5m463cisn4z-bash-5.2p26/bin/bash
/run/current-system/sw/bin/bash
/bin/bash

Does this show that the nixpkgs-provided direnv is wired to launch nixpkgs-provided bash too? If that's so - it should be enough to say that - on macOS - direnv should be installed from nixpkgs (Or you can get a supported bash version some other way - but I don't want to muddy the waters by going into it deeply. There's plenty of other complications in our documentation and I would like to avoid making it more complicated. :P )

[Mic92]

I thought we fixed homebrew to also build with its own version bash? Homebrew/homebrew-core@b800fa7#diff-fb911d9f81daeaf60cca24c468e574351ef2dabe8845478bfedf1ac931587cb4R24

[bbenne10]

Ah - I had forgotten that we'd done this.
Maybe it's not installed from a new enough revision?

[szlend]

Possibly related to NixOS/nix#10263?

[iFreilicht]

@szlend indeed. We're working on a fix in nixpkgs and Nix to make the error message better.

@Mic92 The check is not executed when direnv runs because nix print-dev-env (which direnv executes when use flake is used (and the equivalent command in the old CLI for use nix)) only checks what environment variables and functions were defined after running setup.sh and then builds a new script from that. Not the whole setup.sh content is imported.

The important part for users is to use bash with version 4.0.0 or higher. It can be installed from brew or nix, that doesn't matter, it just needs to be first in the path, so direnv shelling out to bash will call that more recent version

Another potential workaround is to not use nixpkgs unstable, but 23.11, as it does not include commit NixOS/nixpkgs@e07a2fa, which is the source of this error.

EDIT: And the third option is to install direnv explicitly from Nix, and ensure only that version is hooked into the shell, because that version will only call the current version of bash from nixpkgs (currently 5.2+)

[bbenne10]

@iFreilicht: Which check are you talking about above? I believe @Mic92 is referring to this check, which should be run unequivocally because it is checked before we execute print-dev-env. I would expect that the invoked bash version should be the same between the direnv invoked shell and the one behind the scenes in print-dev-env, but maybe I'm wrong?

(I will be very happy once --replace-fail makes it to widespread adoption. It is much better than the current default - we should not be failing silently :) )

[iFreilicht]

Oh sorry, I was talking about this one. Disregard my comment about this, then.

[iFreilicht]

I would expect that the invoked bash version should be the same between the direnv invoked shell and the one behind the scenes in print-dev-env, but maybe I'm wrong?

This depends on how direnv is set up. I actually opened this issue in the direnv github because I had the same problem: direnv/direnv#1256
And as it turned out I had two versions of direnv installed and hooked into the shell, with the one using system bash taking precedence.

direnv itself will use the exact bash path that was specified at compile time, or system bash if nothing was specified at compile time. So, when using direnv installed by Nix, you should be fine. But, if you have direnv installed from brew or some other source, then it will use system bash.

[bbenne10]

I thought we fixed homebrew to also build with its own version bash? Homebrew/homebrew-core@b800fa7#diff-fb911d9f81daeaf60cca24c468e574351ef2dabe8845478bfedf1ac931587cb4R24

Does this not satisfy the "specified at compile time"? This should say that the only bash to use is the one pointed at here, so we can be sure that the bash version we get is one we expect (and in fact - one from a known up-to-date source). and yet we've still got this bug report. So I'm not entirely sure that we've nailed down the entirety of the root cause here.

[iFreilicht]

Hmm, it definitely should, the relevant part of the makefile is pretty clear here.

But how do you know that's how the affected users installed direnv? In my case, the offending version was installed through asdf-vm via the asdf-direnv plugin.

[bbenne10]

A co-worker just pointed out to me that /usr/libexec/path_helper is going to be the "root" cause of the problem in apple provided bash installs (maybe also apple-provided zsh installs? I haven't tested). path_helper puts the Apple provided PATH directories at the front no matter where they are in your current PATH so the Apple provided binaries are always found first if you invoke path_helper.

If you have:

1. A version of direnv installed without the tightening of the Bash version (via the BASH_PATH make variable in either nixpkgs or homebrew) AND
2. An apple provided bash shell as the first bash in $PATH AND
3. Not modified your bash invocation to avoid calling /usr/libexec/path_helper

You'll run into this exact problem.

You're right to point out that we can't control how direnv gets installed - that's outside of our control (I just wish we could avoid triaging bugs that aren't our purview. Many - if not most - of our bugs these days seem to be improper setups or stuff like this that isn't really a thing we control...).

The resolution here is - I think - to update direnv (if installed from nixpkgs or homebrew) OR to install a newer version of bash and avoid calling /usr/libexec/path_helper somehow OR to have nix/nix-darwin manage your shell for you (which avoids calling path_helper at all because it is known to cause issues exactly like this).

EDIT: It still isn't clear to me though why we're not failing earlier in the invocation by screaming about the bash version though...

EDIT2:

But how do you know that's how the affected users installed direnv?

But! Switching from the homebrew to nixpkg version of direnv seems to resolve the issue.

[danielo515]

I was facing this same problem, and the issue was, indeed the bash version.
It will be nice to run a check for the bash version available and provide a better error message, given this is a known requirement

[bbenne10]

We check the version of the invoking shell. We document the version requirement.

I am unsure why the invoking shell isn't triggering. Happy to accept PRs improving the situation.

[danielo515]

Ok, somehow I ended in this repo from devenv, but I think I didn't installed direnv from nix-direnv but plain direnv. Sorry for the confusion

[bbenne10]

Devenv's got a version of nix-direnv built in. You'd need to speak with them about their "fork". Direnv also has a non-caching implementation of use flake and use nix built-in, which may or may not be interesting to you.

[bbenne10]

Given that there is nothing here for the nix-direnv developers to do, I am closing this issue. I think we ought to pin either this issue or a similar one so that users can find it quickly and without reopening a similar one (provided that they actually look).

This is frustrating, but I do not know how we can check the versions of bash with which our dependencies are invoked :/

[rbreslow]

All good. @bbenne10 – sorry for my radio silence. We fixed the original issue in my org by 1) asking people to do a brew upgrade direnv or 2) asking people to uninstall direnv from Homebrew and install direnv via nix profile install nixpkgs#direnv.