Add pwsh PowerShell interpreter

[ghost]

PowerShell is using #!/usr/bin/env pwsh shebang on Unix these days and pwsh is "the thing" that runs powershell :)

I assume after this change, the following would show syntax highlighting:

```pwsh
Get-ChildItem .
```

as

Get-ChildItem .

Description

Checklist:

• I am associating a language with a new file extension.

  • The new extension is used in hundreds of repositories on GitHub.com
    • Search results for each extension:
      • https://github.com/search?utf8=%E2%9C%93&type=Code&ref=searchresults&q=extension%3AFOOBAR+KEYWORDS+NOT+nothack
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am adding a new language.

  • The extension of the new language is used in hundreds of repositories on GitHub.com.
    • Search results for each extension:
      • https://github.com/search?utf8=%E2%9C%93&type=Code&ref=searchresults&q=extension%3AFOOBAR+KEYWORDS+NOT+nothack
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a syntax highlighting grammar.
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am fixing a misclassified language

  • I have included a new sample for the misclassified language:
    • Sample source(s):
      • [URL to each sample source, if applicable]
    • Sample license(s):
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am changing the source of a syntax highlighting grammar

  • Old: https://github-lightshow.herokuapp.com/
  • New: https://github-lightshow.herokuapp.com/

• I am adding new or changing current functionality

  • I have added or updated the tests for the new or changed functionality.

[pchaigno]

Thanks for the contribution!
Would you have an example file with such a shebang we could use (i.e., under permissive license)?

I assume after this change, the following would show syntax highlighting:

Yes it will.

[ghost]

@pchaigno, thanks! i have found https://github.com/PowerShell/PowerShellStandard/blob/3436bfc162d6804dd11d1d76c4faff486b4b405d/build.ps1.

This repo is MIT Licensed https://github.com/PowerShell/PowerShellStandard/blob/master/LICENSE.txt

[pchaigno]

That'll do. Could you add it under samples/PowerShell/?

[ghost]

@pchaigno, added. I renamed the file to shabang.ps1, to clarify its purpose.

[pchaigno]

Uh, I almost missed that. To add a new shebang, you need to add pwsh under the interpreters key, no the aliases one.

[ghost]

Thanks. I have added it under both Shell and PowerShell. One of the Ruby 2.3.3 job is failing to clone:

Error updating vendor/grammars/xc.tmbundle
Cloning into 'vendor/grammars/xc.tmbundle'...
error: RPC failed; result=35, HTTP code = 0
fatal: The remote end hung up unexpectedly

[pchaigno]

I have added it under both Shell and PowerShell.

I though this was a PowerShell interpreter only??

One of the Ruby 2.3.3 job is failing to clone:

That happens sometimes. Probably a momentary issue with Travis CI. I relaunched the build.

[ghost]

If we have pwsh installed on linux, and execute ps1 script from different shell it picks up the interpreter, just like ash/dash etc.

[pchaigno]

But pwsh cannot interpret Bash scripts, right?

[ghost]

Yes it works the other way around as well: executing an sh script with bash shebang in pwsh. :)

[ghost]

Confirmed on IRC (#PowerShell @freenode):

 <kasper> with pwsh shebang, executing a ps1 script in sh or bash etc. invokes pwsh interpreter, but does the opposite works? invoking an sh script with bash shebang in pwsh?
 <+jcotton> yes
 <+jcotton> PS doesn't look at it, it just goes through the standard execution mechanisms
 <+jcotton> shebangs are handled by the kernel, not by the shell

[ghost]

@pchaigno, did it answer your question? I tested on Alpine Linux that using ash, you cannot execute bash script unless you have bash installed. Similarly from ash or bash to pwsh. The other way around also works, that from pwsh, you can execute ash or bash script. From that granularity, it is yet another scripting language with interpreter of its own (like all the rest of them).

[ghost]

@lildude, can this be selected for the next release?
@SteveL-MSFT, @TravisEz13, FYI

[SteveL-MSFT]

@kasper3 very cool! Thanks for this!

[lildude]

@lildude, can this be selected for the next release?

Yup, once all the necessary reviews are in - I don't review until PRs have been reviewed by at least one of the other primary contributors. I plan to make a new release later this week so it'll hopefully be reviewed before then.

[Alhadis]

Installed and confirmed a working pwsh example on macOS with Homebrew's cask:

#!/usr/bin/env pwsh
Write-Output "Hello, world."

Couldn't find a package for PowerShell on Ubuntu, however.

[ghost]

Thanks, for Ubuntu, we need to add repo and key. The instructions are given here for each version of Ubuntu: https://github.com/powershell/powershell#get-powershell.

[Alhadis]

This should be removed. The Shell entry refers specifically to POSIX/Bourne-derived shells, rather than any generic shell.

[pchaigno]

Apparently, the PowerShell interpreter can also interpret Bash scripts. I was able to confirm under Ubuntu with a simple:

#!/usr/bin/env pwsh
echo "ok"
pwd
cd /
pwd

[pchaigno]

Not sure this is actually used as a shebang for Bash in-the-wild (on GitHub.com) though...

[Alhadis]

If so, users should be identifying it using #!/bin/sh or a traditional interpreter.

There's already enough inconsistencies in the shell world...

[ghost]

pwd echo cd /, ls, curl all work on Windows PowerShell since Vista/2006 (without any Unix-y stuff installed). These are official registered aliases to actual powershell cmdlets. If you try bash-only syntax, that will fail.

[Alhadis]

@kasper3 Sorry, I just noticed pwsh was listed under the Shell entry as an interpreter too.

Unless there's a completely unrelated interpreter called pwsh for POSIX shell-scripts, it should be removed.

[ghost]

@Alhadis, thanks, we can remove it for sure. However, PowerShell grammar is based on POSIX 1003.2 (aka POSIX.2) based on https://books.google.com/books?id=MYZQAAAAMAAJ. We can invoke sh,ash,csh,bash,dash etc. scripts from pwsh on Unix. We can conversely invoke pwsh from sh,ash,csh,bash,dash etc. on Unix. Is there any substantial missing feature that makes in ineligible to get enlisted under Shells section?

[pchaigno]

Is there any substantial missing feature that makes in ineligible to get enlisted under Shells section?

That would be usage, which, as far as I can see^Wsearch, is not there yet: "env pwsh" extension:sh.

[Alhadis]

No sensible shell-script author would ever write a POSIX shell-script and specify an interpreter using anything other than sh, bash, or any of the usual "traditional" interpreters.

[ghost]

@Alhadis, done. In our build automation, we have started to use powershell DSC. Though our file extensions are .ps1. Technically, we can have an .sh file with pwsh shebang. But I agree, statically on GitHub, pwsh shebang is not as viral to warrant an addition under Shell: section.

[Alhadis]

However, PowerShell grammar is based on POSIX 1003.2

Uh, no. It isn't POSIX-compatible whatsoever:

a=2
if [ $a -eq 3 ]; then
	printf '%s\n' Foo
elif [ $a -ne 3 ]; then
	printf '%s\n' Bar
fi

pwsh output:

At line:2 char:3
+ if [ $a -eq 3 ]; then
+   ~
Missing '(' after 'if' in if statement.
At line:2 char:5
+ if [ $a -eq 3 ]; then
+     ~
Missing type name after '['.
+ CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : MissingOpenParenthesisInIfStatement

Any POSIX-compliant shell should output Bar. 😉

There's also a side-by-side comparison here which clearly illustrates the differences in syntax between the various command-line interpreters.

[ghost]

I read it https://en.wikipedia.org/wiki/PowerShell#Design

[Alhadis]

PowerShell's developers based the core grammar of the tool on that of POSIX 1003.2.[21]

[21] "Payette, Bruce (2007). Windows PowerShell in Action. Manning Pubs Co Series. Manning. p. 27. ISBN 9781932394900. Retrieved 2016-07-22. The core PowerShell language is based on the POSIX 1003.2 grammar for the Korn shell."

It says "based on", which is loosely subjective and denounced quite clearly in the form of a syntax error. I've no idea what they could have been referring to...

[ghost]

It says "based on"

Yup, that's what i said However, PowerShell grammar is based on POSIX 1003.2

Perhaps POSIX IEEE Std 1003.2 does not mandate the syntax to be if [ $a -eq 2 ] then; ... ; fi vs. if ($a -eq 2) { ... }?

[Alhadis]

The syntax ($a -eq 2) describes a command to run inside a subshell, not an expression. There's actually no requirement for any sort of punctuation to separate if from a condition. E.g., this is legal syntax in any POSIX-compliant or Bourne-derived shell:

if this; then that; fi

In fact, if ($foo) { isn't even legal shell grammar, because foo(){ denotes the beginning of a function definition, which wouldn't actually be possible due to if being a reserved keyword.

Anyway, we've clearly established that PowerShell is by no means a "normal" interpreter for POSIX shell grammar.

[ghost]

I meant maybe IEEE Std 1003.2 does not cover syntax at all and only utilities. Otherwise that would be a lie that PowerShell conforms with the said standard, right?

[Alhadis]

Utilities have nothing to do with shells†. There's GNU on Windows, which makes the standard Unix toolchain available to cmd.exe or even COMMAND.COM. The fact they're available in PowerShell – either as builtins or bundled utilities – really doesn't have much relevance to this discussion.

† - Unless they're implemented as built-ins (like Bash's true and test), but let's not digress this conversation further.

[pchaigno]

Thank you for your patience @kasper3!

[lildude]

LGTM. Thanks for the contribution and welcome to Linguist.

[ghost]

@lildude, was this change picked in the release?

```pwsh
Get-ChildItem test
```

We don't get the expected syntax highlighting:

Get-ChildItem test

as opposed to ```poweshell:

Get-ChildItem test

[lildude]

Yup, it was. However looking back at this PR, pwsh was only added as an interpreter so files that start with a shebang of #!/usr/bin/env pwsh will be highlighted as PowerShell.

Inline code blocks aren't affected by this so PowerShell still needs to be used. For pwsh to be used in code blocks, an alias needs to be created. Looks like we addressed the first point in your OP but completely missed your assumption.

[pchaigno]

@lildude Perhaps interpreters should be included in aliases for a language though?

[ghost]

A quick test reveled that is the case: https://github.com/kasper3/linguist/blob/master/test1 (extension less file with pwsh shebang get picked up for syntax highlighting).

I agree with @pchaigno's idea of implicitly including interpreters as aliases.

[lildude]

@lildude Perhaps interpreters should be included in aliases for a language though?

I agree, though some cases this may not be always desired. When I've got a free moment, I'll see if I can work out why this isn't the case.

[pchaigno]

@kasper3 Could you please open a separate issue so we don't forget about this?

[lildude]

And feel free to open a PR adding pwsh as an alias too.

[ghost]

Thanks, I have opened a pull request #4117 :)