cmd/go: always pass -lang to cmd/compile

[zigo101]

Go version

go version go1.22.0 linux/amd64

Output of go env in your module/workspace:

.

What did you do?

main.go:

//go:build go1.21

package main

func main() {
	for i, p := 0, (*int)(nil); p == nil; println(p == &i) {
		p = &i
	}
}

go run main.go

What did you see happen?

true

What did you expect to see?

false

Note that when the -gcflags=-lang=go1.22 compiler flag is specified, then the "//go:build go1.xy" comment directive is respected (go run . with a go.mod file also respects it).

$ gotv 1.22. run main.go 
[Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run main.go
false
$ gotv 1.22. run -gcflags=-lang=go1.22 main.go 
[Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run -gcflags=-lang=go1.22 main.go
true
$ gotv 1.22. run .
[Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run .
true
$ gotv 1.22. run -gcflags=-lang=go1.22 .
[Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run -gcflags=-lang=go1.22 .
true

[bcmills]

This is closely related to

• cmd/go: always pass a GoVersion to cmd/vet #65612
  and https://go.dev/cl/567435 (CC @samthanawalla @matloob), in that cmd/go sometimes doesn't pass a language version for a main.go file when it could.

However, in the absence of any -lang flag, I expect cmd/compile to still apply whatever version information it has inferred from the //go:build directives, and it appears not to be doing so. This looks like a compiler bug.

[mknyszek]

CC @golang/compiler

[zigo101]

@bcmills
Just a btw question: it looks the -gcflags=-lang=go1.xy compiler flag is only applied to the seed files being passed to the compiler. Other involved source files in building will not be affected by the flag. Is this the intended design?

[mknyszek]

In runtime/compiler triage, we think this is a Go command issue? Is that right? Thanks.

[bcmills]

I think it is a composite of a cmd/go bug (not passing the -lang flag) and a cmd/compile bug (not using //go:build version hints when -lang is not explicitly set).

The cmd/go bug should be fixed in #65612, so I suggest we focus the investigation in this issue on the cmd/compile side.

[zigo101]

Will this be fixed in 1.22.3+?

[griesemer]

When no -lang is specified, in general we don't know how to interpret //go:build lines in files because their interpretation may depend on the current -lang version.

cc: @rsc for input.

[zigo101]

Is it a good idea to use the version of the go command as the -lang arg?

[rsc]

@go101 Is the top comment wrong? It says got true, expected false for go run main.go, but with Go 1.21 I get false and Go 1.22 I get true. If you think there is a bug in Go 1.22, then did you mean "got false, expected true"?

[zigo101]

I mean the //go:build go1.21 line should matter, so that the results should be consistent with Go toolchain 1.21 and 1.22.

[rsc]

I'm just trying to puzzle through this all since I don't know what "gotv 1.22." does either...

[zigo101]

gotv is a tool used to switch Go toolchain versions conveniently: https://go101.org/apps-and-libs/gotv.html

gotv 1.22. means using the latest version prefixed by 1.22. It will be expanded to $HOME/.cache/gotv/tag_go1.22.0/bin/go. :)

[rsc]

For future bug reports it would help to use reproduction instructions that are limited to standard tooling.

[rsc]

Here is a clearer reproduction case:

% cd /tmp
% cat x.go
//go:build go1.21

package main

func main() {
	for i, p := 0, (*int)(nil); p == nil; check(p == &i) {
		p = &i
	}
}

func check(b bool) {
	if b {
		println("using old semantics")
	} else {
		println("using new semantics")
	}
}
% GOTOOLCHAIN=go1.21.0 go run x.go
using old semantics
% GOTOOLCHAIN=go1.22.0 go run x.go
using new semantics
% GOTOOLCHAIN=go1.22.0 go run -gcflags=-lang=go1.22 x.go
using old semantics
% 

[rsc]

CL 567435 in progress should fix this. The fix is to just make sure that cmd/go passes -lang to the compiler always.

I looked into changing what the compiler does when -lang is empty, but right now it intentionally ignores the //go:build lines to avoid downgrades, and for good reason (for example see GOROOT/src/internal/types/testdata/check/go1_xx_19.go). If we change the compiler, we risk breaking other uses of the compiler like Bazel. And the go command needs the logic anyway to help cmd/vet, so we might as well just make sure the go command always makes the decision.

[gopherbot]

Change https://go.dev/cl/567435 mentions this issue: cmd/go: set the GoVersion for files listed on the commandline

[gopherbot]

Change https://go.dev/cl/591136 mentions this issue: cmd/go: set the GoVersion for files listed on the commandline with vet

[gopherbot]

Change https://go.dev/cl/593156 mentions this issue: cmd/go: set GoVersion for files on the command line with vet

[zigo101]

Will this be backported to 1.22 and 1.23?

[samthanawalla]

@gopherbot pretty please backport this to 1.21 and 1.22

[gopherbot]

Backport issue(s) opened: #68048 (for 1.21), #68049 (for 1.22).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

[gopherbot]

Change https://go.dev/cl/593315 mentions this issue: cmd/go: set GoVersion for files on the command line with vet

[gopherbot]

Change https://go.dev/cl/593295 mentions this issue: cmd/go: set GoVersion for files on the command line with vet

[gopherbot]

Change https://go.dev/cl/593297 mentions this issue: go/analysis/passes/buildtag: retire Go 1.15 support

[gopherbot]

Change https://go.dev/cl/593376 mentions this issue: all: update vendored golang.org/x/tools

[matloob]

@zigo101 We're looking into backporting it, but we'd like more information to see if this meets the bar to be backported. The backports policy states that backports should be "fixes for security issues, serious problems with no workaround, and documentation fixes". This isn't a security issue or documentation fix, so the question is whether this is a serious problem with no workaround. Could you give us more information about whether this is a serious problem with no workaround?

[zigo101]

Personally, I think it is a serious problem with no workaround.
Without this fix, Go breaks Go 1 promise of compatibility with any standard.

[matloob]

Hi @zigo101, could you provide more information about what your use case is and why there is no workaround?

[zigo101]

@matloob

The example in the first comment of this thread is my use case.
I use some alike single Go files as scripts (use go run xyz.go to run it).
I don't want to modify the source other than simple adding a //go:build go1.21 line at the start.

What is the workaround do you think?

[matloob]

Hi, the workaround would depend on the environment your programs is running in, what the program does, why you can't modify the programs, etc.

The best workaround would be to put your each of your single file programs in a module and run the program as a package instead of passing in individual files to the go command. If your program depends on the semantics of the version of Go it's being run under, that's your best bet, and I would recommend doing that even after this change is in. What this change does is always pass -lang to cmd/compile. But the primary way to set that -lang is through go.mod, and running your program in a module context.

There are other, less desirable, things you could do to get your builds to work: for example, you could pass in the -gcflags flag as you suggested in your first comment or modify the source. I understand that those options do not result in a good user experience, so I'd recommend going the module route if you can.

[zigo101]

Okay, let me explain it clearly. I have written many Go books and tutorials. I just want to show gophers that how to use //go:build directives to control code behaviors. I hope the content works with Go toolchain v1.22 and v1.23. I hope gophers don't need the workarounds you provided, because it should work just by using //go:build directives, as the Go docs says and as users expect.

Without the fix for v1.22 and v1.23, it is hard to say v1.22 and v1.23 don't break Go 1 promise of compatibility.

[zigo101]

BTW, maybe it is meaningless to backport this to 1.21.
The behavior of panic(nil) is not controlled by go versions.

[matloob]

@zigo101 Thanks for your explanation. Unfortunately this doesn't meet the bar for backporting. It doesn't seem like there are user workflows that are broken by the incorrect behavior with //go:build downgrades in Go 1.22. And although we acknowledge that it isn't ideal to have to use a workaround, a workaround does exist.

[zigo101]

It doesn't seem like there are user workflows that are broken by the incorrect behavior with //go:build downgrades in Go 1.22.

Maybe, The //go:build hack only become so important since Go 1.22. :D
It is used here just to keep some old code behavior unchanged with a minimal effort.
As for workarounds, I think most of other backported issues also have their own wordarounds.

Persnoally, I just hope this fix can be backported, to let your Go dev get a better reputation on code quality.
But if you don't care about it much, I have nothing more to say. :D

This will not affect my production code, because I'm fully aware all the problems caused
by the new semantics of tranditional for;; loops and related bugs in the old toolchain versions.
I just can't make sure whether or not these problems and bugs will affect other people's production code.

I will help you make this known to my readers on my website, books and social platforms.

[samthanawalla]

@zigo101 I'm sorry, I know it's frustrating. Thanks for looking out for Go developers and sharing this with your audience. You're doing the community a solid.

[zigo101]

Just found this fix has been merged into v1.23rc1. Interesting, so it is worth being merged into 1.23, but not 1.22.

[dmitshur]

@zigo101 When fixes are made, they're included in the next major release. That is the default, and this resolved issue is in the Go1.23 milestone. Only in rare cases when it is deemed necessary they are also backported to currently supported major releases (1.22 and 1.21 at this time). See https://go.dev/s/release and https://go.dev/wiki/MinorReleases for more information on the Go release process.

[zigo101]

Okay, glad to know it is deliberately.