The Marshal method in the yaml.v3 package is generating potentially invalid yaml

[debajyoti-truefoundry]

Please find a minimal reproduction here:- https://go.dev/play/p/HfTJF55cg69

package main

import (
	"fmt"
	"log"

	"gopkg.in/yaml.v3"
)

type StringMount struct {
	Data      string `yaml:"data"`
	MountPath string `yaml:"mount_path"`
}

type Mounts struct {
	StringMounts []StringMount `yaml:"mounts"`
}

func newMounts(data string, mountPath string) *Mounts {
	var mounts Mounts
	mounts.StringMounts = append(mounts.StringMounts, StringMount{Data: data, MountPath: mountPath})
	return &mounts
}

func main() {
	mounts := newMounts("\n</configuration>\n", "/bar")
	fmt.Println(mounts)

	yamlData, err := yaml.Marshal(&mounts)
	if err != nil {
		log.Fatalf("Marshal: %v", err)
	}

	fmt.Println(string(yamlData))

	var newMounts Mounts
	err = yaml.Unmarshal(yamlData, &newMounts)
	if err != nil {
		log.Fatalf("Unmarshal: stdlib: %v", err)
	}

	fmt.Println(newMounts)
}

Output:

&{[{
</configuration>
 /bar}]}
mounts:
    - data: |4
        </configuration>
      mount_path: /bar

2009/11/10 23:00:00 Unmarshal: stdlib: yaml: line 1: did not find expected key

Program exited.

If I change gopkg.in/yaml.v3 to gopkg.in/yaml.v2 in the import block, the Unmarshal method works fine.

[rogpeppe]

I did a little bit of investigation into this.

Firstly, it seems to be a regression from v2.

Secondly, this issue has been reported a few times before. Duplicate issue numbers are #940, #963 and #968. There may be others.

Here's a test program that demonstrates the issue with a few variations that may or may not be due to the same underlying cause: https://go.dev/play/p/DBp2CQrlBDa

For the record, this is the problematic test data:

type A struct {
	B []B
}

type B struct {
	C string
}

type D struct {
	B []string
}

var tests = []any{
	A{
		B: []B{{
			C: "\nx",
		}},
	},
	A{
		B: []B{{
			C: "\n",
		}},
	},
	D{
		B: []string{"\nx"},
	},
}

Not all the tests fail at all commits in the v3 history. Some commits at which behaviour seems to have changed:

• test #0 fail:
  fc85683 Tue Apr 9 11:22:14 2019 +0100
• test #2 fail:
  55513ca Thu May 2 11:37:01 2019 +0100
• tests #0, #1, and #2 fail:
  496545a Thu Jan 7 19:29:22 2021 +0000

[rogpeppe]

Also for the record, this doesn't appear to be to do with the Go struct marshaling logic. The same test failures occur when using anonymous types: https://go.dev/play/p/spthrUBMdwS