Fix for #215 Anchors don't decode to the same pointer + Allowing recursive references

[vinzenz]

This mainly is fixing Issue #215 however I have added at the same time support for recursive decoding, since those things should go basically hand in hand.

I took the liberty and rebased my changes on top of PR #210 because I can see that these changes would go the same way and would need a new parser as well. Therefore I added it on top of that.

Here is a test case where the new code comes into play:

package main

import (
	"errors"
	"log"

	"gopkg.in/yaml.v2"
)

type NoopRes struct {
	Path    string  `yaml:"path"`
	Content *string `yaml:"content"` // XXX *string
}

type Recursive struct {
	Self *Recursive `yaml:"self"`
	This *Recursive `yaml:"this"`
	Name string     `yaml:"name"`
}

type GraphConfig struct {
	Graph     string `yaml:"graph"`
	Resources struct {
		Noop      []*NoopRes `yaml:"noop"`
		Recursive Recursive  `yaml:"other"`
	} `yaml:"resources"`
	Comment string `yaml:"comment"`
}

func (c *GraphConfig) Parse(data []byte) error {
    decoder := yaml.NewDecoder()
    decoder.SetAllowRecursive(true)
	if err := decoder.Unmarshal(data, c); err != nil {
		return err
	}
	if c.Graph == "" {
		return errors.New("Graph config: invalid `graph`")
	}
	return nil
}

func main() {
	log.Printf("Hello!")
	str :=
		`

---
graph: mygraph
comment: my comment
resources:
  noop:
  - name: noop1
    path: foo
    content: &ptr "hello"
  - name: noop2
    path: bar
    content: *ptr
  - name: noop3
    path: baz
    content: *ptr
  unused:
  - foo: bar
  other: &recursive
    self: *recursive
    this: *recursive
    name: Recursive item
`

	data := []byte(str)
	var config GraphConfig
	if err := config.Parse(data); err != nil {
		log.Printf("Config: Error: ParseConfigFromFile: Parse: %v", err)
		return
	}

	log.Printf("Success!")
	log.Printf("%+v", &config)
	log.Printf("%+v", config.Resources.Noop[0].Content)
	if x := config.Resources.Noop[0].Content; x != nil {
		log.Printf("inside: %+v", *config.Resources.Noop[0].Content)
	}

	log.Printf("================")

	log.Printf("%+v", config.Resources.Noop[1].Content)
	if x := config.Resources.Noop[1].Content; x != nil {
		log.Printf("inside: %+v", *x)
	} else {
		log.Printf("nothing inside!")
	}

	if config.Resources.Noop[0].Content != config.Resources.Noop[1].Content {
		log.Printf("different pointers! :( %v %v", config.Resources.Noop[0].Content, config.Resources.Noop[1].Content)
	} else {
		log.Printf("same pointers! :)")
	}

	log.Printf("================")

	log.Printf("%+v", config.Resources.Noop[2].Content)
	if x := config.Resources.Noop[2].Content; x != nil {
		log.Printf("inside: %+v", *x)
	} else {
		log.Printf("nothing inside!")
	}

	if config.Resources.Noop[1].Content != config.Resources.Noop[2].Content {
		log.Printf("different pointers! :( %v %v", config.Resources.Noop[1].Content, config.Resources.Noop[2].Content)
	} else {
		log.Printf("same pointers! :)")
	}
	log.Printf("Recursive.This.Self.This.Name: %s", config.Resources.Recursive.This.Self.This.Name)
	config.Resources.Recursive.This.Self.This.Name = "Modified Name"
	log.Printf("Recursive.This.Self.This.Name: %s", config.Resources.Recursive.This.Self.This.Self.This.Name)
}

$ go run /tmp/test.go
2016/11/02 17:00:06 Hello!
2016/11/02 17:00:06 Success!
2016/11/02 17:00:06 &{Graph:mygraph Resources:{Noop:[0xc42000caa0 0xc42000cb80 0xc42000cc40] Recursive:{Self:0xc42000ce00 This:0xc42000ce00 Name:Recursive item}} Comment:my comment}
2016/11/02 17:00:06 0xc42000a8e0
2016/11/02 17:00:06 inside: hello
2016/11/02 17:00:06 ================
2016/11/02 17:00:06 0xc42000a8e0
2016/11/02 17:00:06 inside: hello
2016/11/02 17:00:06 same pointers! :)
2016/11/02 17:00:06 ================
2016/11/02 17:00:06 0xc42000a8e0
2016/11/02 17:00:06 inside: hello
2016/11/02 17:00:06 same pointers! :)
2016/11/02 17:00:06 Recursive.This.Self.This.Name: Recursive item
2016/11/02 17:00:06 Recursive.This.Self.This.Name: Modified Name

[purpleidea]

@vinzenz This is awesome work, thanks! +1

[joshwget]

Nice work! We had to implement a quick and dirty patch similar to this to avoid the massive memory consumption that came from copied references. Much better to have upstream fixed.

[lox]

This PR has a ton of changes in it. Would be great to actually get some of these in, but not as a whole bundle. @vinzenz is it possible to rebase this and scope it down to just a fix for #215 so we can evaluate it?

[purpleidea]

Oh cool, I forgot about this!

@vinzenz While it's still cool, now that we have the language in https://github.com/purpleidea/mgmt/ I actually don't know how important this is anymore.

[niemeyer]

We need something along those lines, but this must wait for v3 as we need to take into account the new public visibility of intermediate state there. Closing this for the time being since it seems abandoned (our fault as we took too long).

[purpleidea]

our fault as we took too long

You can still fixup patch and merge it :)

[niemeyer]

@purpleidea Please give me a hand. It took me a long time to go over this process and find time to work on v3 and fix all these longstanding issues which you're comfortably observing in your inbox. Please help me making sure we make v3 a success instead of pledging for even more work.

[niemeyer]

For those interested on the underlying issue, please move over to #215. It remains open and will track the work once it happens on v3.

[purpleidea]

@niemeyer I didn't know there was a v3 planned. Are there some design docs or an announce somewhere I can read about it and what needs doing?

[niemeyer]

There shouldn't be any surprises for those following the project. Most of the changes have been previously discussed here in the repository, and many of them are things that we could not do without breaking compatibility promises. The major improvement is exposing an intermediate representation.