proto: MarshalText fails and prints to stderr

[dvyukov]

The following program fails:

package main

import (
    "bytes"
    "fmt"
    "reflect"

    pb "github.com/dvyukov/go-fuzz/examples/protobuf/pb"
    "github.com/golang/protobuf/proto"
)

func main() {
    data := []byte("\xf5\a\xb3\xc4\xe1\xbf")
    v := new(pb.M18)
    err := proto.Unmarshal(data, v)
    if err != nil {
        panic(err)
    }
    var buf bytes.Buffer
    err = proto.MarshalText(&buf, v)
    if err != nil {
        fmt.Printf("failed to MarshalText: %#v\n", v)
        panic(err)
    }
    v2 := new(pb.M18)
    err = proto.UnmarshalText(buf.String(), v2)
    if err != nil {
        fmt.Printf("failed to UnmarshalText: %q\n", buf.Bytes())
        panic(err)
    }
    if !reflect.DeepEqual(v, v2) {
        fmt.Printf("v0: %#v\n", v)
        fmt.Printf("v2: %#v\n", v2)
        panic(fmt.Sprintf("non idempotent text marshal of %T", v))
    }
}

proto: failed getting extension:  unexpected EOF
v0: &example.M18{F0:(*string)(nil), XXX_extensions:map[int32]proto.Extension{126:proto.Extension{desc:(*proto.ExtensionDesc)(nil), value:interface {}(nil), enc:[]uint8{0xf5, 0x7, 0xb3, 0xc4, 0xe1, 0xbf}}}, XXX_unrecognized:[]uint8(nil)}
v2: &example.M18{F0:(*string)(nil), XXX_extensions:map[int32]proto.Extension(nil), XXX_unrecognized:[]uint8(nil)}
panic: non idempotent text marshal of *example.M18

There are 2 issues here:

1. Part of the message gets lost.
2. MarshalText prints to stderr. It should not. Either fail of be silent.

Definition of the protobuf is:

message M18 {
  optional string f0 = 1;
  extensions 100 to 199;
}

extend M18 {
  optional int32 f1 = 126;
}

on commit 34a5f24

[dsymonds]

reflect.DeepEqual doesn't work with protos. You need to use proto.Equal.

I'll make the extension failure return an error instead of printing.

I don't know why you're getting an EOF there. That's weird.

[dvyukov]

reflect.DeepEqual doesn't work with protos. You need to use proto.Equal.

But still non-empty extension can't be equal to nil.

I don't know why you're getting an EOF there. That's weird.

You can run the test locally to debug.

[dsymonds]

aece6fb updates the text formatter to return an error instead of printing.

[dsnet]

Closing as duplicate of #667, which is to investigate all usages of log.Print

[dvyukov]

#667 does not cover the main part of this "Part of the message gets lost".

[dsnet]

"Part of the message gets lost".

Are you referring to the fact that round-trip Marshal/Unmarshal produces different results? The proto text format does not have the property of full round-trip stability in the presence of unknown fields. This is not specific Go.