Support ambiguous types with the Any protobuffer.

[ChrisCates]

Is your feature request related to a problem? Please describe.

Currently, the Any datatype is not supported by Node.js... This can be pretty troublesome, due to the nature of the language. Inheriting and transmitting legacy data is essentially impossible with GRPC in Node.js.

Describe the solution you'd like

It would be nice for Node.js to support the Any datatype, and for it to be able to pack and unpack data with ease.

Additional context

I'm available to review and update the codebase with support for this annotation.
I actually noticed that default definition imports don't work...

Example:

import "google/api/annotations.proto";

Will throw an error with the native .load() function.
So it would be nice to fix this up... And also support annotations like I've mentioned without the need for the module @grpc/proto-loader.

[ChrisCates]

Yo, my dudes! Just wondering what's going on!
It's been a minute... I can submit something for review if you guys have higher priority tickets... (This is a higher priority for me right now).

@nicolasnoble

[murgatroid99]

I'm sorry for the late response. I didn't see this issue when it was first filed. There are a few things to unpack here. First, Any is not an annotation, it is a type that can be used for representing arbitrary message types. Second, this is functionality of Protobuf.js, which gRPC uses to handle Protocol Buffers. The load() function uses an older version of Protobuf.js which does not support loading google/protobuf/*.proto files. This is why we are replacing it with the @grpc/proto-loader module, which depends on the newest version of Protobuf.js. According to protobufjs/protobuf.js#677, that newest version should correctly wrap and unwrap Any values, as long as you have the corresponding protobuf type loaded.

[ChrisCates]

Hey! Thanks for responding @murgatroid99.

1. Thanks for correcting that... That's actually a pretty bad error on my part, updated the original comment to call it that... I'm pretty new to this project, but I've reviewed the methodologies of it and considered it a way better use case than normal TCP/UDP.

2. Wow... So, this issue is already resolved then. It just wasn't verbosely specified! Thanks for clearing this up. Perhaps we should update documentation. However, I used @grpc/proto-loader and was not able to get the use of the Any type to compile correctly. I'll provide you with a reproducible test case to work with tomorrow!

Also, I think it would be in the best interests of the project to deprecate the .load() altogether, since the @grpc/proto-loader is a better implementation of a function with the same intentions. Just some things to be mentioned in the documentation. I think it would be especially important in the README.md file, since that's where most people will be looking first when trying to implement this.

Thanks Murg!

[majelbstoat]

I can't figure out if this is fixed and available in grpc 1.12.4? It might be that I don't understand how the Any type works, but I haven't been able to find any examples of how it's supposed to be set. If I have an arbitrary JSON object, which I want to send as a google.protobuf.Any, is there documentation for how I might achieve that? I got as far as data = new Any() 😂

[murgatroid99]

First, it's kind of confusing but google.protobuf.Any isn't actually intended to be used to send arbitrary JSON. It is used to send any existing protobuf type, which is why it includes a URL for retrieving the protobuf definition. For sending arbitrary JSON structures, you may want to check out the google.protobuf.Struct type.

[majelbstoat]

Thanks, that was helpful :) For other people that get here, you can use goog.protobuf.Struct to serialise and deserialise json like this:

let data = {
  foo: "bar"
}
const struct = Struct.fromJavaScript(data)
data = struct.toJavaScript()

[ChrisCates]

That was the money answer Jamie. +10, can we have additional documentation? Do you need help documenting and writing example?

On Mon, Jun 25, 2018 at 4:55 PM Jamie Talbot ***@***.***> wrote: Thanks, that was helpful :) For other people that get here, you can use goog.protobuf.Struct to serialise and deserialise json like this: let data = { foo: "bar" }const struct = Struct.fromJavaScript(data) data = struct.toJavaScript() — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#375 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABfSA3g-TEzWeElex5_72wMMmR_Vc_JCks5uAU4lgaJpZM4UOt01> .

-- Chris Cates, Software Developer

[nicolasnoble]

Documentation pull requests should always be welcome, as long as the CLA is signed, yes.

Also the website's source is over there: https://github.com/grpc/grpc.github.io

[ChrisCates]

Sounds good! @nicolasnoble, let's continue discussion when I make a Pull Request.