Filter/Select by non-key values

[wisotzky]

Based on latest path spec, following search options are possible:

/interfaces/interface/state/counters
/interfaces/interface[name=*]/state/counters

To select only specific interfaces, one could provide a specific key value:

/interfaces/interface[name=Ethernet1/2/3]/state/counters

However, there are use-cases for gNMI clients to narrow down the search on objects that have certain properties (non-key). For example one would like stream counters of interfaces that are administrative/operational up:

/interfaces/interface/state[admin-status=UP]/counters

Is there any plan to extend the gNMI PATH SPEC to allow selecting paths based on non-keys like this?

[wisotzky]

Rational:
While the gNMI streaming telemetry client could read the current config/state to determine the list of objects and finally subscribe just the determined objects, such solution comes with quite some overhead and increases the number of paths subscribed, which may become a scaling factor of the vendors gNMI server implementation. Also there is the challenge, that the list of objects that match the selection criteria and so the subscription may need to change as well.
Alternatively one could simply subscribe the entire table and apply client-side filtering aka squelching updates that are out of interest. However, this may come with quite some processing and transport overhead for both server and client.

[aashaikh]

There is no plan currently to add these sorts of queries to the gNMI spec -- we've generally tried to bias toward keeping the query language simple since adding features adds more complexity to the device implementation. As you mentioned, this sort of filtering is better done on the client side IMO, where there is likely to be more computational resource and flexibility. Agree there is a trade-off with potentially sending telemetry data that is ultimately not wanted.

[wisotzky]

Thanks @aashaikh,

Actually the advantage of my proposal would be, that the 'gNMI language' would not need to be changed at all. It would just relax the requirement that instead of list-keys other attributes can be used to select objects to be monitored. To me at least this would be the much cleaner approach to avoid updating subscriptions.

[wisotzky]

Maybe worth asking, @aashaikh

Is it really required that filter by key/value pairs is limited to list-keys only? At least in the latest PATH spec I did not found this explicitly mentioned. However, all the examples that are provided in both the gNMI spec and the PATH spec are using list-keys only.

Thx

[gcsl]

The case you mention, filtering by admin-status=UP sounds straightforward but in fact is not within the existing framework. The keys used in queries are part of the schema and exist at the level in the tree where they are queried. Your example is pulling from an arbitrary leaf somewhere in the subtree. Further, this is only one value type that can be carried in gNMI. Integer, string, and floating point value matching would be even less straightforward in addition to the problem of knowing where to look for the leaf of interest. We don't support regex or any sort of partial match on keys for server performance reasons. Expanding to recursive tree walks would suffer similar performance implications. The case would be even worse if queries were expected to cross subtree boundaries, e.g. "show the interfaces data where the optical power is below some threshold" (query in interfaces model referencing components model leaf). gNMI is intended to be a low-level transport for efficient, real-time delivery of device config and state and not a full-featured query language. Regarding concerns about performance for getting data that is not of interest. In this particular case, the path we are pursuing is event-driven implementations with duplicate suppression. Effectively, the down interfaces would be expected to have no changes and thus, beyond the first sync should be silent with regard to the gnmi transport. The open source gnmi_collector has this built-in by default. Since it takes gnmi in and supplies gnmi out, a client that wants to view a quieter stream can be written against such a collector that suppresses the unchanged data.

…

On Fri, Dec 4, 2020 at 5:31 AM wiso ***@***.***> wrote: Maybe worth asking, @aashaikh <https://github.com/aashaikh> Is it really required that filter by key/value pairs is limited to list-keys only? At least in the latest PATH spec I did not found this explicitly mentioned. However, all the examples that are provided in both the gNMI spec and the PATH spec are using list-keys only. Thx — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#130 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEL4QL4K7XWLHYRO2LV5UADSTC3ARANCNFSM4UNFRNYQ> .