Fix performance issues with graph edge iteration in ShaderGraph

[nadult]

In complex materials graph and shader graph edge iteration can be extremely slow, because some edges may be visited unnecessarily multiple times. This is especially noticable in two functions: ShaderGraph::addUpstreamDependencies and ShaderGraph::optimize() .

GraphIterator and ShaderGraphEdgeIterator classes iterate over DAGs without marking nodes as visited, which may lead to exponential traversal time for some DAGs: https://stackoverflow.com/a/69326676

This patch adds two functions which efficiently generate a unique list of (shader) graph edges and uses those lists instead of graph iterators.

For one especially complex material on which I tested this fix, shader code generation time decreased from 156.13 sec to 0.08 sec (almost 2000x speedup). Number of visited edges in ShaderGraph::optimize() decreased from 42M to ~600.

I'm not sure if the best approach is to add those two functions next to current graph iterators or maybe those iterators should be replaced altogether. Another solution which I see would be to fix the graph iterators, but IMO they add a lot of overhead and it's faster to simply use a function which returns a list of all edges in a given graph in std::vector.

[jstone-lucasfilm]

This is a really interesting proposal, @nadult, and my main concern is that we'd have two subtly different approaches for traversing graphs in MaterialX, each of which would need to be maintained, validated, and extended in the future.

Perhaps it would actually be better to upgrade the GraphIterator class (as in one of your suggestions above), storing the required visitation data directly in this class to avoid duplicate edges?

This would still allow the addition of a top-level uniqueGraphEdges convenience method, which would be defined as a simple wrapper around traverseGraph, and the traversal system would remain unified for future maintenance.

[nadult]

OK, makes sense. I will try to prepare a new solution this week with your suggestions included.

[kwokcb]

For this, I think it would be useful to have the traversal API have 2 extensions:

1. Still allow for incremental traversal using the existing iterator but allow the option to avoid duplicates. Guess that could be the default on a version bump?
2. Have the ability to get all unique edges via the uniqueGraphEdges which pre-traversses and returns all edges.

Item (1) I think is useful to preserve as an integration may iterate and want to perform some action during traversal, including extracting out additional information which may be hard to get afterwards since not all information during traversal is cached on an edge.

Of course I only looked at the changes briefly, so just my initial thoughts. Thoughts on this ? Thanks.