Tensorflow jni: Expose TF_Graph->graph and TF_Operation->node?

[wumo]

I'm trying to reimplement some Tensorflow Python functions in Java. I found that some functions are provided by Tensorflow C API but not by C++ API (e.g. TF_GraphGetTensorShape, you cannot find an equivalent C++ class/function that does the same thing), and some functions are provided by C++ API but not by C API (e.g. Graph::UpdateEdge, you cannot find an equivalent C function which does the same thing).

Under the hood, C or C++ API are calling each other's functions through some conversion functions (e.g. TF_AddGradients uses C++'s AddSymbolicGradients which requires converting TF_Output to class Output and vice versa). We can do some C/C++ conversion using only current JNI api. But now I cannot implement UpdateEdge when using C API (because we only have TF_Graph, the field graph is not accessible by JNI API) . So if we can get the inner TF_Graph->graph field, then we can use C++'s Graph::UpdateEdge function to update the graph edge which is constructed by C API.

Will you consider exposing some other struct fields like this?

[saudet]

Yes, we can, we just need to include c_api_internal.h. Good idea, I'll do that right away!

[saudet]

/cc @asimshankar @karllessard

[asimshankar]

Thanks for the notification @saudet .

For the supported, stable API surface for TensorFlow in C (defined in tensorflow/c/c_api.h) we intentionally do not provide node mutation methods like UpdateEdge since they make it possible to create invalid graphs and/or break invariants assumed by other code. We've been in the process of cleaning up the Python implementation as well (which does currently do some mutations that we're not thrilled about).

@wumo : If you have a need for graph mutation, we'd like to better understand that and figure out an alternative to graph mutation.

[wumo]

@asimshankar I was to reimplement the class CondContext from tensorflow python api, and found that inside _AddOpInternal, the updateEdge function is required after finishing building operation. The graph mutation happened there maybe simplify the control flow implementation. Maybe graph mutation can be avoided if I restrict the usage for now.

[saudet]

@wumo I've released version 1.4.2 with bindings for c_api_internal.h, so feel free to use them! If you come up with something interesting that you would like to contribute upstream, please contact me, @asimshankar, or others from Google, as they are not currently convinced that JavaCPP is useful and are not interested in having it TensorFlow at this moment. However, having ready-made functionality to integrate might change their minds (but as you can see, they are very picky about what goes in, so be sure to sync with @asimshankar frequently). If you have any additional requests, please let me know! BTW, you might be interested in using the new PointerScope feature for easier memory management:
http://bytedeco.org/news/2018/07/17/bytedeco-as-distribution/