Integrate the TensorFlow inference library into your PhoneGap/Cordova application!
var tf = new TensorFlow('inception-v1');
var imgData = "/9j/4AAQSkZJRgABAQEAYABgAAD//gBGRm ...";
tf.classify(imgData).then(function(results) {
results.forEach(function(result) {
console.log(result.title + " " + result.confidence);
});
});
/* Output:
military uniform 0.647296
suit 0.0477196
academic gown 0.0232411
*/
cordova plugin add https://github.com/heigeo/cordova-plugin-tensorflow
<!-- config.xml -->
<plugin spec="https://github.com/heigeo/cordova-plugin-tensorflow.git" />
- Android
- iOS
The plugin provides a TensorFlow
class that can be used to initialize graphs and run the inference algorithm.
// Use the Inception model (will be downloaded on first use)
var tf = new TensorFlow('inception-v1');
// Use a custom retrained model
var tf = new TensorFlow('custom-model', {
'label': 'My Custom Model',
'model_path': "https://example.com/graphs/custom-model-2017.zip#rounded_graph.pb",
'label_path': "https://example.com/graphs/custom-model-2017.zip#retrained_labels.txt",
'input_size': 299,
'image_mean': 128,
'image_std': 128,
'input_name': 'Mul',
'output_name': 'final_result'
})
To use a custom model, follow the steps to retrain the model and optimize it for mobile use. Put the .pb and .txt files in a HTTP-accessible zip file, which will be downloaded via the FileTransfer plugin. If you use the generic Inception model it will be downloaded from the TensorFlow website on first use.
Each method returns a Promise
(if available) and also accepts a callback and errorCallback.
Classifies an image with TensorFlow's inference algorithm and the registered model. Will automatically download and initialize the model if necessary, but it is recommended to call load()
explicitly for the best user experience.
Note that the image must be provided as base64 encoded JPEG or PNG data. Support for file paths may be added in a future release.
var tf = new TensorFlow(...);
var imgData = "/9j/4AAQSkZJRgABAQEAYABgAAD//gBGRm ...";
tf.classify(imgData).then(function(results) {
results.forEach(function(result) {
console.log(result.title + " " + result.confidence);
});
});
Downloads the referenced model files and loads the graph into TensorFlow.
var tf = new TensorFlow(...);
tf.load().then(function() {
console.log("Model loaded");
});
Downloading the model files can take some time. If you would like to provide a progress indicator, you can do that with an onprogress
event:
var tf = new TensorFlow(...);
tf.onprogress = function(evt) {
if (evt['status'] == 'downloading')
console.log("Downloading model files...");
console.log(evt.label);
if (evt.detail) {
// evt.detail is from the FileTransfer API
var $elem = $('progress');
$elem.attr('max', evt.detail.total);
$elem.attr('value', evt.detail.loaded);
}
} else if (evt['status'] == 'unzipping') {
console.log("Extracting contents...");
} else if (evt['status'] == 'initializing') {
console.log("Initializing TensorFlow");
}
};
tf.load().then(...);
Checks whether the requisite model files have already been downloaded. This is useful if you want to provide an interface for downloading and managing TensorFlow graphs that is separate from the classification interface.
var tf = new TensorFlow(...);
tf.checkCached().then(function(isCached) {
if (isCached) {
$('button#download').hide();
}
});
This plugin is made possible by the following libraries and tutorials:
Source | Files |
---|---|
TensorFlow Android Inference Interface | libtensorflow_inference.so, libandroid_tensorflow_inference_java.jar |
TensorFlow Android Demo | Classifer.java, TensorFlowImageClassifier.java (modified) |
TensorflowPod | Referenced via podspec |
TensorFlow iOS Examples | ios_image_load.mm (modified), tensorflow_utils.mm (+ RunModelViewController.mm) |