tutorials/conversion_onnx/keras2onnx.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img width=\"150\" alt=\"Logo_ER10\" src=\"https://user-images.githubusercontent.com/3244249/151994514-b584b984-a148-4ade-80ee-0f88b0aefa45.png\">\n",
    "\n",
    "## Keras to ONNX conversion\n",
    "This notebook shows how to convert your trained Keras model to ONNX, the generic format supported by DIANNA. <br>\n",
    "\n",
    "The conversion is complete with the tf2onnx Python package, which supports both the SavedModel format and the older HDF5 (.h5 or .keras) format. It can convert multi-backend keras as well as tf.keras models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "\n",
    "import onnx\n",
    "import onnxruntime as ort\n",
    "# In addition to these imports, this notebook\n",
    "# depends on tf2onnx. It is used from the command line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download and initialize built-in model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.applications.mobilenet.MobileNet(weights='imagenet')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate model on some random input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_shape = [1] + model.inputs[0].shape[1:]  # input shape without a 1 for batch size, instead of None\n",
    "input_data = np.random.normal(size=input_shape).astype(np.float32)\n",
    "pred = model.predict(input_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save keras model to SavedModel format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:FOR KERAS USERS: The object that you are saving contains one or more Keras models or layers. If you are loading the SavedModel with `tf.keras.models.load_model`, continue reading (otherwise, you may ignore the following instructions). Please change your code to save with `tf.keras.models.save_model` or `model.save`, and confirm that the file \"keras.metadata\" exists in the export directory. In the future, Keras will only load the SavedModels that have this file. In other words, `tf.saved_model.save` will no longer write SavedModels that can be recovered as Keras models (this will apply in TF 2.5).\n",
      "\n",
      "FOR DEVS: If you are overwriting _tracking_metadata in your class, this property has been used to save metadata in the SavedModel. The metadta field will be deprecated soon, so please move the metadata to a different file.\n",
      "INFO:tensorflow:Assets written to: mysavedmodel\\assets\n"
     ]
    }
   ],
   "source": [
    "savedmodel_dir = 'mysavedmodel'\n",
    "tf.saved_model.save(model, savedmodel_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert to ONNX."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-10 16:21:35.777938: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found\n",
      "2022-03-10 16:21:35.777995: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n",
      "C:\\Users\\ChristiaanMeijer\\anaconda3\\envs\\temp-dianna\\lib\\runpy.py:127: RuntimeWarning: 'tf2onnx.convert' found in sys.modules after import of package 'tf2onnx', but prior to execution of 'tf2onnx.convert'; this may result in unpredictable behaviour\n",
      "  warn(RuntimeWarning(msg))\n",
      "2022-03-10 16:21:39.793252: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found\n",
      "2022-03-10 16:21:39.793284: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)\n",
      "2022-03-10 16:21:39.798447: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: ESLT0114\n",
      "2022-03-10 16:21:39.798693: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: ESLT0114\n",
      "2022-03-10 16:21:39.799632: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2022-03-10 16:21:43,849 - INFO - Signatures found in model: [serving_default].\n",
      "2022-03-10 16:21:43,850 - INFO - Output names: ['predictions']\n",
      "2022-03-10 16:21:43.871683: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0\n",
      "2022-03-10 16:21:43.871942: I tensorflow/core/grappler/clusters/single_machine.cc:357] Starting new session\n",
      "2022-03-10 16:21:43.925808: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:1144] Optimization results for grappler item: graph_to_optimize\n",
      "  function_optimizer: Graph size after: 556 nodes (416), 805 edges (665), time = 30.764ms.\n",
      "  function_optimizer: function_optimizer did nothing. time = 0.402ms.\n",
      "\n",
      "WARNING:tensorflow:From C:\\Users\\ChristiaanMeijer\\anaconda3\\envs\\temp-dianna\\lib\\site-packages\\tf2onnx\\tf_loader.py:706: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use `tf.compat.v1.graph_util.extract_sub_graph`\n",
      "2022-03-10 16:21:45,102 - WARNING - From C:\\Users\\ChristiaanMeijer\\anaconda3\\envs\\temp-dianna\\lib\\site-packages\\tf2onnx\\tf_loader.py:706: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use `tf.compat.v1.graph_util.extract_sub_graph`\n",
      "2022-03-10 16:21:45.160029: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0\n",
      "2022-03-10 16:21:45.160280: I tensorflow/core/grappler/clusters/single_machine.cc:357] Starting new session\n",
      "2022-03-10 16:21:45.407735: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:1144] Optimization results for grappler item: graph_to_optimize\n",
      "  constant_folding: Graph size after: 256 nodes (-274), 531 edges (-274), time = 155.954ms.\n",
      "  function_optimizer: function_optimizer did nothing. time = 2.671ms.\n",
      "  constant_folding: Graph size after: 256 nodes (0), 531 edges (0), time = 32.974ms.\n",
      "  function_optimizer: function_optimizer did nothing. time = 2.043ms.\n",
      "\n",
      "2022-03-10 16:21:45,622 - INFO - Using tensorflow=2.5.0, onnx=1.11.0, tf2onnx=1.9.3/1190aa\n",
      "2022-03-10 16:21:45,623 - INFO - Using opset <onnx, 9>\n",
      "2022-03-10 16:21:46,897 - INFO - Computed 0 values for constant folding\n",
      "2022-03-10 16:21:48,086 - INFO - Optimizing ONNX model\n",
      "2022-03-10 16:21:50,126 - INFO - After optimization: BatchNormalization -27 (27->0), Cast -4 (4->0), Concat -2 (2->0), Const -107 (165->58), GlobalAveragePool +1 (0->1), Identity -6 (6->0), ReduceMean -1 (1->0), Reshape -13 (15->2), Shape -2 (2->0), Slice -2 (2->0), Squeeze -1 (2->1), Transpose -122 (123->1), Unsqueeze -6 (6->0)\n",
      "2022-03-10 16:21:50,172 - INFO - \n",
      "2022-03-10 16:21:50,172 - INFO - Successfully converted TensorFlow model mysavedmodel to ONNX\n",
      "2022-03-10 16:21:50,172 - INFO - Model inputs: ['input_1']\n",
      "2022-03-10 16:21:50,172 - INFO - Model outputs: ['predictions']\n",
      "2022-03-10 16:21:50,172 - INFO - ONNX model is saved at mysavedmodel.onnx\n"
     ]
    }
   ],
   "source": [
    "onnx_savedmodel = 'mysavedmodel.onnx'\n",
    "!python -m tf2onnx.convert --saved-model {savedmodel_dir} --output {onnx_savedmodel} --signature_def serving_default --tag serve"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate ONNX models and compare to keras model output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "# verify the ONNX model is valid\n",
    "onnx_model = onnx.load(onnx_savedmodel)\n",
    "onnx.checker.check_model(onnx_model)\n",
    "\n",
    "# get ONNX predictions\n",
    "sess = ort.InferenceSession(onnx_savedmodel)\n",
    "input_name = sess.get_inputs()[0].name\n",
    "output_name = sess.get_outputs()[0].name\n",
    "\n",
    "onnx_input = {input_name: input_data}\n",
    "pred_onnx = sess.run([output_name], onnx_input)[0]\n",
    "\n",
    "print(np.allclose(pred_onnx, pred, atol=1e-5))"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "e7604e8ec5f09e490e10161e37a4725039efd3ab703d81b1b8a1e00d6741866c"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}