main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZsTnVufoNjkM"
   },
   "source": [
    "# Lab (extra) | Corner detection and ORB\n",
    "\n",
    "At the end of this laboratory, you would get familiarized with\n",
    "\n",
    "*   Corner Detection using FAST and Harris Corners\n",
    "*   ORB feature detectors and descriptors\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 32000,
     "status": "ok",
     "timestamp": 1666086572494,
     "user": {
      "displayName": "Bhalaji Nagarajan",
      "userId": "08128536337584225726"
     },
     "user_tz": -120
    },
    "id": "jJIC9t_DV7js",
    "outputId": "c47fb2ab-77f9-49cd-8ccd-6edd7ed15440"
   },
   "outputs": [],
   "source": [
    "# mount drive to the notebook\n",
    "\n",
    "#from google.colab import drive\n",
    "#drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 1374,
     "status": "ok",
     "timestamp": 1666086581144,
     "user": {
      "displayName": "Bhalaji Nagarajan",
      "userId": "08128536337584225726"
     },
     "user_tz": -120
    },
    "id": "k1T3jZ0kI-XX"
   },
   "outputs": [],
   "source": [
    "# Loading necessary libraries (Feel free to add new libraries if you need for any computation)\n",
    "\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "import os\n",
    "from skimage import color, data, feature, filters, io, transform "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ir6WNbB4L05G"
   },
   "source": [
    "# Corner Detection\n",
    "\n",
    "### Exercise: #1\n",
    "\n",
    "*    Detect the corners in the image starbucks4.jpg. Use corner_fast to extract FAST corners for the given image and corner_peaks to find corners in the corner measure response image.\n",
    "*   Show the original image, fast corners, peaks overlayed on the original image using matplotlib subplot.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "iun4uPF7MVIj"
   },
   "outputs": [],
   "source": [
    "# solution\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qlR3AhnMMV9n"
   },
   "source": [
    "*   Analyze and discuss the effect of changing **'threshold'** in the corner_fast function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vbtRxCp73vDH"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XfOF0CeJNMMD"
   },
   "source": [
    "*   Repeat the exercise by replacing FAST corners with Harris corners."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "fJcx3ZZFNT48"
   },
   "outputs": [],
   "source": [
    "# solution\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KSqjxrhINVur"
   },
   "source": [
    "*   Do you find difference between both the techniques? If so, why and what difference do you observe?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o8nfKNaANh7R"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EjTXqXXX4_Lj"
   },
   "source": [
    "# ORB feature detector and binary descriptor\n",
    "\n",
    "### Exercise #0: Compute ORB descriptors and find descriptors match\n",
    "\n",
    "*   Load 'astronaut' image from data module.\n",
    "*   Convert the image to grayscale.\n",
    "*   Create a copy of the image and rotate it by 180&deg; \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 267,
     "status": "ok",
     "timestamp": 1666086662537,
     "user": {
      "displayName": "Bhalaji Nagarajan",
      "userId": "08128536337584225726"
     },
     "user_tz": -120
    },
    "id": "uh61dfwg_NgJ"
   },
   "outputs": [],
   "source": [
    "img1 = color.rgb2gray(data.astronaut())\n",
    "img2 = transform.rotate(img1, 180)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H0jcF2EbDRnp"
   },
   "source": [
    "*   Create an ORB feature detector with default parameters. *Hint: ORB feature detector is available in skimage.feature module.*\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 269,
     "status": "ok",
     "timestamp": 1666086668739,
     "user": {
      "displayName": "Bhalaji Nagarajan",
      "userId": "08128536337584225726"
     },
     "user_tz": -120
    },
    "id": "hI3dVPnj_xj_"
   },
   "outputs": [],
   "source": [
    "descriptor_extractor = feature.ORB(n_keypoints=200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CvU6NlYCDcdY"
   },
   "source": [
    "*   Extract the keypoints and descriptors using detect_and_extract function for both the images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 2778,
     "status": "ok",
     "timestamp": 1666086675629,
     "user": {
      "displayName": "Bhalaji Nagarajan",
      "userId": "08128536337584225726"
     },
     "user_tz": -120
    },
    "id": "NlCJtotkDi7j"
   },
   "outputs": [],
   "source": [
    "descriptor_extractor.detect_and_extract(img1)\n",
    "keypoints1 = descriptor_extractor.keypoints\n",
    "descriptors1 = descriptor_extractor.descriptors\n",
    "\n",
    "descriptor_extractor.detect_and_extract(img2)\n",
    "keypoints2 = descriptor_extractor.keypoints\n",
    "descriptors2 = descriptor_extractor.descriptors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kRR6cdGEEFJJ"
   },
   "source": [
    "*   Descriptors can be matched using match_descriptors function available in skimage.feature module\n",
    "*   The matches can be plotted using the plot_matches function available in skimage.feature module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 214
    },
    "executionInfo": {
     "elapsed": 1290,
     "status": "ok",
     "timestamp": 1666086685641,
     "user": {
      "displayName": "Bhalaji Nagarajan",
      "userId": "08128536337584225726"
     },
     "user_tz": -120
    },
    "id": "4O2sTS4h73jz",
    "outputId": "dad9b923-f12b-4bc1-9109-cedd9bc1c49b"
   },
   "outputs": [],
   "source": [
    "matches12 = feature.match_descriptors(descriptors1, descriptors2, cross_check=True)\n",
    "feature.plot_matches(plt, img1, img2, keypoints1, keypoints2, matches12)\n",
    "plt.axis('off')\n",
    "plt.title(\"Original Image vs. Transformed Image\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tSaDWEpfFrge"
   },
   "source": [
    "### Exercise: #1\n",
    "\n",
    "*   Analyze and discuss the effect of changing the parameter **'max_ratio'** in the match_descriptors function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WqTGqWE-F0Z7"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uJQV1TIeIgGf"
   },
   "source": [
    "*   What do you infer from **'harris_k'**, **'n_scales'**, **'fast_n'**, **'fast_threshold'** parameters in ORB function?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n1qUG2FuJCUi"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qewFLt7sEs93"
   },
   "source": [
    "### Exercise: #2\n",
    "*   Using the above example, create a function get_ORB, that takes two images as parameters and returns the keypoints of both images and descriptor matches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "g3H6M75aEsaC"
   },
   "outputs": [],
   "source": [
    "# solution\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ee3MqFlgFfUP"
   },
   "source": [
    "*   Using the above function, detect the correspondences between the model image 'starbucks.jpg' with the scene image 'starbucks4.jpg'\n",
    "*   Plot the matches between the two images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "OfuTiwfxFhRf"
   },
   "outputs": [],
   "source": [
    "# solution\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "C7iNt-h8GIub"
   },
   "source": [
    "### Exercise: #3\n",
    "\n",
    "*   Using \"starbucks.jpg\" image as a model, show its matches to all Starbucks images. \n",
    "*   Show the number of the matches for the pair as title for each plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Gpz290LqGcDk"
   },
   "outputs": [],
   "source": [
    "# solution\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lXTgi_zeGeiS"
   },
   "source": [
    "*   Comment on the performance of the algorithm with respect to each image. When do you think the algorithm works better?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "79m3V4oiGpJM"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7cY0FdG0G-Oq"
   },
   "source": [
    "### Exercise: #4\n",
    "\n",
    "*   Repeate **Exercise #2**, by changing the orientation of the model image by rotating it and comparing it with its original version.\n",
    "*   Create 8 orientations of the model image by rotating the image by 45&deg; (0, 45, 90, 135, 180, 225, 270, 315). *Hint: You can use the rotate() function from skimage.transform.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "iAdLWbkGHysK"
   },
   "outputs": [],
   "source": [
    "# solution\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zpQtm37M2AD5"
   },
   "source": [
    "*   What do you observe with respect to the change in orientations?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WgMbkR_g18nP"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kwb8qZz3IGCo"
   },
   "source": [
    "### Exercise: #5\n",
    "\n",
    "*   Repeat **Exercise #3** using Coco-cola images using cocacola_logo.png as the model image.\n",
    "*   Does the same parameters used for Starbucks images work in this case? Or was it necessary to modify the ORB parameters?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "YEUq65_fIC3u"
   },
   "outputs": [],
   "source": [
    "# solution\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "83tMJ67a15G-"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "W4C5K2SzByRL"
   },
   "source": [
    "### Exercise #6. Analysis of the applied techniques and results\n",
    "\n",
    "*   What are the advantages of ORB object detection compared with HOG object detector?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ret9l0NHimS1"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bVjZhZOSCA4u"
   },
   "source": [
    "*   What would happen if you analyse an image that does not contain the Starbucks logo (as is the case of \"edificio.jpg\")?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "URIfq2NbCMRL"
   },
   "outputs": [],
   "source": [
    "# solution\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Pt5XSvpU12jP"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Gvg-n9QJCM9f"
   },
   "source": [
    "*   What are the different ways of defining a quality measure for the correspondance between two images? (implementation of measures is not necessary)?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L5M5MGDUCPqr"
   },
   "source": [
    "**Solution**\n",
    "\n",
    "*(Double-click or enter to edit)*\n",
    "\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "G2odhSVaJ44I"
   },
   "source": [
    "# Panorama Stitching (Not graded)\n",
    "\n",
    "Image Mosaicing is a technique of combining multiple overlapping images into a single image.\n",
    "\n",
    "*   A simple tutorial can be found in this notebook: \n",
    "*   https://github.com/scikit-image/skimage-tutorials/blob/main/lectures/solutions/adv3_panorama-stitching-solution.ipynb"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "authorship_tag": "ABX9TyMTGQbpRfjsMqZNw++050n/",
   "collapsed_sections": [],
   "mount_file_id": "1jKrOONLYPmArVSf5AKIJFtxMRIy7lQXb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}