Skip to content

Commit

Permalink
update dog vision v2 notebook
Browse files Browse the repository at this point in the history
  • Loading branch information
@mrdbourke
mrdbourke committed Apr 12, 2024
1 parent 79e9de6 commit a22c402
Showing 1 changed file with 71 additions and 31 deletions.
102 changes: 71 additions & 31 deletions section-4-unstructured-data-projects/end-to-end-dog-vision-v2.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,15 @@
"\n",
"Welcome, welcome!\n",
"\n",
"The focus of this notebook is to give a quick overview of deep learning with TensorFlow.\n",
"The focus of this notebook is to give a quick overview of deep learning with TensorFlow/Keras.\n",
"\n",
"How?\n",
"\n",
"We're going to go through the machine learning workflow steps and build a computer vision project to classify photos of dogs into their respective dog breed.\n",
"We're going to go through the machine learning workflow steps and build a computer vision project to classify photos of dogs into their respective dog breed (a Predictive AI task, see below for more).\n",
"\n",
"TK - image of workflow - e.g. dog photo -> model -> dog breed"
"<img src=\"https://github.com/mrdbourke/zero-to-mastery-ml/blob/master/images/unstructured-data-dog-vision-project-outline.png?raw=true\" width=750 alt=\"outline of dog vision project from taking a dataset of dog images and then creating a neural network for identifying different dogs in custom images\"/>\n",
"\n",
"*What we're going to build: Dog Vision 🐶👁️, a neural network capable of identifying different dog breeds in images. All the way from dataset preparation to model building, training and evaluation.*"
]
},
{
Expand Down Expand Up @@ -67,7 +69,7 @@
"id": "nYCWwASwpPNl"
},
"source": [
"## TK - What we're going to cover\n",
"## What we're going to cover\n",
"\n",
"In this project, we're going to be introduced to the power of deep learning and more specifically, transfer learning using TensorFlow/Keras.\n",
"\n",
Expand Down Expand Up @@ -96,14 +98,15 @@
"id": "BLK0aCjmo8j6"
},
"source": [
"## TK - Where can can you get help?\n",
"## Where can can you get help?\n",
"\n",
"All of the materials for this course are [available on GitHub](https://github.com/mrdbourke/zero-to-mastery-ml/tree/master).\n",
"\n",
"If you run into trouble, you can ask a question on the course [GitHub Discussions page](https://github.com/mrdbourke/zero-to-mastery-ml/discussions) there too.\n",
"\n",
"* TK - Ask Stack Overflow\n",
"* TK - Ask ChatGPT/Gemini"
"You can also:\n",
"* Search for questions online and end up at places such as Stack Overflow (a great resource of developer-focused Q&A).\n",
"* Ask AI Assistants such as [ChatGPT](https://chat.openai.com/), [Gemini](https://gemini.google.com/) and [Claude](https://claude.ai/) for help with various coding problems and errors."
]
},
{
Expand All @@ -116,21 +119,19 @@
"\n",
"Let's start by breaking down some of the most important topics we're going to go through.\n",
"\n",
"### TK - What is TensorFlow?\n",
"### What is TensorFlow/Keras?\n",
"\n",
"[TensorFlow](https://www.tensorflow.org/) is an open source machine learning and deep learning framework originally developed by Google. Inside TensorFlow, you can also use Keras which is another very helpful machine learning framework known for its ease of use. \n",
"[TensorFlow](https://www.tensorflow.org/) is an open source machine learning and deep learning framework originally developed by Google. Inside TensorFlow, you can also use [Keras](https://keras.io/) which is another very helpful machine learning framework known for its ease of use. \n",
"\n",
"### TK - Why use TensorFlow?\n",
"### Why use TensorFlow?\n",
"\n",
"TensorFlow allows you to manipulate data and write deep learning algorithms using Python code.\n",
"\n",
"It also has several built-in capabilities to leverage accelerated computing hardware (e.g. GPUs, Graphics Processing Units and TPUs, Tensor Processing Units).\n",
"\n",
"Many of world's largest companies [power their machine learning workloads with TensorFlow](https://www.tensorflow.org/about/case-studies).\n",
"\n",
"TK - image of people using TensorFlow\n",
"\n",
"### TK - What is deep learning?\n",
"### What is deep learning?\n",
"\n",
"[Deep learning](https://en.wikipedia.org/wiki/Deep_learning) is a form of machine learning where data passes through a series of progressive layers which all contribute to learning an overall representation of that data.\n",
"\n",
Expand All @@ -144,37 +145,73 @@
"\n",
"So when someone says deep learning or (artificial neural networks), they're typically referring to same thing.\n",
"\n",
"### TK - What can deep learning be used for?\n",
"> **Note:** Artificial intelligence (AI), machine learning (ML) and deep learning are all broad terms. You can think of AI as the overall technology, machine learning as a type of AI, and deep learning as a type of machine learning. So if someone refers to AI, you can often assume they are often talking about machine learning or deep learning."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "h05v4NwY7EHj"
},
"source": [
"### What can deep learning be used for?\n",
"\n",
"Deep learning is such a powerful technique that new use cases are being discovered everyday.\n",
"\n",
"Most of the modern forms of artifical intelligence (AI) applications you see, are powered by deep learning.\n",
"\n",
"[ChatGPT](https://chat.openai.com) and other large language models (LLMs) such as Llama, Claude and Gemini use deep learning to process text and return a response.\n",
"Two of main types of AI are predictive and generative.\n",
"\n",
"Predictive AI learns the relationship between data and labels such as photos of dog and their breeds.\n",
"\n",
"Tesla's [self-driving cars use deep learning](https://www.tesla.com/AI) to power their computer vision systems.\n",
"Generative AI generates something new given an input such as creating new text given input text.\n",
"\n",
"Apple's Photos app uses deep learning to [recognize faces in images](https://machinelearning.apple.com/research/recognizing-people-photos) and create Photo Memories.\n",
"Some examples of Predictive AI problems include:\n",
"* Tesla's [self-driving cars use deep learning](https://www.tesla.com/AI) use object detection models to power their computer vision systems.\n",
"* Apple's Photos app uses deep learning to [recognize faces in images](https://machinelearning.apple.com/research/recognizing-people-photos) and create Photo Memories.\n",
"* Siri and Google Assistant use deep learning to transcribe speech and understand voice commands.\n",
"* [Nutrify](https://nutrify.app) (an app my brother and I build) uses predictive AI to recognize food in images.\n",
"* [Magika](https://google.github.io/magika/) uses deep learning to classify a file into what type it is (e.g. `.jpeg`, `.py`, `.txt`).\n",
"* [Text classification models](https://huggingface.co/MoritzLaurer/deberta-v3-large-zeroshot-v2.0) such as DeBERTa use deep learning to classify text into different categories such as \"positive\" and \"negative\" or \"spam\" or \"not spam\".\n",
"\n",
"Siri and Google Assistant use deep learning to recognize and understand voice commands.\n",
"Some examples of Generative AI problems include:\n",
"* [Stable Diffusion](https://huggingface.co/spaces/stabilityai/stable-diffusion) uses generative AI to generate images given a text prompt.\n",
"* [ChatGPT](https://chat.openai.com) and other large language models (LLMs) such as [Llama](https://llama.meta.com), [Claude](https://claude.ai), [Gemini](https://gemini.google.com) and [Mistral](https://mistral.ai) use deep learning to process text and return a response.\n",
"* [GitHub Copilot](https://github.com/features/copilot) uses generative AI to generate code snippets given surrounding context.\n",
"\n",
"[Nutrify](https://nutrify.app) (an app my brother and I build) uses deep learning to recognize food in images.\n",
"All of these AI use cases are powered by deep learning.\n",
"\n",
"TK - image of examples where deep learning can be used for\n",
"And more often than not, whenever you get started on a deep learning problem, you'll start with transfer learning. \n",
"\n",
"### TK - What is transfer learning?\n",
"<img src=\"https://github.com/mrdbourke/zero-to-mastery-ml/blob/master/images/unstructured-data-example-ai-problems-visualized.png?raw=true\" width=750 alt=\"\n",
"This image is a collage showcasing examples of different artificial intelligence applications. The top left shows an image classification example with a smartphone identifying a pineapple and suggesting recipes. Next to it is a representation of voice recognition with colorful circles symbolizing the interaction. In the middle top row is a text-to-text interface depicting a chat with an AI producing poetry. To the right is a text-to-image service with a request and generated images of a dog studying machine learning. The bottom left demonstrates text classification with an email being marked as not spam. Lastly, the bottom right depicts a self-driving car’s dashboard view, indicating the car's navigation and road awareness. All segments have titles and some feature source attributions like 'nutrify.app' and 'AI DRIVR YouTube'.\"/>\n",
"\n",
"*Example of different every day problems where AI/machine learning gets used.*"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "h05v4NwY7EHj"
},
"source": [
"### What is transfer learning?\n",
"\n",
"[Transfer learning](https://en.wikipedia.org/wiki/Transfer_learning) is one of the most powerful and useful techniques in modern AI and machine learning.\n",
"\n",
"It involves taking what one model (or neural network) has learned in a similar domain and applying to your own.\n",
"\n",
"In our case, we're going to use transfer learning to take the patterns a neural network has learned from the 1 million+ images and over 1000 classes in [ImageNet](https://www.image-net.org/) (a gold standard computer vision benchmark) and apply them to our own problem of recognizing dog breeds.\n",
"\n",
"However, this concept can be applied to many different domains. You could take a large language model (LLM) that has been pre-trained on *most* of the text on the internet and learned very well the patterns in naturual language and customize it for your own specific chat use-case.\n",
"However, this concept can be applied to many different domains. \n",
"\n",
"You could take a large language model (LLM) that has been pre-trained on *most* of the text on the internet and learned very well the patterns in naturual language and customize it for your own specific chat use-case.\n",
"\n",
"The biggest benefit of transfer learning is that it often allows you to get outstanding results with less data and time.\n",
"\n",
"TK - image Transfer learning workflow - Large data -> Large model -> Patterns -> Custom data -> Custom model"
"<img src=\"https://github.com/mrdbourke/zero-to-mastery-ml/blob/master/images/unstructured-data-a-transfer-learning-workflow.png?raw=true\" width=750 alt='This image is an infographic that explains the process of using pre-trained models from Hugging Face and Keras to apply to custom data. The top section asks, \"Does my problem exist on Hugging Face/Keras?\" indicating a starting point for a solution search. The left section shows the ImageNet database webpage, representing a large dataset, leading into the center where a network graph signifies a \"Pretrained Model.\" This model is then applied to custom data, illustrated by a diagram featuring various dog images being processed and resulting in checkmarks, implying successful identification or classification. At the bottom, it emphasizes that \"Keras applications make using a pre-trained model as simple as calling a Python class,\" with a specific code example given for EfficientNet V2. The overall message is that these tools can handle various data types, not just images, and simplify machine learning tasks.'/>\n",
"\n",
"*A transfer learning workflow. Many publicly available models have been pretrained on large datasets such as ImageNet (1 million+ images). These models can then be applied to similar tasks downstream. For example, we can take a model pretrained on ImageNet and apply it to our Dog Vision 🐶👁️ problem. This same process can be repeated for many different styles of data and problem.*"
]
},
{
Expand All @@ -183,9 +220,9 @@
"id": "Mr8xFrmA0qeG"
},
"source": [
"## TK - Getting setup\n",
"## Getting setup\n",
"\n",
"This section of the course is taught with [Google Colab](https://colab.research.google.com/), an online Jupyter Notebook that provides free access to GPUs (Graphics Processing Units, we'll hear more on these later).\n",
"This notebook is designed to run in [Google Colab](https://colab.research.google.com/), an online Jupyter Notebook that provides free access to GPUs (Graphics Processing Units, we'll hear more on these later).\n",
"\n",
"For a quick rundown on how to use Google Colab, see their [introductory guide](https://colab.research.google.com/notebooks/basic_features_overview.ipynb) (it's quite similar to a Jupyter Notebook with a few different options).\n",
"\n",
Expand All @@ -196,7 +233,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Getting a GPU\n",
"### Getting a GPU on Google Colab\n",
"\n",
"Before running any code, we'll make sure our Google Colab instance is connected to a GPU.\n",
"\n",
Expand Down Expand Up @@ -301,7 +338,7 @@
"id": "PDvfRmUqx0C5"
},
"source": [
"## TK - Getting Data\n",
"## Getting Data\n",
"\n",
"All machine learning (and deep learning) projects start with data.\n",
"\n",
Expand Down Expand Up @@ -362,7 +399,7 @@
"id": "1tHF13s8UbzK"
},
"source": [
"### TK - Download data directly from Stanford Dogs website\n",
"### Download data directly from Stanford Dogs website\n",
"\n",
"Our overall project goal is to build a computer vision model which performs better than the original Stanford Dogs paper (average of 22% accuracy per class across 120 classes).\n",
"\n",
Expand Down Expand Up @@ -409,7 +446,7 @@
"\n",
"Plus, this is all good practice for dealing with and manipulating data, a very important skill in the machine learning engineers toolbox.\n",
"\n",
"> **Note:** The following section is designed to run in Google Colab. If you are running locally, feel free to modify the code to save to a local directory instead of Google Drive."
"> **Note:** The following data download section is designed to run in Google Colab. If you are running locally, feel free to modify the code to save to a local directory instead of Google Drive."
]
},
{
Expand Down Expand Up @@ -4623,9 +4660,12 @@
"And the good news is, there are plenty of places to find pretrained models!\n",
"\n",
"* [`tf.keras.applications`](https://www.tensorflow.org/api_docs/python/tf/keras/applications) - A module built-in to TensorFlow and Keras with a series of pretrained models ready to use.\n",
"* TK - keras CV, keras NLP \n",
"* [`Hugging Face Models Hub`](https://huggingface.co/models) - A large collection of pretrained models on a wide range on tasks, from computer vision to natural language processing to audio processing. \n",
"* [`Kaggle Models`](https://www.kaggle.com/models) - A huge collection of different pretrained models for many different tasks.\n",
"\n",
"TK - image of where to find pretrained models\n",
"\n",
"> **Note:** For most new machine learning problems, if you're looking to get good results quickly, you should generally look for a pretrained model similar to your problem and use transfer learning to adapt it to your own domain.\n",
"\n",
"Since we're focused on TensorFlow/Keras, we're going to be using a pretrained model from `tf.keras.applications`.\n",
Expand Down Expand Up @@ -11684,9 +11724,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.8"
"version": "3.11.6"
}
},
"nbformat": 4,
"nbformat_minor": 1
"nbformat_minor": 4
}

0 comments on commit a22c402

Please sign in to comment.