Welcome_Introduction_to_Machine_Learning_Labs.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Welcome: Introduction to Machine Learning Labs.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [
        "JndnmDMp66FL"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "metadata": {
        "colab_type": "text",
        "id": "JndnmDMp66FL"
      },
      "cell_type": "markdown",
      "source": [
        "#### Copyright 2018 Google LLC."
      ]
    },
    {
      "metadata": {
        "cellView": "both",
        "colab_type": "code",
        "id": "hMqWDc_m6rUC",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "tfHOpuas8Uez"
      },
      "cell_type": "markdown",
      "source": [
        "#Applied Machine Learning Semester Course, Tech Exchange Spring 2019#"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "7nzTPsFQqEMA"
      },
      "cell_type": "markdown",
      "source": [
        "## Course Overview\n",
        "\n",
        "Here are the [**Course Slides**](https://services.google.com/fh/files/misc/intro_machine_learning_course_slides_techx_fall2018.pdf), which are divided into the following topics.  Links to relevant material in the [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course) are also listed.\n",
        "\n",
        "* [Goals](https://services.google.com/fh/files/misc/ml_semester_course_goals.pdf)\n",
        "* [Applications](https://services.google.com/fh/files/misc/ml_semester_course_applications.pdf)\n",
        "* [Intro to AI and ML](https://services.google.com/fh/files/misc/ml_semester_course_intro_ai_ml.pdf)\n",
        "    * [Video: Intro to ML](https://developers.google.com/machine-learning/crash-course/ml-intro)\n",
        "    * [Video: Framing](https://developers.google.com/machine-learning/crash-course/framing/video-lecture)\n",
        "    * [Self Study: ML Framing/Terminology](https://developers.google.com/machine-learning/crash-course/framing/ml-terminology)\n",
        "    * [Check Your Understanding](https://developers.google.com/machine-learning/crash-course/framing/check-your-understanding)\n",
        "* [Intro to Linear Regression](https://services.google.com/fh/files/misc/ml_semester_course_intro_linear_regression.pdf)\n",
        "    * [Self-Study: Linear Regression](https://developers.google.com/machine-learning/crash-course/descending-into-ml/linear-regression)\n",
        "* [ Linear Regression with many features](https://services.google.com/fh/files/misc/ml_semester_course_multivariate_linear_regression.pdf)\n",
        "* [ Visualizing Loss with a Scatter Plot](https://services.google.com/fh/files/misc/ml_semester_course_scatter_plot.pdf)\n",
        "    * [Check Your Understanding](https://developers.google.com/machine-learning/crash-course/descending-into-ml/check-your-understanding)\n",
        "* [Visualizing Loss with a Calibration Plot](https://services.google.com/fh/files/misc/ml_semester_course_calibration_plot.pdf)\n",
        "* [Quantifying Loss](https://services.google.com/fh/files/misc/ml_semester_course_quantifying_loss.pdf)\t\n",
        "* [Squared Loss and RMSE](https://services.google.com/fh/files/misc/ml_semester_course_rmse.pdf)\n",
        "    * [Video: Squared Loss](https://developers.google.com/machine-learning/crash-course/descending-into-ml/video-lecture)\n",
        "    * [Self-Study: Training and Loss](https://developers.google.com/machine-learning/crash-course/descending-into-ml/training-and-loss)\n",
        "* [Intro to SGD (Stochastic Gradient Descent)](https://services.google.com/fh/files/misc/ml_semester_course_intro_to_sgd.pdf)\n",
        "    * [Video: Reducing Loss](https://developers.google.com/machine-learning/crash-course/reducing-loss/video-lecture)\n",
        "    * [Self-Study: An Iterative Approach to Reducing Loss](https://developers.google.com/machine-learning/crash-course/reducing-loss/an-iterative-approach)\n",
        "    * [Self-Study: Gradient Descent](https://developers.google.com/machine-learning/crash-course/reducing-loss/gradient-descent)\n",
        "    * [Self-Study: SGD](https://developers.google.com/machine-learning/crash-course/reducing-loss/stochastic-gradient-descent)\n",
        "    * [Check Your Understanding : Batch Size](https://developers.google.com/machine-learning/crash-course/reducing-loss/check-your-understanding)\n",
        "* [Using SGD in Tensorflow](https://services.google.com/fh/files/misc/ml_semester_course_sgd_in_tensorflow.pdf)\n",
        "    * ([Video: First Steps with TF](https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/video-lecture)\n",
        "    * [TF Estimators](https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit)\n",
        "* [Using Learning Curve to Adjust the Learning Rate](https://services.google.com/fh/files/misc/ml_semester_course_learning_rate.pdf)\n",
        "    * [Self-Study: Learning Rate](https://developers.google.com/machine-learning/crash-course/reducing-loss/learning-rate),\n",
        "    * [Exercise: Optimizing Learning Rate](https://developers.google.com/machine-learning/crash-course/fitter/graph)\n",
        "    * [Playground Exercise: Learning Rate and Convergence](https://developers.google.com/machine-learning/crash-course/reducing-loss/playground-exercise)\n",
        "    * [Additional Programming Exercises](https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/programming-exercises)\n",
        "* [Introduction to Feature Engineering](https://services.google.com/fh/files/misc/ml_semester_course_intro_feature_engineering.pdf)\n",
        "    * [Video: Representation](https://developers.google.com/machine-learning/crash-course/representation/video-lecture)\n",
        "    * [Self-Study: Qualities of a Good Feature](https://developers.google.com/machine-learning/crash-course/representation/qualities-of-good-features)\n",
        "* [Scaling and Clipping Numeric Features](https://services.google.com/fh/files/misc/ml_semester_course_scaling_and_clipping.pdf)\n",
        "    * [Self-Study: Cleaning Data](https://developers.google.com/machine-learning/crash-course/representation/cleaning-data)\n",
        "* [Introducing Non-Linearities via Bucketizing Features](https://services.google.com/fh/files/misc/ml_semester_course_bucketizing_features.pdf)\n",
        "* [Using Quantiles to Compute Bucket Boundaries](https://services.google.com/fh/files/misc/ml_semester_course_quantiles_for_bucket_boundaries.pdf)\n",
        "* [ Representing Categorical Features](https://services.google.com/fh/files/misc/ml_semester_course_categorical_features.pdf)\n",
        "    * [Self-Study](https://developers.google.com/machine-learning/crash-course/representation/feature-engineering)\n",
        "* [Encoding Text](https://services.google.com/fh/files/misc/ml_semester_course_encoding_text.pdf)\n",
        "* [When to Treat Numerical Data as Categorical](https://services.google.com/fh/files/misc/ml_semester_course_numerical_vs_categorical.pdf)\n",
        "* [ Handling Missing Data](https://services.google.com/fh/files/misc/ml_semester_course_handling_missing_data.pdf),\n",
        "* [Introducing Non-Linearities via Feature Crosses](https://services.google.com/fh/files/misc/ml_semester_course_feature_crosses.pdf)\n",
        "    * [Video: Feature Crosses](https://developers.google.com/machine-learning/crash-course/feature-crosses/video-lecture)\n",
        "    * [Self-Study:Encoding Non-Linearity](https://developers.google.com/machine-learning/crash-course/feature-crosses/encoding-nonlinearity)\n",
        "    * [Self-Study: Crossing One-Hot Vectors](https://developers.google.com/machine-learning/crash-course/feature-crosses/crossing-one-hot-vectors)\n",
        "    * [Playground Exercise: Feature Crosses](https://developers.google.com/machine-learning/crash-course/feature-crosses/playground-exercises)\n",
        "    * [Playground Exercise: Overcrossing](https://developers.google.com/machine-learning/crash-course/regularization-for-simplicity/playground-exercise-overcrossing)\n",
        "* [Mathematics of Stochastic Gradient Descent (SGD)](https://services.google.com/fh/files/misc/ml_semester_course_mathematics_of_sgd.pdf)\n",
        "* [Recognizing and Preventing Overfitting](https://services.google.com/fh/files/misc/ml_semester_course_overfitting.pdf)\n",
        "    * [Video: Generalization](https://developers.google.com/machine-learning/crash-course/generalization/video-lecture)\n",
        "    * [Self-Study: Perils of Overfitting](https://developers.google.com/machine-learning/crash-course/generalization/peril-of-overfitting)\n",
        "* [Model Complexity and Occam's Razor](https://services.google.com/fh/files/misc/ml_semester_course_occams_razor.pdf)\n",
        "* [The Role of Test Data](https://services.google.com/fh/files/misc/ml_semester_course_role_of_test_data.pdf)\n",
        "* [Training, Validation, and Test Data Sets](https://services.google.com/fh/files/misc/ml_semester_course_validation_and_test_data.pdf)\n",
        "    * [Video: Training and Test Sets ](https://developers.google.com/machine-learning/crash-course/training-and-test-sets/video-lecture),\n",
        "    * [Video Lecture: Validation](https://developers.google.com/machine-learning/crash-course/validation/video-lecture)\n",
        "    * [Self Study: Splitting Data](https://developers.google.com/machine-learning/crash-course/training-and-test-sets/splitting-data)\n",
        "    * [Self-Study: Validation Data](https://developers.google.com/machine-learning/crash-course/validation/another-partition)\n",
        "    * [Playground Exercise: Training and Test Sets](https://developers.google.com/machine-learning/crash-course/training-and-test-sets/playground-exercise) \n",
        "    * [Check Your Understanding](https://developers.google.com/machine-learning/crash-course/validation/check-your-intuition)\n",
        "* [k-fold Cross Validation](https://services.google.com/fh/files/misc/ml_semester_course_cross_validation.pdf)\n",
        "* [Regularization for Simplicty](https://services.google.com/fh/files/misc/ml_semester_course_regularization_for_simplicity.pdf)\n",
        "    * [Video: Regularization for Simplicity](https://developers.google.com/machine-learning/crash-course/regularization-for-simplicity/video-lecture)\n",
        "* [L2 Regularization to Reduce Overfitting\t201-203](https://services.google.com/fh/files/misc/ml_semester_course_l2_regularization.pdf)\n",
        "    * [Self-Study: L₂ Regularization](https://developers.google.com/machine-learning/crash-course/regularization-for-simplicity/l2-regularization),\n",
        "    * [Self-Study: Lambda](https://developers.google.com/machine-learning/crash-course/regularization-for-simplicity/lambda),\n",
        "    * [Playground Exercise: Examining L2 Regularization](https://developers.google.com/machine-learning/crash-course/regularization-for-simplicity/playground-exercise-examining-l2-regularization),\n",
        "    * [Check Your Understanding](https://developers.google.com/machine-learning/crash-course/regularization-for-simplicity/check-your-understanding)\n",
        "* [Linear Classification (Logistic Regression)](https://services.google.com/fh/files/misc/ml_semester_course_logistic_regression.pdf)\n",
        "    * [Video: Logistic Regression](https://developers.google.com/machine-learning/crash-course/logistic-regression/video-lecture)\n",
        "    * [Video: Classification](https://developers.google.com/machine-learning/crash-course/classification/video-lecture)\n",
        "    * [Self-Study: Calculating A Probability](https://developers.google.com/machine-learning/crash-course/logistic-regression/calculating-a-probability)\n",
        "    * [Self-Study: Model Training](https://developers.google.com/machine-learning/crash-course/logistic-regression/model-training)\n",
        "* [Log Loss and Need for Regularization](https://services.google.com/fh/files/misc/ml_semester_course_log_loss.pdf)\n",
        "* [Callibration Plot to Recognize Prediction Bias](https://services.google.com/fh/files/misc/ml_semester_course_prediction_bias.pdf)\n",
        "    * [Self-Study: Prediction Bias](https://developers.google.com/machine-learning/crash-course/classification/prediction-bias)\n",
        "* [Evaluation Metrics for Linear Classification](https://services.google.com/fh/files/misc/ml_semester_course_metrics_for_classification.pdf)\n",
        "    * [Self-Study: Thresholding](https://developers.google.com/machine-learning/crash-course/classification/thresholding)\n",
        "    * [Self-Study: True vs. False; Positive vs. Negative](https://developers.google.com/machine-learning/crash-course/classification/true-false-positive-negative)\n",
        "    * [Self-Study: Accuracy](https://developers.google.com/machine-learning/crash-course/classification/accuracy),\n",
        "    * [Self-Study: Precision and Recall](https://developers.google.com/machine-learning/crash-course/classification/precision-and-recall)\n",
        "    * [Check Your Understanding](https://developers.google.com/machine-learning/crash-course/classification/check-your-understanding-accuracy-precision-recall)\n",
        "* [ROC Curve to Measure Prediciont vs Recall Trade-Off](https://services.google.com/fh/files/misc/ml_semester_course_roc_curve.pdf)\n",
        "    * [Self-Study: ROC and AUC](https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc)\n",
        "    * [Check Your Understanding](https://developers.google.com/machine-learning/crash-course/classification/check-your-understanding-roc-and-auc)\n",
        "* [Confusion Matrix](https://services.google.com/fh/files/misc/ml_semester_course_confusion_matrix.pdf)\n",
        "* [L1 Regularization for Reducing Model Size](https://services.google.com/fh/files/misc/ml_semester_course_l1_regularization.pdf)\n",
        "    * [Video: Regularization for Sparsity](https://developers.google.com/machine-learning/crash-course/regularization-for-sparsity/video-lecture)\n",
        "    * [Self-Study: L1 Regularization](https://developers.google.com/machine-learning/crash-course/regularization-for-sparsity/l1-regularization)\n",
        "    * [Playground Exercise: L1 Regularization](https://developers.google.com/machine-learning/crash-course/regularization-for-sparsity/playground-exercise)\n",
        "    * [Check Your Understanding](https://developers.google.com/machine-learning/crash-course/regularization-for-sparsity/check-your-understanding)\n",
        "* [Introduction to Deep Neural Networks](https://services.google.com/fh/files/misc/ml_semester_course_intro_to_dnns.pdf)\n",
        "    * [Video](https://developers.google.com/machine-learning/crash-course/introduction-to-neural-networks/video-lecture)\n",
        "    * [Self-Study: Anatomy](https://developers.google.com/machine-learning/crash-course/introduction-to-neural-networks/anatomy)\n",
        "    * [Playground Exercise](https://developers.google.com/machine-learning/crash-course/introduction-to-neural-networks/playground-exercises)\n",
        "* [Training DNNs with Backpropogation](https://services.google.com/fh/files/misc/ml_semester_course_backprop.pdf)\n",
        "    * [Video: Training Neural Nets](https://developers.google.com/machine-learning/crash-course/training-neural-networks/video-lecture)\n",
        "    * [Self-Study: Best Practices](https://developers.google.com/machine-learning/crash-course/training-neural-networks/best-practices)\n",
        "* [Multi-Class DNNs](https://services.google.com/fh/files/misc/ml_semester_course_multi_class_dnns.pdf)\n",
        "    * [Video](https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/video-lecture)\n",
        "    * [Self-Study: One vs. All](https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/one-vs-all)\n",
        "    * [Self-Study: Softmax](https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/softmax)\n",
        "* [Embeddings Motivation From Collaborative Filtering](https://services.google.com/fh/files/misc/ml_semester_course_collaborative_filtering.pdf)\n",
        "    * [Video: Embeddings](https://developers.google.com/machine-learning/crash-course/embeddings/video-lecture),\n",
        "    * [Self-Study: Motivation from Collaborative Filtering](https://developers.google.com/machine-learning/crash-course/embeddings/motivation-from-collaborative-filtering),\n",
        "    * [Self-Study: Categorical Input Data](https://developers.google.com/machine-learning/crash-course/embeddings/categorical-input-data)\n",
        "    * [Self-Study: Translating to a Lower-Dimensional Space](https://developers.google.com/machine-learning/crash-course/embeddings/translating-to-a-lower-dimensional-space)\n",
        "    * [Self-Study: Obtaining Embeddings](https://developers.google.com/machine-learning/crash-course/embeddings/obtaining-embeddings)\n",
        "* [Learning Embeddings in a DNN](https://services.google.com/fh/files/misc/ml_semester_course_learning_embeddings_in_dnn.pdf)\n",
        "* [Unsupervised Learning: Clustering](https://services.google.com/fh/files/misc/ml_semester_course_unsupervised_learning.pdf)\n",
        "* [K-Means Clustering](https://services.google.com/fh/files/misc/ml_semester_course_k_means.pdf)\n",
        "* [Course Overview/Review](https://services.google.com/fh/files/misc/ml_semester_course_overview_review.pdf)\n"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "ryLibrBnduH7"
      },
      "cell_type": "markdown",
      "source": [
        "## Additional Topics\n",
        "\n",
        "* ML Engineering ([Video: Production Systems](https://developers.google.com/machine-learning/crash-course/production-ml-systems))\n",
        "* ML Real World Examples ([Video: Cancer Prediction](https://developers.google.com/machine-learning/crash-course/cancer-prediction), [Video: 18th Century Literature](https://developers.google.com/machine-learning/crash-course/18th-century-literature), [Video: Real-World Guidelines](https://developers.google.com/machine-learning/crash-course/real-world-guidelines))\n",
        "* Fairness ([Video](https://developers.google.com/machine-learning/crash-course/fairness/video-lecture), [Types of Bias](https://developers.google.com/machine-learning/crash-course/fairness/types-of-bias), [Self-Study: Identifying Bias](https://developers.google.com/machine-learning/crash-course/fairness/identifying-bias), [Self-Study: Evaluating For Bias](https://developers.google.com/machine-learning/crash-course/fairness/evaluating-for-bias), [Check Your Understanding](https://developers.google.com/machine-learning/crash-course/fairness/check-your-understanding), [Programming Exercise](https://colab.sandbox.google.com/github/google/eng-edu/blob/master/ml/cc/exercises/intro_to_fairness.ipynb?utm_source=mlcc&utm_campaign=colab-external&utm_medium=referral&utm_content=fairness-colab&hl=en))"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "7_a8JPWCqVYu"
      },
      "cell_type": "markdown",
      "source": [
        "## Creating Your Copy for Each Lab\n",
        "\n",
        "* Start each lab by saving a copy, which you can do by going to the `File` menu and selecting \"`Save a copy in Drive`\"\n",
        "* In Drive, modify \"`Copy of ...`\" to whatever name you'd like for your notebook\n",
        "* Colab notebooks can be shared just as you would with Google Docs or Sheets. Simply click the `Share button` at the top right of your notebook, or follow these [Google Drive file sharing instructions](https://support.google.com/drive/answer/2494822?co=GENIE.Platform%3DDesktop&hl=en).\n",
        "\n",
        "## Some Basics of Using Colab\n",
        "\n",
        "* Since Colab runs in the Cloud you do not need to load any libraries on your computer.\n",
        "* When doing these labs, you'll generally run one cell at a time looking at and thinking about the results before moving to the next cell.\n",
        "* Remember whenever you modify a cell you will need to run that cell again for the change to take effect.\n",
        "* It is common to have code blocks that do not generate any output (e.g. imports, definition of procedures,...). These generally run quickly, and you'll know code execution is complete when the \"arrow\" to the left of the cell is shown again. When you select the next cell you will see a number showing the order in which the code blocks were executed.  If you want an output when a code block is run you can always choose to place a print statement at the end of the block.\n",
        "* The state for a notebook is global so even if you remove the definition of a variable once a cell has been run, it will still be defined.  Go to the `Runtime` menu and select \"`Restart runtime...`\" if you want to reset the runtime.  After doing this you will need to re-run the cells.  If you have made some changes to fix a bug, you might want to do this to be sure you are starting from a fresh state.\n",
        "* Code is executed in a virtual machine dedicated to your account. Virtual machines are recycled when idle for a while, and have a maximum lifetime enforced by the system. You will need to rerun all cells when your virtual machine is recycled.\n",
        "* Within the `Runtime` menu you can use \"`Run all`\", `\"Run before`\", or \"`Run after`\" to run multiple cells at once.\n",
        "* Use the `+ CODE` button between cells (or at the top) to add a cell for code.\n",
        "* Use the `+ TEXT` button between cells (or at the top) to add a cell for text.\n",
        "* To edit a text cell double click on it and then edit using the [mark-up language](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/markdown_guide.ipynb).\n",
        "* For a lot more info see [Overview of Colaboratory Features](https://colab.sandbox.google.com/notebooks/basic_features_overview.ipynb)"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "LyH6DsTj7sIH"
      },
      "cell_type": "markdown",
      "source": [
        "## Labs that use the UCI Automobile Data Set for Predicting Real-Valued Labels (Regression)\n",
        "\n",
        "We start with the  UCI Automobile data set since it is an easy to understand data set that has missing data and both numerical and categorical data.\n",
        "\n",
        "We start with using Pandas to load and explore the raw data.  Next we put together all the pieces needed to train a linear regression model in TensorFlow and to visualize the results and learning curve. You will begin to explore setting the learning rate and number of steps in this first lab.\n",
        "\n",
        "Next, you will learn how to train a model with multiple features including some key feature enginering such as feature normalization, bucketizing real-valued features, and feature crosses .Finally,you will also incorporate categorical features with the goal of training the best model then can to predict the city mpg for a car based on the other available features.\n",
        "\n",
        "\n",
        "###[Lab 1: Loading and Understanding Your Data](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/intro_to_ml_semester_course/Spring2019/Lab_1__Loading_and_Understanding_Your_Data.ipynb)##\n",
        "\n",
        "**Learning Objectives:**\n",
        "\n",
        "* Learn the basics of reading data with Pandas.\n",
        "* Learn the basics of data cleaning and handling missing data using Pandas.\n",
        "* Learn how to visualize data with a scatter plot.\n",
        "* Use Numpy to generate the line minimizing squared loss.\n",
        "* Explore visually the difference in the model when replacing missing items by 0s versus the mean value for that feature.\n",
        "\n",
        "###[Lab 2: Training Your First TF Linear Regression Model](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/intro_to_ml_semester_course/Spring2019/Lab_2__Training_Your_First_TF_Linear_Regression_Model.ipynb)\n",
        "\n",
        "**Learning Objectives:**\n",
        "\n",
        "* Use pyplot to help visualize the data, the learned model, and how the loss is evolving during training.\n",
        "* Learn how to set up the features in TensorFlow to train a model.\n",
        "* Use the LinearRegressor class in TensorFlow to predict a real-valued featured based on one real-valued input feature.\n",
        "* Evaluate the accuracy of a model's predictions using Root Mean Squared Error (RMSE).\n",
        "* Improve the accuracy of a model by tuning the learning rate and number of training steps.\n",
        "\n",
        "###[Lab 3: Using Multiple Numerical Features and Feature Scaling](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/intro_to_ml_semester_course/Spring2019/Lab_3__Using_Multiple_Numerical_Features_and_Feature_Scaling.ipynb)\n",
        "\n",
        "**Learning Objectives:**\n",
        "* Train a model using more than one feature and use a calibration plot to visualize the model quality.\n",
        "* Learn the importance of feature transformations.\n",
        "* Introduce linear and log transformations of features.\n",
        "\n",
        "###[Lab 4: Using Bucketized and Categorical Features](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/intro_to_ml_semester_course/Spring2019/Lab_4__Using_Bucketized_and_Categorical_Features.ipynb)\n",
        "\n",
        "**Learning Objectives:**\n",
        "* Create bucketized numerical features in TF and use them to train a model.\n",
        "* Use visualizations to understand the value of using bucketized features.\n",
        "* Use numerical and categorical features in TF to train a model.\n"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "ROaNpjcSKCJu"
      },
      "cell_type": "markdown",
      "source": [
        "## Lab using the California Housing Data for Predicting Real-Valued Labels\n",
        "\n",
        "The next lab uses the California Housing Data. We start by splitting the training data into a train and validation set, and visually demonstrating what can happen if you don’t randomize the data before creating the train/validation data split.  Next we introduce synthetic features since those are important for this data set and this is a nice illustration of the kind of feature engineering that can be done to train good linear models on real data sets.\n",
        "\n",
        "###[Lab 5: Creating Validation Data and Synthetic Features](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/intro_to_ml_semester_course/Spring2019/Lab_5__Creating_Validation_Data_and_Synthetic_Features.ipynb)\n",
        "\n",
        "**Learning Objectives:**\n",
        " * Generate a training and validation data set for housing data that we will use to predict the median housing price, at the granularity of city blocks.\n",
        " * Debug issues in the creation of the training and validation splits.\n",
        " * Use a validation data set and test set to make sure that our model will generalize and is not overfitting the training data.\n",
        " * Use test data only after tuning hyperparameters as a measure of how the model will generalize to new data.\n",
        " * Create synthetic features from the existing features (e.g., taking a ratio of two other features).\n",
        " * More practice with feature transformations including identifying and clipping (removing) outliers out of the input data to obtain the best model.\n"
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "91XQFwilMEuL"
      },
      "cell_type": "markdown",
      "source": [
        "## Labs that use the Census Data for a Classification Problem\n",
        "\n",
        "\n",
        "We start with a linear classifier that just uses the raw numerical and categorical features.  We had introduced bucketized features earlier but in the first lab on this data sets introduces quantiles as a way to avoid hand picking the thresholds and also combines the bucketized features with the raw features. The next step to further improve a linear model is to introduce feature crosses.  After doing this, the students should begin to see some overfitting so it’s natural to introduce L2 regularization, and then L1 regularization to reduce the model size.  Finally, a DNN can be introduced and compared to a linear model.  Students should begin to think about the pros and cons in moving from a linear model with crosses to a DNN.\n",
        "\n",
        "###[Lab 6: Training a Linear Classifier with Numerical and Categorical Features](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/intro_to_ml_semester_course/Spring2019/Lab_6__Training_a_Linear_Classifier_with_Numerical_and_Categorical_Features.ipynb)\n",
        "\n",
        "**Learning Objectives:**\n",
        " * Introduce logistic regression to train a binary classifier.\n",
        " * Understand metrics such as ROC curves, AUC, log loss, classification errors.\n",
        " * Train a linear classifier using the raw numerical and categorical features.\n",
        " * Learn to use quantiles to create bucketized features.\n",
        " * Learn how to introduce feature crosses.\n",
        " * Train a linear classifier using numerical features, categorical features, bucketized features, and feature crosses.\n",
        "\n",
        "###[Lab 7: Regularization to Reduce Overfitting and Model Size](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/intro_to_ml_semester_course/Spring2019/Lab_7__Feature_Crosses_and_Regularization_to_Reduce_Overfitting_and_Model_Size.ipynb)\n",
        "\n",
        "**Learning Objectives:**\n",
        "* Replace the `StochasticGradientDescent` optimizer by the `FTRLOptimizer`.\n",
        "* Use L2 regularization to help reduce overfitting.\n",
        "* Use L1 regularization to create sparsity and reduce model size.\n",
        "* Look at ROC curves to understand trade-off between model size and accuracy.\n",
        "* Continue to develop more independence in training and evaluating models.."
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "MjZmbofz-7M7"
      },
      "cell_type": "markdown",
      "source": [
        "## Lab Training a DNN using the MNIST Digit Classification Data Set\n",
        "\n",
        "###[Lab 8: DNN For Classifying Handwritten Digits](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/intro_to_ml_semester_course/Spring2019/Lab_8__DNN_For_Classifying_Handwritten_Digits.ipynb)\n",
        "\n",
        "**Learning Objectives:**\n",
        "  * Train both a linear model and a neural network to classify handwritten digits from the classic [MNIST](http://yann.lecun.com/exdb/mnist/) data set.\n",
        "  * Create a DNN with a Softmax layer for a multiclass classification problem.\n",
        "  * Compare the performance of the linear and neural network classification models.\n",
        "  * Use a confusion matrix to visualize the model performance\n",
        "  * Visualize the weights of a neural-network hidden layer."
      ]
    },
    {
      "metadata": {
        "colab_type": "text",
        "id": "xT8U2H7HM6bz"
      },
      "cell_type": "markdown",
      "source": [
        "## Lab to Apply Embeddings for Sentiment Analysis\n",
        "\n",
        "The final data set introduces the students to training embeddings in a DNN with the application are of sentiment analysis.\n",
        "\n",
        "###[Lab 9: Learning Embeddings for Sentiment Analysis](https://colab.research.google.com/notebook#fileId=/v2/external/notebooks/intro_to_ml_semester_course/Spring2019/Lab_9__Learning_Embeddings_For_Sentiment_Analysis.ipynb)\n",
        "\n",
        "**Learning Objectives:**\n",
        "* Represent movie-review as a bag of words\n",
        "* Implement a sentiment-analysis linear model\n",
        "* Implement a sentiment-analysis DNN model using an embedding that projects data into two dimensions\n",
        "* Visualize the embedding to see what the model has learned about the relationships between words"
      ]
    }
  ]
}