34-Introduction to Scikit-Learn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6364347b",
   "metadata": {},
   "source": [
    "# Introduction to Scikit-Learn "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f762c904",
   "metadata": {},
   "source": [
    "Hi Guys, Welcome to [Tirendaz Academy](https://youtube.com/c/tirendazacademy) 😀\n",
    "\n",
    "---\n",
    "I'm goint to talk about Scikit-Learn library.\n",
    "In short, the notebooks covers the following topics:\n",
    "- Buiding the model\n",
    "- Data scaling\n",
    "- Pipelines\n",
    "- Model evaluation\n",
    "- Automatic parameter searches\n",
    "---\n",
    "Happy Learning 😀"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30cdddb0",
   "metadata": {},
   "source": [
    "## Building the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "480fe85e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4cad5f14",
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = RandomForestClassifier(random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "32955226",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = [[1,2,3],\n",
    "    [11,12,13]]\n",
    "y = [0,1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d62cc629",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(random_state=0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2ace347e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "cee2fa98",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.predict([[4,5,6],[14,15,16]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66b5faca",
   "metadata": {},
   "source": [
    "## Data Scaling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "da27d7f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "65a57d41",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = [[0,15],\n",
    "    [1,-10]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "cf25ea2e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.,  1.],\n",
       "       [ 1., -1.]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "StandardScaler().fit(X).transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d73ec657",
   "metadata": {},
   "source": [
    "## Pipelines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "dbd78b6d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.pipeline import make_pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "bc0fdc58",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "569ef689",
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe = make_pipeline(\n",
    "    StandardScaler(),\n",
    "    LogisticRegression())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "0d65c2e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0bd2e277",
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = load_iris(return_X_y=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "efb8fd0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "7174c6b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c29bb899",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(steps=[('standardscaler', StandardScaler()),\n",
       "                ('logisticregression', LogisticRegression())])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "d1f495da",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "73aad939",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9736842105263158"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy_score(pipe.predict(X_test), y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31cf4451",
   "metadata": {},
   "source": [
    "## Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "596b5681",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "29dfa76e",
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = make_regression(n_samples=1000, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "195c4393",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "641adfca",
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = LinearRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "8fe5ed29",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_validate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "c5be67b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "result = cross_validate(lr, X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "850270d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1., 1., 1., 1., 1.])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result[\"test_score\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0673e6ea",
   "metadata": {},
   "source": [
    "## Automatic Parameter Searches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "38f558da",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_california_housing "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "4c726be6",
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = fetch_california_housing(return_X_y=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "d1711a04",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X,y,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "781a0520",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import RandomizedSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "ae20b66b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.stats import randint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "8d39412a",
   "metadata": {},
   "outputs": [],
   "source": [
    "param_distributions = {\"n_estimators\" : randint(1,5),\n",
    "                      \"max_depth\" : randint(5,10)} "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "225a140f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "search = RandomizedSearchCV(\n",
    "    estimator = RandomForestRegressor(random_state=0),\n",
    "    n_iter = 5, \n",
    "    param_distributions = param_distributions,\n",
    "    random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "6cb5ab7a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomizedSearchCV(estimator=RandomForestRegressor(random_state=0), n_iter=5,\n",
       "                   param_distributions={'max_depth': <scipy.stats._distn_infrastructure.rv_frozen object at 0x00000246C75F2610>,\n",
       "                                        'n_estimators': <scipy.stats._distn_infrastructure.rv_frozen object at 0x00000246C8145C10>},\n",
       "                   random_state=0)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "150412c8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'max_depth': 9, 'n_estimators': 4}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "e18e5390",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.735363411343253"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b493bac",
   "metadata": {},
   "source": [
    "Don't forget to follow us on [YouTube](http://youtube.com/tirendazacademy) | [Medium](http://tirendazacademy.medium.com) | [Twitter](http://twitter.com/tirendazacademy) | [GitHub](http://github.com/tirendazacademy) | [Linkedin](https://www.linkedin.com/in/tirendaz-academy) | [Kaggle](https://www.kaggle.com/tirendazacademy) 😎"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}