Basic Numerical Linear Algebra/2-Multiplying_Matrices_and_Vectors.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyMSmY5IIoQCJBI9GQH5HztX",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/MonitSharma/Numerical-Linear-Algebra/blob/main/Multiplying_Matrices_and_Vectors.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Author: [Monit Sharma](https://github.com/MonitSharma),\n",
        "        LinkedIn: [Monit Sharma](https://www.linkedin.com/in/monitsharma/),\n",
        "        Twitter: [@MonitSharma1729](https://twitter.com/MonitSharma1729)"
      ],
      "metadata": {
        "id": "Qnlg0jmmRIdP"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "-DqOJ5BnRHCT"
      },
      "outputs": [],
      "source": [
        "# import numpy\n",
        "import numpy as np\n",
        "\n",
        "# avoid innacurate floating points \n",
        "np.set_printoptions(suppress=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Introduction\n",
        "\n",
        "We will see some very important concepts in this chapter. The dot product is used in every equation explaining data science algorithms so it's worth the effort to understand it. Then we will see some properties of this operation. Finally, we will to get some intuition on the link between matrices and systems of linear equations."
      ],
      "metadata": {
        "id": "XjXmGj2dRnpF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Multiplying Matrices and Vectors\n",
        "\n",
        "The standard way to multiply matrices is not to multiply each element of one with each elements of the other (this is the element-wise product) but to calculate the sum of the products between rows and columns. \n",
        "\n",
        "-----\n",
        "\n",
        "The number of columns of the first matrix must be equal to the number of rows of the second matrix. Thus, if the dimensions, or the shape of the first matrix, is ($m\\times n$) the second matrix need to be of shape ($n \\times x$\n",
        "). The resulting matrix will have the shape ($m \\times x$\n",
        ").\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "wcLRSyGXRxje"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Example 1\n",
        "\n",
        "As a starter we will see the multiplication of a matrix and a vector\n",
        "\n",
        "$$ {A} \\times {b} = {C} $$\n",
        "\n",
        "with ${A}=\n",
        "\\begin{bmatrix}\n",
        "    1 & 2\\\\\\\\\n",
        "    3 & 4\\\\\\\\\n",
        "    5 & 6\n",
        "\\end{bmatrix}$ and $ {b}=\\begin{bmatrix}\n",
        "    2\\\\\\\\\n",
        "    4\n",
        "\\end{bmatrix}$.\n",
        "\n",
        "We saw that the formula is the following:\n",
        "\n",
        "$$\\begin{align*}\n",
        "&\\begin{bmatrix}\n",
        "    A_{1,1} & A_{1,2} \\\\\\\\\n",
        "    A_{2,1} & A_{2,2} \\\\\\\\\n",
        "    A_{3,1} & A_{3,2}\n",
        "\\end{bmatrix}\\times\n",
        "\\begin{bmatrix}\n",
        "    B_{1,1} \\\\\\\\\n",
        "    B_{2,1}\n",
        "\\end{bmatrix}=\\\\\\\\\n",
        "&\\begin{bmatrix}\n",
        "    A_{1,1}B_{1,1} + A_{1,2}B_{2,1} \\\\\\\\\n",
        "    A_{2,1}B_{1,1} + A_{2,2}B_{2,1} \\\\\\\\\n",
        "    A_{3,1}B_{1,1} + A_{3,2}B_{2,1}\n",
        "\\end{bmatrix}\n",
        "\\end{align*}$$\n",
        "\n",
        "So we will have:\n",
        "\n",
        "$$ \\begin{align*}\n",
        "&\\begin{bmatrix}\n",
        "    1 & 2 \\\\\\\\\n",
        "    3 & 4 \\\\\\\\\n",
        "    5 & 6\n",
        "\\end{bmatrix}\\times\n",
        "\\begin{bmatrix}\n",
        "    2 \\\\\\\\\n",
        "    4\n",
        "\\end{bmatrix}=\\\\\\\\\n",
        "&\\begin{bmatrix}\n",
        "    1 \\times 2 + 2 \\times 4 \\\\\\\\\n",
        "    3 \\times 2 + 4 \\times 4 \\\\\\\\\n",
        "    5 \\times 2 + 6 \\times 4\n",
        "\\end{bmatrix}=\n",
        "\\begin{bmatrix}\n",
        "    10 \\\\\\\\\n",
        "    22 \\\\\\\\\n",
        "    34\n",
        "\\end{bmatrix}\n",
        "\\end{align*} $$\n",
        "\n",
        "It is a good habit to check the dimensions of the matrix so see what is going on. We can see in this example that the shape of $A$\n",
        " is ($ 3 \\times 2$\n",
        ") and the shape of $b$\n",
        " is ($2 \\times 1$\n",
        "). So the dimensions of $C$\n",
        " are ($3 \\times 1$\n",
        ").\n"
      ],
      "metadata": {
        "id": "PWDNpjlsSHnQ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## With Numpy\n",
        "\n",
        "The Numpy function `dot()` can be used to compute the matrix product (or dot product). Let's try to reproduce the last exemple:\n"
      ],
      "metadata": {
        "id": "0OxVLDJmS4wW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[1, 2], [3, 4], [5, 6]])\n",
        "A"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ciZgF1HrRcGb",
        "outputId": "b0c5e813-0341-49be-c018-403e15ffdcd0"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1, 2],\n",
              "       [3, 4],\n",
              "       [5, 6]])"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "B = np.array([[2], [4]])\n",
        "B"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xQQ0PA55TAp4",
        "outputId": "f288ea0d-9177-4bb4-941e-a74bfd08e99d"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[2],\n",
              "       [4]])"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "C = np.dot(A, B)\n",
        "C"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Od5Tp-nMTCNQ",
        "outputId": "5b71d021-990e-4e4c-d700-e6319f240c7f"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[10],\n",
              "       [22],\n",
              "       [34]])"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It is equivalent to use the method `dot()` of Numpy arrays:"
      ],
      "metadata": {
        "id": "uDOi3zfqTGyA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "C = A.dot(B)\n",
        "C"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kVFOFjUVTD1H",
        "outputId": "1d8e84d5-bb29-45f1-dde0-5e9029f042e0"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[10],\n",
              "       [22],\n",
              "       [34]])"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Example 2\n",
        "\n",
        "Multiplication of two matrices\n",
        "\n",
        "$$ {A} \\times {B} = {C} $$\n",
        "\n",
        "with \n",
        "\n",
        "$$ {A}=\\begin{bmatrix}\n",
        "    1 & 2 & 3 \\\\\\\\\n",
        "    4 & 5 & 6 \\\\\\\\\n",
        "    7 & 8 & 9 \\\\\\\\\n",
        "    10 & 11 & 12\n",
        "\\end{bmatrix} $$\n",
        "\n",
        "and \n",
        "\n",
        "$$ {B}=\\begin{bmatrix}\n",
        "    2 & 7 \\\\\\\\\n",
        "    1 & 2 \\\\\\\\\n",
        "    3 & 6\n",
        "\\end{bmatrix}$$\n",
        "\n",
        "So we have:\n",
        "\n",
        "$$ \\begin{align*}\n",
        "&\\begin{bmatrix}\n",
        "    1 & 2 & 3 \\\\\\\\\n",
        "    4 & 5 & 6 \\\\\\\\\n",
        "    7 & 8 & 9 \\\\\\\\\n",
        "    10 & 11 & 12\n",
        "\\end{bmatrix}\\times\n",
        "\\begin{bmatrix}\n",
        "    2 & 7 \\\\\\\\\n",
        "    1 & 2 \\\\\\\\\n",
        "    3 & 6\n",
        "\\end{bmatrix}=\\\\\\\\\n",
        "&\\begin{bmatrix}\n",
        "    2 \\times 1 + 1 \\times 2 + 3 \\times 3 & 7 \\times 1 + 2 \\times 2 + 6 \\times 3 \\\\\\\\\n",
        "    2 \\times 4 + 1 \\times 5 + 3 \\times 6 & 7 \\times 4 + 2 \\times 5 + 6 \\times 6 \\\\\\\\\n",
        "    2 \\times 7 + 1 \\times 8 + 3 \\times 9 & 7 \\times 7 + 2 \\times 8 + 6 \\times 9 \\\\\\\\\n",
        "    2 \\times 10 + 1 \\times 11 + 3 \\times 12 & 7 \\times 10 + 2 \\times 11 + 6 \\times 12 \\\\\\\\\n",
        "\\end{bmatrix}\\\\\\\\\n",
        "&=\n",
        "\\begin{bmatrix}\n",
        "    13 & 29 \\\\\\\\\n",
        "    31 & 74 \\\\\\\\\n",
        "    49 & 119 \\\\\\\\\n",
        "    67 & 164\n",
        "\\end{bmatrix}\n",
        "\\end{align*} $$\n",
        "\n",
        "Let's verify with `numpy`"
      ],
      "metadata": {
        "id": "qpkyCeoeTMHM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]])\n",
        "A"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uzLViCOnTK49",
        "outputId": "7d76e225-c920-4ce4-b8ac-f61305b85649"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1,  2,  3],\n",
              "       [ 4,  5,  6],\n",
              "       [ 7,  8,  9],\n",
              "       [10, 11, 12]])"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "B = np.array([[2, 7], [1, 2], [3, 6]])\n",
        "B"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wUm8qK15TpK2",
        "outputId": "929711aa-5b51-4c26-a939-82ef46059887"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[2, 7],\n",
              "       [1, 2],\n",
              "       [3, 6]])"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "C = A.dot(B)\n",
        "C"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ukkDsHS8TrD_",
        "outputId": "2b9ea87c-ebd6-434d-a3eb-b443bdf5e6f4"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 13,  29],\n",
              "       [ 31,  74],\n",
              "       [ 49, 119],\n",
              "       [ 67, 164]])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Formalization of the dot product\n",
        "\n",
        "$$ C_{i,j} = A_{i,k}B_{k,j} = \\sum_{k}A_{i,k}B_{k,j} $$\n",
        "\n",
        "## Properties of the dot product\n",
        "\n",
        "We will now see some interesting properties of the matrix multiplication. It will become useful as we move forward in the chapters. Using simple examples for each property will provide a way to check them while we get used to the Numpy functions.\n",
        "\n",
        "\n",
        "## Matrices multiplication is distributive\n",
        "\n",
        "$$ {A}({B}+{C}) = {AB}+{AC} $$\n",
        "\n"
      ],
      "metadata": {
        "id": "hvQZWz3NTuW0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Example 3\n",
        "\n",
        "$$ {A}=\\begin{bmatrix}\n",
        "    2 & 3 \\\\\\\\\n",
        "    1 & 4 \\\\\\\\\n",
        "    7 & 6\n",
        "\\end{bmatrix}, \n",
        "{B}=\\begin{bmatrix}\n",
        "    5 \\\\\\\\\n",
        "    2\n",
        "\\end{bmatrix}, \n",
        "{C}=\\begin{bmatrix}\n",
        "    4 \\\\\\\\\n",
        "    3\n",
        "\\end{bmatrix}$$\n",
        "\n",
        "\n",
        "----\n",
        "\n",
        "$$ \\begin{align*}\n",
        "{A}({B}+{C})&=\\begin{bmatrix}\n",
        "    2 & 3 \\\\\\\\\n",
        "    1 & 4 \\\\\\\\\n",
        "    7 & 6\n",
        "\\end{bmatrix}\\times\n",
        "\\left(\\begin{bmatrix}\n",
        "    5 \\\\\\\\\n",
        "    2\n",
        "\\end{bmatrix}+\n",
        "\\begin{bmatrix}\n",
        "    4 \\\\\\\\\n",
        "    3\n",
        "\\end{bmatrix}\\right)=\n",
        "\\begin{bmatrix}\n",
        "    2 & 3 \\\\\\\\\n",
        "    1 & 4 \\\\\\\\\n",
        "    7 & 6\n",
        "\\end{bmatrix}\\times\n",
        "\\begin{bmatrix}\n",
        "    9 \\\\\\\\\n",
        "    5\n",
        "\\end{bmatrix}\\\\\\\\\n",
        "&=\n",
        "\\begin{bmatrix}\n",
        "    2 \\times 9 + 3 \\times 5 \\\\\\\\\n",
        "    1 \\times 9 + 4 \\times 5 \\\\\\\\\n",
        "    7 \\times 9 + 6 \\times 5\n",
        "\\end{bmatrix}=\n",
        "\\begin{bmatrix}\n",
        "    33 \\\\\\\\\n",
        "    29 \\\\\\\\\n",
        "    93\n",
        "\\end{bmatrix}\n",
        "\\end{align*}$$\n",
        "\n",
        "\n",
        "is equivalent to \n",
        "\n",
        "$$ \\begin{align*}\n",
        "{A}{B}+{A}{C} &= \\begin{bmatrix}\n",
        "    2 & 3 \\\\\\\\\n",
        "    1 & 4 \\\\\\\\\n",
        "    7 & 6\n",
        "\\end{bmatrix}\\times\n",
        "\\begin{bmatrix}\n",
        "    5 \\\\\\\\\n",
        "    2\n",
        "\\end{bmatrix}+\n",
        "\\begin{bmatrix}\n",
        "    2 & 3 \\\\\\\\\n",
        "    1 & 4 \\\\\\\\\n",
        "    7 & 6\n",
        "\\end{bmatrix}\\times\n",
        "\\begin{bmatrix}\n",
        "    4 \\\\\\\\\n",
        "    3\n",
        "\\end{bmatrix}\\\\\\\\\n",
        "&=\n",
        "\\begin{bmatrix}\n",
        "    2 \\times 5 + 3 \\times 2 \\\\\\\\\n",
        "    1 \\times 5 + 4 \\times 2 \\\\\\\\\n",
        "    7 \\times 5 + 6 \\times 2\n",
        "\\end{bmatrix}+\n",
        "\\begin{bmatrix}\n",
        "    2 \\times 4 + 3 \\times 3 \\\\\\\\\n",
        "    1 \\times 4 + 4 \\times 3 \\\\\\\\\n",
        "    7 \\times 4 + 6 \\times 3\n",
        "\\end{bmatrix}\\\\\\\\\n",
        "&=\n",
        "\\begin{bmatrix}\n",
        "    16 \\\\\\\\\n",
        "    13 \\\\\\\\\n",
        "    47\n",
        "\\end{bmatrix}+\n",
        "\\begin{bmatrix}\n",
        "    17 \\\\\\\\\n",
        "    16 \\\\\\\\\n",
        "    46\n",
        "\\end{bmatrix}=\n",
        "\\begin{bmatrix}\n",
        "    33 \\\\\\\\\n",
        "    29 \\\\\\\\\n",
        "    93\n",
        "\\end{bmatrix}\n",
        "\\end{align*}$$"
      ],
      "metadata": {
        "id": "G3e7wB46UFUt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[2, 3], [1, 4], [7, 6]])\n",
        "A"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "H18MnekiTuH3",
        "outputId": "ad65e7f3-4bf3-4bea-fd92-45c22228deee"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[2, 3],\n",
              "       [1, 4],\n",
              "       [7, 6]])"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "B = np.array([[5], [2]])\n",
        "B"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Jw9S50OXTspU",
        "outputId": "18ea0e55-1881-4693-e2cd-48defaa55c58"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[5],\n",
              "       [2]])"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "C = np.array([[4], [3]])\n",
        "C"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2N6U0FKIUikB",
        "outputId": "45e3fe84-ba9b-4feb-b3eb-a2a48e294f4d"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[4],\n",
              "       [3]])"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "$A(B+C)$"
      ],
      "metadata": {
        "id": "rJPwNB1lUlpv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "D = A.dot(B+C)\n",
        "D"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4migHKzXUk4c",
        "outputId": "3f9a0cad-b718-43a7-8416-c5bafb8d1b0d"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[33],\n",
              "       [29],\n",
              "       [93]])"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "is equivalent to $AB + AC$"
      ],
      "metadata": {
        "id": "6NeSLvoLUru6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "D = A.dot(B) + A.dot(C)\n",
        "D"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6iqzmkI-UqZW",
        "outputId": "c86cb5fe-05c2-49ed-b286-680cb017d0c4"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[33],\n",
              "       [29],\n",
              "       [93]])"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Matrices mutliplication is associative\n",
        "\n",
        "$$ {A}({BC}) = ({AB}){C} $$"
      ],
      "metadata": {
        "id": "RBYkT3ChUwdC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[2, 3], [1, 4], [7, 6]])\n",
        "A"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PEM4yXV3UvO_",
        "outputId": "34b7b68a-5c2e-46b4-c2e3-494531948b75"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[2, 3],\n",
              "       [1, 4],\n",
              "       [7, 6]])"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "B = np.array([[5, 3], [2, 2]])\n",
        "B"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PYJRkbPAU2Sx",
        "outputId": "9e1f88c3-2bed-4e3f-9875-855fcb60498a"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[5, 3],\n",
              "       [2, 2]])"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "$A(BC)$"
      ],
      "metadata": {
        "id": "f6j_no3EU8M7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "D = A.dot(B.dot(C))\n",
        "D"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cXkcxhlgU7KE",
        "outputId": "74feefe8-7eab-4272-b183-61f745c9f55d"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[100],\n",
              "       [ 85],\n",
              "       [287]])"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "is equivalent to $(AB)C$\n",
        ":"
      ],
      "metadata": {
        "id": "7DF5jOVvVANU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "D = (A.dot(B)).dot(C)\n",
        "D"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hbRIU6JXU-ny",
        "outputId": "a6008a4b-bb8f-4e0c-c9c5-2e4cd81fd6b9"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[100],\n",
              "       [ 85],\n",
              "       [287]])"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Matrix multiplication is not commutative\n",
        "\n",
        "$$ {AB} \\neq {BA} $$"
      ],
      "metadata": {
        "id": "fwQ3LibsVJeJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[2, 3], [6, 5]])\n",
        "A"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IZ39M-9KVGW-",
        "outputId": "848804a8-1e39-49af-fa33-eeb94c2f6fcf"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[2, 3],\n",
              "       [6, 5]])"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "B = np.array([[5, 3], [2, 2]])\n",
        "B"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ax8pm--XVPt1",
        "outputId": "8a5ab3a1-e8ed-463f-d538-3491e4b79e80"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[5, 3],\n",
              "       [2, 2]])"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "$AB$ "
      ],
      "metadata": {
        "id": "1C0LyVoJjlci"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "AB = np.dot(A, B)\n",
        "AB"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4E7FBSprVRGw",
        "outputId": "59f11284-36e4-4bb5-ae4c-c7ee0fd9443f"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[16, 12],\n",
              "       [40, 28]])"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "is different from $BA$"
      ],
      "metadata": {
        "id": "jUlmWvnqjn69"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "BA = np.dot(B, A)\n",
        "BA"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mKDg90wCVSPW",
        "outputId": "434a3880-cb46-4aa9-e0ac-643145c3388a"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[28, 30],\n",
              "       [16, 16]])"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## However vector multiplication is commutative\n",
        "\n",
        "$$ {x^{ \\text{T}}y} = {y^{\\text{T}}x}$$"
      ],
      "metadata": {
        "id": "N7sA-sjxjvr3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = np.array([[2], [6]])\n",
        "x"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dXylDCrOjtaJ",
        "outputId": "11fe3c41-b8d4-4b42-fa27-98ba9c4f7a53"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[2],\n",
              "       [6]])"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "y = np.array([[5], [2]])\n",
        "y\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "D00pl_i0j2YK",
        "outputId": "3c3ea0aa-ae2e-4d48-c006-a07bac66616a"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[5],\n",
              "       [2]])"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "${x^\\text{T}y}$"
      ],
      "metadata": {
        "id": "O8GhLpcxj5xv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x_ty = x.T.dot(y)\n",
        "x_ty"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XM35EuW1j3_8",
        "outputId": "8c740c2f-9048-4737-e333-5f9ab06a004d"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[22]])"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "is equivalent to ${y^\\text{T}x}$\n",
        ": "
      ],
      "metadata": {
        "id": "zFe42CkKj_DT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "y_tx = y.T.dot(x)\n",
        "y_tx"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XBc9Z_6dj9jE",
        "outputId": "46162d90-a037-43a7-b7b8-a8c3d61fa930"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[22]])"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Simplification of the matrix product\n",
        "\n",
        "$$ ({AB})^{\\text{T}} = {B}^\\text{T}\n",
        "{A}^\\text{T} $$"
      ],
      "metadata": {
        "id": "_r2EoCVSkFPl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[2, 3], [1, 4], [7, 6]])\n",
        "A"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C_dcnvXrkNjs",
        "outputId": "5ccf9550-9554-4127-f2c4-d7322a916203"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[2, 3],\n",
              "       [1, 4],\n",
              "       [7, 6]])"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "B = np.array([[5, 3], [2, 2]])\n",
        "B"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "O20MT5lPkPGb",
        "outputId": "b854b30c-c346-4f3c-bb2f-bf494716124c"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[5, 3],\n",
              "       [2, 2]])"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "$({AB})^{\\text{T}}$"
      ],
      "metadata": {
        "id": "1RjMq44vkSB9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "AB_t = A.dot(B).T\n",
        "AB_t"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4r4OXkeQkQg8",
        "outputId": "37f19028-fbb2-40b2-95c7-82a3e68300f8"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[16, 13, 47],\n",
              "       [12, 11, 33]])"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "is equivalent to ${B}^\\text{T}{A}^\\text{T}$\n",
        ": "
      ],
      "metadata": {
        "id": "ufbWCJv4kXbX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "B_tA = B.T.dot(A.T)\n",
        "B_tA"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AvbZumkBkV26",
        "outputId": "eafa1e99-9b22-47da-9ef3-7bc471d4db8e"
      },
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[16, 13, 47],\n",
              "       [12, 11, 33]])"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "LB8uy02-kdsi"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}