008_iam.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/rzl-ds/gu511/blob/master/008_iam.ipynb\" target=\"_parent\">\n",
    "    <img src=\"https://colab.research.google.com/assets/colab-badge.svg\"/>\n",
    "</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# `aws` identity access management (`iam`)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "I like to say that if you're having a problem with `aws`, it's probably going to get fixed on the `iam` console."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the most common problem area for new users of cloud services -- and *especially* `aws` -- is permissions and security. in fact, the task can be confusing and daunting enough that entire companies have sprung up offering to take care of these issues for a fee. given that this is already a full time job for experienced IT staffers in every organization with shared resources (*e.g.* any company or university), this is perhaps not surprising.\n",
    "\n",
    "it's easy to take that role for granted. I think learning some *basics* about what they do will not only allow you to appreciate what they're doing a bit more, it will give you the ability to think a bit more like they do, to diagnose and solve problems like they do, and to help communicate with them when help is needed.\n",
    "\n",
    "plus, you just have to figure it out. it may get easier over time, but that doesn't matter right now!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "with that set of \"this is hard\" caveats out of the way, you should know: the basics of user control are time-tested and well organized and in aws, generally speaking, the defaults work.\n",
    "\n",
    "aws has already gone to great lengths to make sure that everything you do in the cloud is as secure as possible -- that is, everything is locked down by default. our job is to learn how to *unlock* down... or is it to lock *up*... or...\n",
    "\n",
    "never mind."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## you already know a lot\n",
    "\n",
    "at this point we have a lot of experience and hopefully a fair amount of comfort navigating the `linux` environment and the command line. in particular, you know about permission structures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "ll ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "On my mac, I have a\n",
    "\n",
    "+ user `zach.lamberty`\n",
    "+ group `staff`\n",
    "+ different permission structures for different files\n",
    "\n",
    "On my `ec2`, I have a\n",
    "\n",
    "+ user `ubuntu`\n",
    "+ group `ubuntu`\n",
    "+ different permission structures for different files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "my mac user `zach.lamberty` and `ec2` user `ubuntu` is also a member of possibly multiple groups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "%%sh\n",
    "groups"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "this arrangement (having users, having users belong to groups, and having each file hold permissions on the user and group level) is fairly common and time-tested.\n",
    "\n",
    "we have users, we assign users to multiple groups, and we grant read, write, and some other combination of list / execute permission to different resources on a per-user, per-group, or global basis\n",
    "\n",
    "most of this carries over into `aws` in a service called `iam`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## semantics and relationships"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "permissions structures usually exist as *people* who belong to *groups* that can *do things*. the language changes from application to application, so let's lay down a couple of the main terms and the relationships between them:\n",
    "\n",
    "1. single people and things (\"things?\", you ask) are **`users`**\n",
    "2. collections of one or more **`users`** are called **`groups`**\n",
    "3. **`users`** and **`groups`** of **`users`** can \"take on\" (aka temporarily step into and out of) **`roles`**\n",
    "4. **`users`**, **`groups`**, and **`roles`** can possess **`permissions`** or **`policies`**\n",
    "5. **`policies`** are collections of **`permissions`**\n",
    "\n",
    "you're a sysad now, right?\n",
    "\n",
    "maybe some more details will help."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "open the [`iam` console](https://console.aws.amazon.com/iam/home?region=us-east-1) for your `aws` account"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## users\n",
    "\n",
    "a `user` is an individual account for your aws instance. although we typically think of a `user` as being a living person, the idea is generalized to include any independent entity which could do something (or be prevented from doing something):\n",
    "\n",
    "1. people who log in to machines and do things\n",
    "2. accounts you create to run specific programs (e.g. a database) or to do a specific job (e.g. run an ETL process)\n",
    "3. system accounts that need special elevated privileges\n",
    "4. other\n",
    "\n",
    "the end goal is to give every user permission to do the things they need to do -- and *only* those permissions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "in our initial account setup, we created a user for our personal aws use. let's look through the properties of that user object.\n",
    "\n",
    "1. open [the `iam` users dashboard](https://console.aws.amazon.com/iam/home?region=us-east-1#/users) (\"Users\" in the left menu)\n",
    "2. click the link in the \"User name\" column of the table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "1. permissions tab\n",
    "    1. the summary page starts off with the permission tab active\n",
    "    2. permissions that our special user has are collected here as \"policies\" (more below).\n",
    "    3. users can get permissions directly, or via group membership\n",
    "    4. the “policy type” field will explain the source of a given permission policy.\n",
    "    5. click the drop-down on any of the attached policies to see the full list of permissions this user has.\n",
    "2. groups tab\n",
    "    1. the \"groups\" tab displays all the groups to which your user belongs (and which permissions those `groups` grant you)\n",
    "3. Tags tab\n",
    "    1. this tab allows you to add \"tags\" to your user -- useful if you want to track information across users, add metadata, or group applications for billing based on the users involved\n",
    "4. security credentials tab\n",
    "    1. Access keys are created for API access to various services (see `aws cli` lecture)\n",
    "5. access advisor tab\n",
    "    1. for more advanced usage and creating access policies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**<div align=\"center\">mini exercise: look at your user's access advisor</div>**\n",
    "\n",
    "1. go to the `users` dashboard for the `iam` service\n",
    "1. access your user (the one you log in as, probably your `guid`)\n",
    "1. click on the \"Access Advisor\" tab\n",
    "1. search for `ec2` in the \"allowed services\" text box\n",
    "1. see the \"Last accessed\" section for information on when you last interacted with `ec2`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## groups\n",
    "\n",
    "groups are simply collections of users (and possibly other groups). from a security perspective, these exist to be just a *bit* lazy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "we create `group`s as a way of saying \"all these `user`s are basically the same in *this* way\", where \"*this* way\" is a statement about their permissions.\n",
    "\n",
    "we could go through the list of people in that group and update each person's permissions one at a time, but that would be very boring."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "in the real world, these are usually permissions assigned to people based on business logic, especially employment position, for example\n",
    "\n",
    "+ these people *are* data scientists\n",
    "+ these people *are* in our DC office\n",
    "+ these people *are* working on project X\n",
    "\n",
    "you create a group for that identity (*e.g.* \"data_scientists\") and give the *group* the needed permissions. this means that\n",
    "\n",
    "1. when you add a new data scientist to your company they get identical permissions to every other data scientist\n",
    "1. when you need to grant or revoke permissions for data scientists, you can do it in one fell swoop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "click now on [the `iam` groups dashboard](https://console.aws.amazon.com/iam/home?region=us-east-1#/groups) (\"groups\" in the left menu).\n",
    "\n",
    "this should bring up a table with your groups in them. click on one of your groups and you will land on a summary page for that group, which contains the following tabs:\n",
    "\n",
    "1. users tab\n",
    "    1. a list of the users in this group\n",
    "    2. add or remove any number of users from this group -- note: this is instantaneous\n",
    "2. permissions tab\n",
    "    1. a list of the policies that are attached to this group (no drop-down like with users)\n",
    "3. access advisor tab\n",
    "    1. same as above for users -- advanced dialog for designing access policies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**<div align=\"center\">mini exercise: look at your group's access advisor</div>**\n",
    "\n",
    "1. go to the `groups` dashboard for the `iam` service\n",
    "1. access your `admin` group (created in our first lecture when we set up billing)\n",
    "1. click on the \"Access Advisor\" tab\n",
    "1. search for `ec2` in the \"filter\" text box\n",
    "1. see the \"Last accessed\" section for information on when you last interacted with `ec2` using a group policy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## roles\n",
    "\n",
    "a role is similar to a group in that it is an abstract set of permissions that an `individual` could get. however, it is *different* than a `group`, in that it is something a `user` *becomes* and perhaps also *stops being*\n",
    "\n",
    "+ a `user` *is* in a `group` (persistent)\n",
    "+ a `user` can take on and then shed a `role` (temporary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "a metaphor: the position of `GU MAST professor` (a `role`) has certain privileges and responsibilities, but different people (individual `user`s) take turns stepping into that `role`.\n",
    "\n",
    "while I am in my role as a `GU MAST professor`, I can do some special things that I could not do outside of that role. for example, I am allowed to assign you homework assignments about AWS services.\n",
    "\n",
    "but I am not *always* in this role. if I started posting homework assignments next summer and demanded you complete them, it would not go well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "click now on [the `iam` roles dashboard](https://console.aws.amazon.com/iam/home?region=us-east-1#/roles) (\"Roles\" in the left menu).\n",
    "\n",
    "this should bring up a table with your roles listed there (you may have none). if you do, click on one of your roles and you will land on a summary page for that role, which contains the following tabs:\n",
    "\n",
    "1. permissions tab\n",
    "    1. a list of the policies that are attached to this group (no drop-down like with users)\n",
    "1. trust relationships tab\n",
    "    1. often organizations will have external user management systems (e.g. windows active directory). \"trust relationships\" are statements that what those external user systems say should be trusted\n",
    "1. access advisor tab\n",
    "    1. same as above for users -- advanced dialog for designing access policies\n",
    "1. revoke sessions\n",
    "    1. an emergency lever to restrict access to this role and its associated permissions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## policies\n",
    "\n",
    "a policy is a collection of conceptually related permissions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "in the `linux` world, permission is set on the file level with a set of `allow`/`deny` indicators about read, write, or execute permissions. those permissions are collected into a `mode` (`rwx`). so we already know of times where we have \"things\" and we give \"users\" specific \"permissions\" to them.\n",
    "\n",
    "in `aws`, everything is a `rest`ful api endpoint that is -- by default -- not accessible. we grant permissions to these endpoints to different `user`s, `group`s, or `role`s in a similar format."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "there's a useful generalization in `aws`: a `policy` is a big bundle of permission grants or restrictions.\n",
    "\n",
    "policies are generally given descriptive names like `AmazonS3ReadOnlyAccess`, and hold a definition of every permission needed to interact with a given \"service\" (a collection of `api`s for one generic purpose, like `ec2`).\n",
    "\n",
    "for example, `AmazonS3ReadOnlyAccess` will grant all the permissions required to read files on `s3` but no further."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "amazon cultivates many of these collections of permissions, and usually the ones you need are\n",
    "\n",
    "1. named in an obvious way, and\n",
    "2. added to the desired accounts as you set up the services"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "click now on [the `iam` policies dashboard](https://console.aws.amazon.com/iam/home?region=us-east-1#/policies) (\"Policies\" in the left menu).\n",
    "\n",
    "this should bring up a table with thousands of policies (mostly created by AWS and not you). click on any one of them and you will land on a summary page for that policy, which contains the following tabs:\n",
    "\n",
    "1. permissions tab\n",
    "    1. unlike the previous tabs, this permissions tab lists real *permissions* as a `json` string. see below for more\n",
    "2. policy usage\n",
    "    1. the `user`s, `group`s, or `role`s that have this policy attached to them\n",
    "3. policy versions\n",
    "    1. a versioned (numbered) history of the permission json statement\n",
    "4. access advisor tab\n",
    "    1. same as above -- advanced dialog for designing access policies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## permissions\n",
    "\n",
    "A permission is a codification of an access to or restriction from specific service. each permission consists of four pieces:\n",
    "\n",
    "1. an aws service (e.g. ec2, s3, rds)\n",
    "1. an access level (read, write, etc)\n",
    "1. a resource\n",
    "    1. if you have multiple resources (like hundreds of `s3` buckets) in your organization, this allows you indicate which instance(s) of those services this rule applies to\n",
    "1. request condition\n",
    "    1. a parameterizable prerequisite for exercising this permission"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "the tabular view on the \"policy summary\" button tab is a nice overview, but the real policy definition is the `json` blob on the second tab. for example, the `AmazonS3ReadOnlyAccess` policy has the following definition\n",
    "\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"Version\": \"2012-10-17\",\n",
    "    \"Statement\": [\n",
    "        {\n",
    "            \"Effect\": \"Allow\",\n",
    "            \"Action\": [\n",
    "                \"s3:Get*\",\n",
    "                \"s3:List*\"\n",
    "            ],\n",
    "            \"Resource\": \"*\"\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "```\n",
    "\n",
    "you can read this policy block as saying: \"whoever has this policy attached to it may execute any `Get` or `List` function on anything in `s3`\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**<div align=\"center\">mini exercise: look at the billing admin policy we created</div>**\n",
    "\n",
    "cast your mind wayyyyy back to lecture 1, in an exercise titled \"really awful billing permission hack that sucks but now is the best time to do it\"\n",
    "\n",
    "1. go to the `policies` dashboard for the `iam` service\n",
    "1. find the `billing_admin` policy we created (might have a different name, but this is what we recommended)\n",
    "1. look at the policy `json` on the \"permissions\" tab\n",
    "\n",
    "this policy should look roughly like\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"Version\": \"2012-10-17\",\n",
    "    \"Statement\": [\n",
    "        {\n",
    "            \"Sid\": \"Stmt1503972100000\",\n",
    "            \"Effect\": \"Allow\",\n",
    "            \"Action\": [\n",
    "                \"aws-portal:*\"\n",
    "            ],\n",
    "            \"Resource\": [\n",
    "                \"*\"\n",
    "            ]\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## putting it together\n",
    "\n",
    "let's create a user, assign that user into a group, and attach meaningful policies to that group"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**<div align=\"center\">mini exercise: let's create a `gu511` user</div>**\n",
    "\n",
    "1. go to the `users` dashboard for the `iam` service\n",
    "1. create a user (\"Add user\")\n",
    "    1. call this user `gu511`\n",
    "    1. give this user *programmatic access only*\n",
    "    1. create a group `ec2_s3_full_access`\n",
    "        1. find the `AmazonEC2FullAccess` policy and add it\n",
    "        1. find the `AmazonS3FullAccess` policy and add it\n",
    "    1. put `gu511` in this group"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## other\n",
    "\n",
    "the remaining `iam` tabs are all useful in certain contexts, but we won't dive into them right now.\n",
    "\n",
    "1. identity providers: allows you to \"trust\" outside resources for identity authentication\n",
    "2. account settings: general security settings\n",
    "3. access analyzer: create a monitor for certain access uses, and trigger responses\n",
    "4. credential report: full automated export of \"who\" can do \"what\"\n",
    "5. organization activity and service control policies: for checking all access activity of an external organization into your account, or from your account into someone else's (outside scope)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<strong><div align=\"center\">UNLEASH THE POWER</div></strong>\n",
    "<div align=\"center\"><img width=600 src=\"https://cdn.dribbble.com/users/125948/screenshots/3818579/lock-variation-dribbble.png\"></div>\n",
    "\n",
    "# END OF LECTURE\n",
    "\n",
    "next lecture: [AWS programmatic interfaces](009_cli_boto.ipynb)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}