From 70647d7d9ea4ae8aee2ea35679e1d946d503072f Mon Sep 17 00:00:00 2001
From: RaviKhandavilli <113392740+RaviKhandavilli@users.noreply.github.com>
Date: Fri, 2 Dec 2022 04:19:59 +0530
Subject: [PATCH] Installation steps for ec2 instance (#280)

Updated Readme with instructions to set up minikube in ec2 instance and steps to enable GPU on minikube
segregated distribution wise instructions

Co-authored-by: rakhanda <rakhanda@rakhandas-MacBook-Pro.local>
---
 docs/03-fiab.md         | 102 +++-------------------------------------
 docs/03.A-fiab-linux.md |  51 ++++++++++++++++++++
 docs/03.B-fiab-ec2.md   |  91 +++++++++++++++++++++++++++++++++++
 docs/03.C-fiab-mac.md   |  42 +++++++++++++++++
 4 files changed, 190 insertions(+), 96 deletions(-)
 create mode 100644 docs/03.A-fiab-linux.md
 create mode 100644 docs/03.B-fiab-ec2.md
 create mode 100644 docs/03.C-fiab-mac.md

diff --git a/docs/03-fiab.md b/docs/03-fiab.md
index d48239b8f..581f56601 100644
--- a/docs/03-fiab.md
+++ b/docs/03-fiab.md
@@ -14,101 +14,10 @@ The `flame/fiab` folder contains several scripts to configure and set up the fia
 Thus, the working directory for this guideline is `flame/fiab`.
 
 ## Prerequisites
-
-### MacOS
-fiab relies on `minikube`, `kubectl`, `helm`, `docker` and `jq`.
-
-
-fiab doesn't support docker driver (hence, docker desktop). fiab uses ingress and ingress-dns addons in minikube.
-When docker driver is chosen, these two addons are only supported on Linux (see [here](https://minikube.sigs.k8s.io/docs/drivers/docker/)
-and [here](https://github.com/kubernetes/minikube/issues/7332)). Note that while the issue 7332 is now closed and appears to be fixed,
-ingress and ingress-dns still don't work under fiab environment on MacOs.
-In addition, note that the docker subscription service agreement has been updated for `docker desktop`.
-Hence, `docker desktop` may not be free. Please check out the [agreement](https://www.docker.com/products/docker-desktop).
-
-Hence, fiab uses `hyperkit` as its default vm driver. Using `hyperkit` has some drawbacks.
-First, as of May 21, 2022, `hyperkit` driver doesn't support M1 chipset.
-Second, the `hyperkit` driver doesn't work with dnscrypt-proxy or dnsmasq well.
-Thus, if dnscrypt-proxy or dnsmasq is installed in the system, see [here](#fixing-docker-build-error) for details and a workaround.
-
-Note that other drivers such as `virtualbox` are not tested.
-
-#### Step 1: Installing VM driver for minikube
-To install `hyperkit`,
-```bash
-brew install hyperkit docker
-```
-Here we need to install `docker` to make the docker cli tool available.
-
-#### Step 2: Installing minikube and its related tools
-Now we install `minikube`, `kubectl` and `helm` as follows.
-```bash
-brew install minikube kubectl helm
-```
-
-#### Step 3: Miscellaneous tools
-`flame.sh` script uses `jq` to parse and update coredns's configmap in json format. Run the following to install the tool:
-```bash
-brew install jq
-```
-
-`robo-3t` is a GUI tool for MongoDB. This tool comes in handy when debugging mongodb-related issues in the flame system.
-This tool is optional.
-```bash
-brew install --cask robo-3t
-```
-
-### Linux
-For linux, no VM hypervisor is needed. The following tools are sufficient: `minikube`, `kubectl`, `helm`, `docker` and `jq`.
-The fiab env was tested under Archlinux in a x86 machine.
-
-#### step 1: Installing minikube
-We install the latest minikube stable release on x86-64 Linux using binary downloaded from [here](https://minikube.sigs.k8s.io/docs/start/).
-
-#### step 2: Installing kubectl, helm and jq
-To install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-using-other-package-management).
-```bash
-sudo snap install kubectl --classic
-kubectl version --client
-```
-To install [helm](https://helm.sh/docs/intro/install/).
-```bash
-sudo snap install helm --classic
-```
-To install jq.
-```bash
-sudo apt update
-sudo apt install -y jq
-```
-#### step3: Install docker
-1. Update the apt package index and install packages to allow apt to use a repository over HTTPS.                                                                       
-```bash
-sudo apt-get update
-sudo apt-get install \
-    ca-certificates \
-    curl \
-    gnupg \
-    lsb-release
-```
-2. Add Docker’s official GPG key.
-```bash
-sudo mkdir -p /etc/apt/keyrings
-curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
-```
-3. Use the following command to set up the repository.
-```bash
-echo \
-  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
-  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
-```
-4. Update the apt package index, and install the latest version of Docker Engine.
-```bash
-sudo apt-get update
-sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
-sudo groupadd docker
-sudo usermod -aG docker $USER && newgrp docker 
-```
-
+Install prerequisites for a given distribution as per documents provided below
+* [Linux](03.A-fiab-linux.md)
+* [Ec2](03.B-fiab-ec2.md)
+* [Mac](03.C-fiab-mac.md)
 
 ## Starting minikube
 Run the following command to start minikube.
@@ -117,7 +26,7 @@ minikube start
 ```
 The default resource allocation is 2 CPU, 4GB memory and 20GB disk.
 In order to change these parameters, use `--cpus`, `--memory` and `--disk-size` respectively.
-For example, 
+For example,
 ```bash
 minikube start --cpus 4 --memory 4096m --disk-size 100gb
 ```
@@ -140,6 +49,7 @@ selfsigned certificate creation. Run the following command:
 ./setup-cert-manager.sh
 ```
 
+
 ## Building flame
 A Docker daemon comes within the minikube VM. To build flame container image, set the environment variables with the following command.
 
diff --git a/docs/03.A-fiab-linux.md b/docs/03.A-fiab-linux.md
new file mode 100644
index 000000000..de67239e6
--- /dev/null
+++ b/docs/03.A-fiab-linux.md
@@ -0,0 +1,51 @@
+
+### Linux Prerequisites
+For linux, no VM hypervisor is needed. The following tools are sufficient: `minikube`, `kubectl`, `helm`, `docker` and `jq`.
+The fiab env was tested under Archlinux in a x86 machine.
+
+#### step 1: Installing minikube
+We install the latest minikube stable release on x86-64 Linux using binary downloaded from [here](https://minikube.sigs.k8s.io/docs/start/).
+
+#### step 2: Installing kubectl, helm and jq
+To install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-using-other-package-management).
+```bash
+sudo snap install kubectl --classic
+kubectl version --client
+```
+To install [helm](https://helm.sh/docs/intro/install/).
+```bash
+sudo snap install helm --classic
+```
+To install jq.
+```bash
+sudo apt update
+sudo apt install -y jq
+```
+#### step3: Install docker
+1. Update the apt package index and install packages to allow apt to use a repository over  HTTPS.                                             
+```bash
+sudo apt-get update
+sudo apt-get install \
+    ca-certificates \
+    curl \
+    gnupg \
+    lsb-release
+```
+2. Add Docker’s official GPG key.
+```bash
+sudo mkdir -p /etc/apt/keyrings
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
+```
+3. Use the following command to set up the repository.
+```bash
+echo \
+  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
+  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+```
+4. Update the apt package index, and install the latest version of Docker Engine.
+```bash
+sudo apt-get update
+sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
+sudo groupadd docker
+sudo usermod -aG docker $USER && newgrp docker
+```
diff --git a/docs/03.B-fiab-ec2.md b/docs/03.B-fiab-ec2.md
new file mode 100644
index 000000000..76c9635ff
--- /dev/null
+++ b/docs/03.B-fiab-ec2.md
@@ -0,0 +1,91 @@
+### EC2 instance Prerequisites
+For Amazon linux, no VM hypervisor is needed. The following tools are sufficient: `minikube`, `kubectl`, `helm`, `cri-dockerd`, `crictl` , `docker` and `jq`.
+The fiab env was tested under amzn2 in a x86 machine.
+
+#### step 1: Install Docker CRI
+To install cri-dockerd
+1. Download cri-dockerd source code
+``` bash
+sudo -i 
+git clone https://github.com/Mirantis/cri-dockerd.git 
+```
+
+2. Install Golang compiler
+```
+wget https://storage.googleapis.com/golang/getgo/installer_linux 
+chmod +x ./installer_linux 
+./installer_linux 
+source ~/.bash_profile 
+ ```
+
+ 3. Install and start cri-dockerd
+ ```
+cd cri-dockerd 
+mkdir bin 
+go build -o bin/cri-dockerd 
+install -o root -g root -m 0755 bin/cri-dockerd /usr/bin/cri-dockerd 
+cp -a packaging/systemd/* /etc/systemd/system 
+systemctl daemon-reload 
+systemctl enable cri-docker.service 
+systemctl enable --now cri-docker.socket 
+exit
+```
+#### step 2: Install docker
+Install docker as per [this](https://docs.docker.com/engine/install/) document
+
+#### step 3:Install crictl
+1. Download crictl tar file and install
+```
+VERSION="v1.25.0" 
+wget https://github.com/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-$VERSION-linux-amd64.tar.gz 
+sudo tar zxvf crictl-$VERSION-linux-amd64.tar.gz -C /usr/local/bin 
+rm -f crictl-$VERSION-linux-amd64.tar.gz 
+```
+
+#### step 4: Installing minikube
+1. Install minukube
+
+```
+curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-latest.x86_64.rpm 
+sudo rpm -Uvh minikube-latest.x86_64.rpm 
+```
+
+Note: If `Exiting due to HOST_JUJU_LOCK_PERMISSION` error happens run below command:
+
+```
+sudo sysctl fs.protected_regular=0 
+```
+
+2. Start Minikube
+```
+sudo minikube start --driver=none --apiserver-ips 127.0.0.1 --apiserver-name localhost --cni=bridge
+```
+
+#### step 5: Install kubectl
+1. Run below commands to install kubectl command
+```
+curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" 
+sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl 
+```
+
+#### step 6 (Optional): Install NVIDIA'S device plugin
+1. If there are NVIDIA's GPU's available in the machine, inorder to enable them deploy below yml
+```
+sudo kubectl create -f https://github.com/raw/NVIDIA/k8s-device-plugin/master/nvidia-device-plugin.yml 
+```
+2. To check if GPUs are enabled run below command
+```
+sudo kubectl get nodes -ojson | jq .items[].status.capacity 
+```
+Output should look similar to:
+```
+{ 
+  "cpu": "4", 
+  "ephemeral-storage": "524275692Ki", 
+  "hugepages-1Gi": "0", 
+  "hugepages-2Mi": "0", 
+  "memory": "62766704Ki", 
+  "nvidia.com/gpu": "1", 
+  "pods": "110" 
+} 
+```
diff --git a/docs/03.C-fiab-mac.md b/docs/03.C-fiab-mac.md
new file mode 100644
index 000000000..f78250a09
--- /dev/null
+++ b/docs/03.C-fiab-mac.md
@@ -0,0 +1,42 @@
+### MacOS Prerequisites
+fiab relies on `minikube`, `kubectl`, `helm`, `docker` and `jq`.
+
+
+fiab doesn't support docker driver (hence, docker desktop). fiab uses ingress and ingress-dns addons in minikube.
+When docker driver is chosen, these two addons are only supported on Linux (see [here](https://minikube.sigs.k8s.io/docs/drivers/docker/)
+and [here](https://github.com/kubernetes/minikube/issues/7332)). Note that while the issue 7332 is now closed and appears to be fixed,
+ingress and ingress-dns still don't work under fiab environment on MacOs.
+In addition, note that the docker subscription service agreement has been updated for `docker desktop`.
+Hence, `docker desktop` may not be free. Please check out the [agreement](https://www.docker.com/products/docker-desktop).
+
+Hence, fiab uses `hyperkit` as its default vm driver. Using `hyperkit` has some drawbacks.
+First, as of May 21, 2022, `hyperkit` driver doesn't support M1 chipset.
+Second, the `hyperkit` driver doesn't work with dnscrypt-proxy or dnsmasq well.
+Thus, if dnscrypt-proxy or dnsmasq is installed in the system, see [here](#fixing-docker-build-error) for details and a workaround.
+
+Note that other drivers such as `virtualbox` are not tested.
+
+#### Step 1: Installing VM driver for minikube
+To install `hyperkit`,
+```bash
+brew install hyperkit docker
+```
+Here we need to install `docker` to make the docker cli tool available.
+
+#### Step 2: Installing minikube and its related tools
+Now we install `minikube`, `kubectl` and `helm` as follows.
+```bash
+brew install minikube kubectl helm
+```
+
+#### Step 3: Miscellaneous tools
+`flame.sh` script uses `jq` to parse and update coredns's configmap in json format. Run the following to install the tool:
+```bash
+brew install jq
+```
+
+`robo-3t` is a GUI tool for MongoDB. This tool comes in handy when debugging mongodb-related issues in the flame system.
+This tool is optional.
+```bash
+brew install --cask robo-3t
+```