-
Notifications
You must be signed in to change notification settings - Fork 27
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Installation steps for ec2 instance (#280)
Updated Readme with instructions to set up minikube in ec2 instance and steps to enable GPU on minikube segregated distribution wise instructions Co-authored-by: rakhanda <rakhanda@rakhandas-MacBook-Pro.local>
- Loading branch information
1 parent
eb0dbc4
commit 70647d7
Showing
4 changed files
with
190 additions
and
96 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
|
|
||
| ### Linux Prerequisites | ||
| For linux, no VM hypervisor is needed. The following tools are sufficient: `minikube`, `kubectl`, `helm`, `docker` and `jq`. | ||
| The fiab env was tested under Archlinux in a x86 machine. | ||
|
|
||
| #### step 1: Installing minikube | ||
| We install the latest minikube stable release on x86-64 Linux using binary downloaded from [here](https://minikube.sigs.k8s.io/docs/start/). | ||
|
|
||
| #### step 2: Installing kubectl, helm and jq | ||
| To install [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-using-other-package-management). | ||
| ```bash | ||
| sudo snap install kubectl --classic | ||
| kubectl version --client | ||
| ``` | ||
| To install [helm](https://helm.sh/docs/intro/install/). | ||
| ```bash | ||
| sudo snap install helm --classic | ||
| ``` | ||
| To install jq. | ||
| ```bash | ||
| sudo apt update | ||
| sudo apt install -y jq | ||
| ``` | ||
| #### step3: Install docker | ||
| 1. Update the apt package index and install packages to allow apt to use a repository over HTTPS. | ||
| ```bash | ||
| sudo apt-get update | ||
| sudo apt-get install \ | ||
| ca-certificates \ | ||
| curl \ | ||
| gnupg \ | ||
| lsb-release | ||
| ``` | ||
| 2. Add Docker’s official GPG key. | ||
| ```bash | ||
| sudo mkdir -p /etc/apt/keyrings | ||
| curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg | ||
| ``` | ||
| 3. Use the following command to set up the repository. | ||
| ```bash | ||
| echo \ | ||
| "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ | ||
| $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null | ||
| ``` | ||
| 4. Update the apt package index, and install the latest version of Docker Engine. | ||
| ```bash | ||
| sudo apt-get update | ||
| sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin | ||
| sudo groupadd docker | ||
| sudo usermod -aG docker $USER && newgrp docker | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| ### EC2 instance Prerequisites | ||
| For Amazon linux, no VM hypervisor is needed. The following tools are sufficient: `minikube`, `kubectl`, `helm`, `cri-dockerd`, `crictl` , `docker` and `jq`. | ||
| The fiab env was tested under amzn2 in a x86 machine. | ||
|
|
||
| #### step 1: Install Docker CRI | ||
| To install cri-dockerd | ||
| 1. Download cri-dockerd source code | ||
| ``` bash | ||
| sudo -i | ||
| git clone https://github.com/Mirantis/cri-dockerd.git | ||
| ``` | ||
|
|
||
| 2. Install Golang compiler | ||
| ``` | ||
| wget https://storage.googleapis.com/golang/getgo/installer_linux | ||
| chmod +x ./installer_linux | ||
| ./installer_linux | ||
| source ~/.bash_profile | ||
| ``` | ||
|
|
||
| 3. Install and start cri-dockerd | ||
| ``` | ||
| cd cri-dockerd | ||
| mkdir bin | ||
| go build -o bin/cri-dockerd | ||
| install -o root -g root -m 0755 bin/cri-dockerd /usr/bin/cri-dockerd | ||
| cp -a packaging/systemd/* /etc/systemd/system | ||
| systemctl daemon-reload | ||
| systemctl enable cri-docker.service | ||
| systemctl enable --now cri-docker.socket | ||
| exit | ||
| ``` | ||
| #### step 2: Install docker | ||
| Install docker as per [this](https://docs.docker.com/engine/install/) document | ||
|
|
||
| #### step 3:Install crictl | ||
| 1. Download crictl tar file and install | ||
| ``` | ||
| VERSION="v1.25.0" | ||
| wget https://github.com/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-$VERSION-linux-amd64.tar.gz | ||
| sudo tar zxvf crictl-$VERSION-linux-amd64.tar.gz -C /usr/local/bin | ||
| rm -f crictl-$VERSION-linux-amd64.tar.gz | ||
| ``` | ||
|
|
||
| #### step 4: Installing minikube | ||
| 1. Install minukube | ||
|
|
||
| ``` | ||
| curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-latest.x86_64.rpm | ||
| sudo rpm -Uvh minikube-latest.x86_64.rpm | ||
| ``` | ||
|
|
||
| Note: If `Exiting due to HOST_JUJU_LOCK_PERMISSION` error happens run below command: | ||
|
|
||
| ``` | ||
| sudo sysctl fs.protected_regular=0 | ||
| ``` | ||
|
|
||
| 2. Start Minikube | ||
| ``` | ||
| sudo minikube start --driver=none --apiserver-ips 127.0.0.1 --apiserver-name localhost --cni=bridge | ||
| ``` | ||
|
|
||
| #### step 5: Install kubectl | ||
| 1. Run below commands to install kubectl command | ||
| ``` | ||
| curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" | ||
| sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl | ||
| ``` | ||
|
|
||
| #### step 6 (Optional): Install NVIDIA'S device plugin | ||
| 1. If there are NVIDIA's GPU's available in the machine, inorder to enable them deploy below yml | ||
| ``` | ||
| sudo kubectl create -f https://github.com/raw/NVIDIA/k8s-device-plugin/master/nvidia-device-plugin.yml | ||
| ``` | ||
| 2. To check if GPUs are enabled run below command | ||
| ``` | ||
| sudo kubectl get nodes -ojson | jq .items[].status.capacity | ||
| ``` | ||
| Output should look similar to: | ||
| ``` | ||
| { | ||
| "cpu": "4", | ||
| "ephemeral-storage": "524275692Ki", | ||
| "hugepages-1Gi": "0", | ||
| "hugepages-2Mi": "0", | ||
| "memory": "62766704Ki", | ||
| "nvidia.com/gpu": "1", | ||
| "pods": "110" | ||
| } | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| ### MacOS Prerequisites | ||
| fiab relies on `minikube`, `kubectl`, `helm`, `docker` and `jq`. | ||
|
|
||
|
|
||
| fiab doesn't support docker driver (hence, docker desktop). fiab uses ingress and ingress-dns addons in minikube. | ||
| When docker driver is chosen, these two addons are only supported on Linux (see [here](https://minikube.sigs.k8s.io/docs/drivers/docker/) | ||
| and [here](https://github.com/kubernetes/minikube/issues/7332)). Note that while the issue 7332 is now closed and appears to be fixed, | ||
| ingress and ingress-dns still don't work under fiab environment on MacOs. | ||
| In addition, note that the docker subscription service agreement has been updated for `docker desktop`. | ||
| Hence, `docker desktop` may not be free. Please check out the [agreement](https://www.docker.com/products/docker-desktop). | ||
|
|
||
| Hence, fiab uses `hyperkit` as its default vm driver. Using `hyperkit` has some drawbacks. | ||
| First, as of May 21, 2022, `hyperkit` driver doesn't support M1 chipset. | ||
| Second, the `hyperkit` driver doesn't work with dnscrypt-proxy or dnsmasq well. | ||
| Thus, if dnscrypt-proxy or dnsmasq is installed in the system, see [here](#fixing-docker-build-error) for details and a workaround. | ||
|
|
||
| Note that other drivers such as `virtualbox` are not tested. | ||
|
|
||
| #### Step 1: Installing VM driver for minikube | ||
| To install `hyperkit`, | ||
| ```bash | ||
| brew install hyperkit docker | ||
| ``` | ||
| Here we need to install `docker` to make the docker cli tool available. | ||
|
|
||
| #### Step 2: Installing minikube and its related tools | ||
| Now we install `minikube`, `kubectl` and `helm` as follows. | ||
| ```bash | ||
| brew install minikube kubectl helm | ||
| ``` | ||
|
|
||
| #### Step 3: Miscellaneous tools | ||
| `flame.sh` script uses `jq` to parse and update coredns's configmap in json format. Run the following to install the tool: | ||
| ```bash | ||
| brew install jq | ||
| ``` | ||
|
|
||
| `robo-3t` is a GUI tool for MongoDB. This tool comes in handy when debugging mongodb-related issues in the flame system. | ||
| This tool is optional. | ||
| ```bash | ||
| brew install --cask robo-3t | ||
| ``` |