Piraeus is a high performance (i.e., in Linux kernel), highly-available, simple, secure, and cloud agnostic data store for Kubernetes.
Using a Kubernetes cluster with at least 4 worker nodes is recommended. Due to CSI compatibility, kubelet version must be one of:
- v1.14.x
- v1.15.x
- v1.16.x
- v1.17.x
The hosts should use Docker as their container runtime and be running one of the following distributions to enable automatic DRBD kernel module injection:
- CentOS/RHEL 7
- CentOS/RHEL 8
- Ubuntu 16
- Ubuntu 18
The 3 nodes on which piraeus runs etcd clusters should be labelled as follows:
kubectl label nodes $NODE_NAME piraeus/etcd=true
The nodes on which piraeus should provide or consume storage should be labelled as follows:
kubectl label nodes $NODE_NAME piraeus/node=true
These nodes must also have the appropriate kernel development package installed unless DRBD is already present.
This is kernel-devel for CentOS based distributions and linux-headers-`uname -r` for Ubuntu.
Install as follows:
kubectl apply -f https://github.com/raw/piraeusdatastore/piraeus/master/deploy/all.yaml
This may take several minutes. You may observe the pods by command:
kubectl -n kube-system get pod -l app.kubernetes.io/name=piraeus
Once the pods have started, the status of Piraeus can be checked by following commands.
On each Kubernetes work node where piraeus is deployed:
/opt/piraeus/client/linstor node list
Also on Kuberntes master nodes:
kubectl -n piraeus-system exec -it piraeus-controller-0 -- linstor node list
This should show that the selected nodes are Online at the LINSTOR level.
all.yaml is only for installation.
For upgrade, please use upgrade.yaml. It skips etcd upgrade (which is dangerous), and storageclass upgrade (which is immutable).
kubectl apply -f https://github.com/raw/piraeusdatastore/piraeus/master/deploy/upgrade.yaml
Piraeus preconfigures a DfltStorPool by using LINSTOR's FileThin backend, which is ready to use after yaml deployment.
The demo directory contains examples of how to use DfltStorPool.
For instance:
kubectl apply -f https://github.com/raw/piraeusdatastore/piraeus/master/demo/demo-sts.yaml
This demo statefulset is a 3-node MySQL cluster. Demo pods and pvcs are under piraeus namespace.
Piraeus can use storage that is local to the application as well as storage on other nodes. On the nodes that should provide storage, backing devices must be available. Assuming the hosts have empty storage devices of at least 1GB capacity, they can be listed as follows:
linstor physical-storage list
Piraeus can then configure LVM on these devices and create a storage pool. Use the following steps for each node:
linstor physical-storage create-device-pool --pool-name pool0 LVM $NODE_NAME /dev/$DEVICE
linstor storage-pool create lvm $NODE_NAME DfltStorPool pool0
Piraeus consists of a number of open source components.
DRBD is used as the underlying storage replication mechanism. Documentation is provided by LINBIT.
LINSTOR is used for storage management. Documentation is provided by LINBIT.
The LINSTOR CSI plugin integrates LINSTOR with the Container Storage Interface. Documentation is included with the LINSTOR documentation.
This directory contains Dockerfiles and shell scripts for Piraeus components. Each of the subdirectories has
a README.md with more information.
- drbd-driver-loader contains is a collection of Piraeus container images containing the DRBD kernel source code that can be used to compile DRBD kernel modules from source and load them into the host kernel.
- piraeus-server combines LINBIT's linstor-controller and linstor-satellite components, as well as the linstor-client which can be used for cluster setup. It also contains some useful debugging tools.
- piraeus-client is a stand alone version of the linstor-client.