A set of tools to deploy and operate a multi-tenant etcd datastore for Kamaji control-plane.

Kamaji turns any Kubernetes cluster into an “admin cluster” to orchestrate other Kubernetes clusters called “tenant clusters”. The Control Plane of a “tenant cluster” is made of regular pods running in a namespace of the “admin cluster” instead of a dedicated set of Virtual Machines. This solution makes running control planes at scale cheaper and easier to deploy and operate.

As of any Kubernetes cluster, a “tenant cluster” needs a datastore where to save the state and be able to retrieve data. Kamaji provides multiple options: a multi-tenant etcd as well as MySQL, and PostgreSQL, thanks to the kine integration.

A multi-tenant deployment for etcd is not common practice. However, etcd provides simple and robust APIs for creating users and setting up role based access control (RBAC) policies to define which user have access to what key prefix.

Refer to the etcd documentation. Following sections provide additional procedures to help with a specific setup as it is used into project Kamaji.

• Recovery from a snapshot
• Backup and Restore with Velero
• Rotate Certificates
• Performance and Optimization

• Install High Available etcd cluster as StatefulSet
• Provide data persistence through Persistent Volumes
• Multi-tenancy
• Autocompaction
• Scheduled defragmentation
• Auto generate certificates
• Scheduled snapshots
• Metrics Service Monitors
• Alert rules
• Grafana dashboard
• Benchmarking

On the Kamaji's “admin cluster”, install the multi-tenant etcd with the provided Helm Chart:

helm repo add clastix https://clastix.github.io/charts
helm install kamaji-etcd clastix/kamaji-etcd -n kamaji-etcd --create-namespace

The certificates of etcd, are stored as secrets into the same namespace:

• <release_name>-certs contains CA, peers, and server certificates
• <release_name>-root-client-certs contains the user root certificates

Make sure the Kamaji controller can access these secrets in their namespaces.