This is fully Kubernetes-native testbed environment. Please contribute for add additional OSS (Vitess, NATS, etc) or microservices.
Note: This repository is still alpha release, and we focus on:
- BETA-1 at JUNE 2020 (Kubernetes Community Day Tokyo)
- BETA-2 at SEP 2020 (CloudNative Days Tokyo 2020, Kubernetes Forum Tokyo)
- GA at KubeCon + CNCon NA 2020
For Cloud Providers, we are looking for a sponsor to provide cloud environment. If you can help, please email us ([email protected]).
| microservice | datastore |
|---|---|
| cart | TiKV |
| comment | MongoDB |
| delivery-status | Cassandra, NATS |
| order | TiDB |
| payment-info | PostgreSQL |
| point | YugabyteDB, Kafka, Memcached(MCRouter) |
| product | MySQL |
| rate | Redis(Centinel) |
| search | Elasticsearch |
| user | MySQL |
| admin | - |
You can try to commit to your own repo for CI/CD.
| Name | Genre | Version | CNCF Project | URL |
|---|---|---|---|---|
| Vitess | Relational Database | v0.0.0 | ○ | planetscale/vitess-operator |
| MySQL | Relational Database | v0.0.0 | presslabs/mysql-operator | |
| MySQL | Relational Database | v0.0.0 | oracle/mysql-operator | |
| YugabyteDB | Relational Database | v0.0.0 | rook-yugabytedb | |
| PostgreSQL | Relational Database | v0.0.0 | zalando/postgres-operator | |
| TiDB | Relational Database | v0.0.0 | pingcap/tidb-operator | |
| TiKV | Key Value Store | v0.0.0 | ○ | pingcap/tidb-operator |
| Memcached | Key Value Store | v0.0.0 | geerlingguy/mcrouter-operator | |
| Redis | Key Value Store | v0.0.0 | spotahome/redis-operator | |
| Apache Cassandra | NoSQL (RDB) | v0.0.0 | Orange-OpenSource/casskop | |
| MongoDB | NoSQL (Document DB) | v0.0.0 | kubedb/operator | |
| NATS | Message Queue | v0.0.0 | ○ | nats-io/nats-operator |
| Apache Kafka | Message Queue | v0.0.0 | ○(*1) | strimzi/strimzi-kafka-operator |
| MinIO | Object Storage | v0.0.0 | minio/minio-operator | |
| Ceph | Block Storage | v0.0.0 | rook-ceph | |
| Rook | Block Storage | v0.0.0 | ○ | rook/rook |
| Nginx | Ingress Controller | v0.0.0 | kubernetes/ingress-nginx | |
| Envoy | L7 LoadBalancer | v0.0.0 | ○ | projectcontour/contour |
| Harbor | Container Registry | v0.0.0 | ○ | goharbor/harbor , goharbor/harbor-operator |
| Kaniko | Container Build Tool | v0.0.0 | GoogleContainerTools/kaniko | |
| Tekton Triggers | CI | v0.0.0 | tektoncd/triggers | |
| Tekton Pipelines | CI | v0.0.0 | tektoncd/pipeline | |
| ArgoCD | CD | v0.0.0 | argoproj/argo-cd | |
| Prometheus | Monitoring | v0.0.0 | ○ | coreos/prometheus-operator |
| Weave Scope | Monitoring | v0.0.0 | weaveworks/scope | |
| Kubernetes Dashboard | Monitoring | v2.0.0 | kubernetes/dashboard | |
| Loki | Logging | v0.0.0 | grafana/loki | |
| Telepresence | Local Development | v0.0.0 | ○ | telepresenceio/telepresence |
| Kustomize | Manifest Management | v0.0.0 | kubernetes-sigs/kustomize | |
| conftest | Manifest Management | v0.0.0 | instrumenta/conftest | |
| gRPC | Microservice Interconnect | v0.0.0 | ○ | grpc/grpc-go |
| gRPC-web | Microservice Interconnect | v0.0.0 | grpc/grpc-web | |
| Elasticsearch/Kibana | EFKStack | v0.0.0 | elastic/cloud-on-k8s | |
| Clair | Security | v0.0.0 | quay/clair | |
| (TBR): OPA Gatekeeper | Security | v0.0.0 | ○ | open-policy-agent/gatekeeper |
| (TBR): Knative | Serverless | v0.0.0 | knative/serving, knative/eventing |
*1: strictly speaking, strimzi is just a member of CNCF project, not Apache Kafka.
- kubectl ~1.17.x
- in kubectl 1.18.x~, we cannot use CRD and CR obj on one manifest file (kubernetes/kubectl#845)
- You need over 40 cores+ compute resources
from https://github.com/kubernetes-native-testbed/kubernetes-native-testbed
This testbed use your cloud settings, so please check your settings.
# at GCP
$ gcloud config list
[compute]
region = asia-northeast1
[core]
account = [email protected]
project = GCP_PROJECT
PLEASE CREATE A NEW DEDICATED GITHUB USER FOR ONLY THIS TESTBED REPO. NOW TOKEN IS WEAK
export TB_GITHUB_USER=XXXXX
export TB_GITHUB_TOKEN=XXXXX
Please install Krew which is kubectl plugins manager.
# for macOS / Linux
(
set -x; cd "$(mktemp -d)" &&
curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/krew.{tar.gz,yaml}" &&
tar zxvf krew.tar.gz &&
KREW=./krew-"$(uname | tr '[:upper:]' '[:lower:]')_amd64" &&
"$KREW" install --manifest=krew.yaml --archive=krew.tar.gz &&
"$KREW" update
)
export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"For other operation system, please refer to official installation document.
Following script do:
- Download client tools
- Allocate static IP address
- Replace placeholder in manifests on this repo, with above IP address
./tools/init.sh
Add webhook settings for forked repo
from https://github.com/__TB_GITHUB_ORG_NAME__/kubernetes-native-testbed/settings/hooks/new
* Payload URL: https://tekton.__LOADBALANCER_IP_ADDRESS__.nip.io/event-listener
* replace to your allocated address
* Content type: application/json
* Secret: sample-github-webhook-secret
* if you want to change, please edit manifests/infra/instances/ci.yaml
* Enable SSL verification: [*]
* Just the push event: [*]
* Active: [*]
Following script do:
- Create cluster
- Deploy applications to Kubernetes
./tools/start.sh
Following script do:
- Delete "Service" resource which use allocated IP address
- Destroy cluster
./tools/shutdown.sh
Following script do:
- Deallocate IP Address
./tools/finalize.sh
- Infrastructure
- Microservice
| Kind | Endpoint | Username | Password |
|---|---|---|---|
| Admin | https://testbed.__LOADBALANCER_IP_ADDRESS__.nip.io/admin/ | NO | NO |
- manifests/
- Kubernetes manifests
- infra/: system or infrastructure manifests
- cicd/: CI/CD pipeline settings
- ci-manifests/: tekton pipelines manifests
- cd-manifests/: argocd pipelines manifests
- microservices/
- microservice application source code
- development/
- development manifests (skaffold, kustomization patch file)
- tools/
- tools and scripts
# initializing
source ./development/initialize.sh
# start local development for admin
./development/local-development.sh admin
# access service with whole system or only local check
https://testbed.__LOADBALANCER_IP_ADDRESS__.nip.io/admin/index.html
or
http://localhost:8080/
- architecture and scheme memo


