Description
What happened: Tring to deploy EKS anywhere cluster on bare metal, using created with the image-builder ubuntu image. Cluster hangs on "Creating new workload cluster". Thinkerbell boot finishes successfully. OS is installed and the cluster is created on the bare metal machine.
On the admin machine:
k logs -n capi-kubeadm-control-plane-system capi-kubeadm-control-plane-controller-manager-xxxxxxxxx
gives a bunch of:
I0720 13:44:24.746087 1 controller.go:221] "Could not connect to workload cluster to fetch status" controller="kubeadmcontrolplane" controllerGroup="controlplane.cluster.x-k8s.io" controllerKind="KubeadmControlPlane" KubeadmControlPlane="eksa-system/mgmt-ubuntu" namespace="eksa-system" name="mgmt-ubuntu" reconcileID="deba6704-27d2-4852-b235-05f27327ea3b" Cluster="eksa-system/mgmt-ubuntu" err="failed to create remote cluster client: eksa-system/mgmt-ubuntu: failed to create cluster accessor: error creating client for remote cluster "eksa-system/mgmt-ubuntu": error getting rest mapping: failed to get API group resources: unable to retrieve the complete list of server APIs: v1: Get "https://192.168.1.251:6443/api/v1?timeout=10s\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
I0720 13:44:24.759627 1 controller.go:429] "Reconcile KubeadmControlPlane" controller="kubeadmcontrolplane" controllerGroup="controlplane.cluster.x-k8s.io" controllerKind="KubeadmControlPlane" KubeadmControlPlane="eksa-system/mgmt-ubuntu" namespace="eksa-system" name="mgmt-ubuntu" reconcileID="5a79440a-2ce0-4bd8-a1e5-ac3365901e32" Cluster="eksa-system/mgmt-ubuntu"
k describe pod -n capi-kubeadm-control-plane-system capi-kubeadm-control-plane-controller-manager:
Warning FailedMount 25m (x4 over 25m) kubelet MountVolume.SetUp failed for volume "cert" : secret "capi-kubeadm-control-plane-webhook-service-cert" not found
What you expected to happen: Successfully deploy EKS Anywhere cluster
How to reproduce it (as minimally and precisely as possible):
image-builder: 0.4.0
Node image build command: image-builder build --os ubuntu --hypervisor baremetal --release-channel 1-28
eksctl anywhere version: 0.19.7
hardware: attached
cluster: attached
eksctl create command:
eksctl anywhere create cluster
--hardware-csv hardware.csv --per-machine-wait-timeout 30m
-f eksa-mgmt-cluster-ubuntu.yaml
eksa-mgmt-cluster-ubuntu.yaml:
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: mgmt
spec:
clusterNetwork:
cniConfig:
cilium: {}
pods:
cidrBlocks:
- 10.112.0.0/16
services:
cidrBlocks:
- 10.96.0.0/12
controlPlaneConfiguration:
count: 1
endpoint:
host: "192.168.1.253"
machineGroupRef:
kind: TinkerbellMachineConfig
name: mgmt-cp
datacenterRef:
kind: TinkerbellDatacenterConfig
name: mgmt
kubernetesVersion: "1.28"
managementCluster:
name: mgmt
---
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: TinkerbellDatacenterConfig
metadata:
name: mgmt
spec:
osImageURL: "http://192.168.1.106:8000/ubuntu-2204-kube-1-28-eks-a-19-7.gz"
hookImagesURLPath: "http://192.168.1.106:8000/hook"
tinkerbellIP: "192.168.1.252"
---
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: TinkerbellMachineConfig
metadata:
name: mgmt-cp
spec:
hardwareSelector:
type: "cp"
osFamily: ubuntu
osImageURL: ""
hookImagesURLPath: ""
templateRef: {}
users:
- name: ubuntu
sshAuthorizedKeys:
- ssh-rsa AAAAB.....
---
Anything else we need to know?:
Environment: 1.28
hardware.csv
- EKS Anywhere Release: 0.19.7
- EKS Distro Release: 1.28