Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_eks: Cluster Name generation can cause inability to create BottleRocket clusters #32299

Open
1 task
Gum-Christopher-bah opened this issue Nov 26, 2024 · 1 comment
Labels
@aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@Gum-Christopher-bah
Copy link

Describe the bug

When supplying a cluster name that is not Kubernetes label compliant, bottlerocket nodes are unable to join the cluster and the cdk deployment fails.

Regression Issue

  • Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

Cluster can properly handle bottlerocket nodes

Current Behavior

Error from Ec2 system logs:

Unable to deserialize into KubernetesClusterName: Given invalid cluster name 'eksclusterEksClusterEastBr5useast1clusterE0E4572F-c36ad2b5b0494751969f4e15a4c91a6d': cluster names must be valid Kubernetes label

Reproduction Steps

Create a cluster with either no name or a name like the one provided above (which was cdk generated)

const clusterName = eksclusterEksClusterEastBr5useast1clusterE0E4572F-c36ad2b5b0494751969f4e15a4c91a6d;
const cluster = new eks.Cluster(this, 'hello-eks', {
  version: eks.KubernetesVersion.V1_31,
  kubectlLayer: new KubectlV31Layer(this, 'kubectl'),
  defaultCapacity: 0,
  clusterName: clusterName,
});

Create user data for bottlerocket and add it to a launch template, add nodegroup capacity

const userData = ec2.UserData.forLinux();
userData.addCommands(`[settings.kubernetes]
api-server = "${this.cluster.clusterEndpoint}"
cluster-certificate = "${this.cluster.clusterCertificateAuthorityData}"
cluster-name = "${clusterName}"
[settings.host-containers.admin]
enabled = ${bottleRocketAdminContainerEnabled}`,
      );
      
    launchTemplate = new ec2.LaunchTemplate(this, `${Stack.of(this).stackName}-${Stack.of(this).region}-launch-template-br`, {
        blockDevices: [{
          deviceName: '/dev/xvda',
          volume: autoscaling.BlockDeviceVolume.ebs(
            50,
            {
              volumeType: autoscaling.EbsDeviceVolumeType.GP3,
            },
          ),
        }],
        machineImage: ec2.MachineImage.fromSsmParameter('/aws/service/bottlerocket/aws-k8s-1.31/x86_64/latest/image_id'),
        instanceType: 't3.medium',
        userData: userData,
      });
      Tags.of(launchTemplate).add('eks:cluster-name', clusterName);
      Tags.of(launchTemplate).add('eks:nodegroup-name', nodeGroupName);
      cluster.addNodegroupCapacity(`${Stack.of(this).stackName}-${Stack.of(this).region}-ng`, nodeGroupOptions);

In theory, you could skip all the launch template stuff and just try to pass the bottlerocket ami type to the nodegroup via options directly, this is just how I did it so I could set userData settings.

Possible Solution

generate cluster names in all lowercase, potentially truncate for very large cluster names

Additional Information/Context

No response

CDK CLI Version

2.170.0

Framework Version

No response

Node.js Version

18.20.4

OS

Mac

Language

TypeScript

Language Version

No response

Other information

No response

@Gum-Christopher-bah Gum-Christopher-bah added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Nov 26, 2024
@github-actions github-actions bot added the @aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service label Nov 26, 2024
@Gum-Christopher-bah
Copy link
Author

The workaround is to supply a k8s label compliant cluster name, but this is not apparent at all without troubleshooting failed deployments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
Development

No branches or pull requests

1 participant