Skip to content

bug: EKS cluster failes to start with "K3D cluster: Port https://10.225.18.120:43589 (path: /) was not open" #13497

@DW8Reaper

Description

@DW8Reaper

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When I create an EKS cluster in LocalStack and I am connected to our corporate VPN then it attempts to use my VPN ip address when verifying the cluster is up and ultimately fails since my VPN adapter will not allow additional services to be hosted. In the debug logs I get an error like this (NOTE: our VPN address range is 10.225.0.0/16 ):

2025-12-10T17:30:20.201 ERROR --- [-lifecycle_0] l.p.c.services.eks.cluster : Error starting K3D cluster: Port https://10.225.18.120:40081 (path: /) was not open
Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.13/site-packages/localstack/pro/core/services/eks/cluster.py.enc", line 168, in start
    try:A._do_start_cluster();A.status=EmulatedClusterStatus.ACTIVE
        ~~~~~~~~~~~~~~~~~~~^^
  File "/opt/code/localstack/.venv/lib/python3.13/site-packages/localstack/pro/core/services/eks/cluster.py.enc", line 150, in _do_start_cluster
    try:wait_for_port_open(A.internal_endpoint,http_path='/',expect_success=_B,sleep_time=1,retries=A._leftover_time(E))
        ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/code/localstack/.venv/lib/python3.13/site-packages/localstack/utils/net.py", line 131, in wait_for_port_open
    return wait_for_port_status(
        port,
    ...<3 lines>...
        sleep_time=sleep_time,
    )
  File "/opt/code/localstack/.venv/lib/python3.13/site-packages/localstack/utils/net.py", line 172, in wait_for_port_status
    return retry(check, sleep=sleep_time, retries=retries)
  File "/opt/code/localstack/.venv/lib/python3.13/site-packages/localstack/utils/sync.py", line 64, in retry
    raise raise_error
  File "/opt/code/localstack/.venv/lib/python3.13/site-packages/localstack/utils/sync.py", line 60, in retry
    return function(**kwargs)
  File "/opt/code/localstack/.venv/lib/python3.13/site-packages/localstack/utils/net.py", line 166, in check
    raise Exception(
    ...<3 lines>...
    )
Exception: Port https://10.225.18.120:40081 (path: /) was not open
2025-12-10T17:30:30.317  INFO --- [et.reactor-0] localstack.request.aws     : AWS eks.DescribeCluster => 200
^F^[[6~^[[6~^[[6~^[[6~^A^CShutting down...
2025-12-10T12:35:40.513 DEBUG --- [  MainThread] l.u.c.docker_cmd_client    : Stopping container with cmd ['docker', 'stop', '--time', '10', '1ee6bfbeaf45c95dfd710ed4095457982d726c967110e0972f7049c103a83255']
2025-12-10T12:35:40.513 DEBUG --- [  MainThread] localstack.utils.run       : Executing command: ['docker', 'stop', '--time', '10', '1ee6bfbeaf45c95dfd710ed4095457982d726c967110e0972f7049c103a83255']
LocalStack supervisor: Shutting down localstack (PID 19)...
Localstack runtime received signal 15
2025-12-10T17:35:40.547 DEBUG --- [ime-Shutdown] localstack.runtime.runtime : [shutdown] Running shutdown hooks ...
2025-12-10T17:35:40.547 DEBUG --- [ime-Shutdown] l.p.c.extensions.plugins   : calling extensions on_platform_shutdown
^CERROR: '['docker', 'stop', '--time', '10', '1ee6bfbeaf45c95dfd710ed4095457982d726c967110e0972f7049c103a83255']': exit code 130; output: b'Flag --time has been deprecated, use --timeout instead\n'
2025-12-10T12:35:40.680  INFO --- [  MainThread] localstack.utils.bootstrap : error cleaning up localstack container localstack-main: ('Docker process returned with errorcode 130', b'Flag --time has been deprecated, use --timeout instead\n', None)
2025-12-10T17:35:41.042  INFO --- [ime-Shutdown] l.runtime.shutdown         : [shutdown] Stopping all services
2025-12-10T17:35:41.047 DEBUG --- [ime-Shutdown] l.u.c.docker_sdk_client    : Disconnecting container 'localstack-main' from network 'k3d-dev-gbl-eks-clus-d31271a9'
2025-12-10T17:35:41.048 DEBUG --- [ime-Shutdown] l.p.c.services.eks.cluster : Could not disconnect LocalStack container from k3d network
2025-12-10T17:35:41.048 DEBUG --- [ime-Shutdown] localstack.utils.run       : Executing command: ['/var/lib/localstack/lib/k3d/v5.8.3/k3d-linux-amd64', 'cluster', 'delete', 'dev-gbl-eks-clus-d31271a9']
2025-12-10T17:35:41.847 ERROR --- [ime-Shutdown] l.p.c.services.eks.cluster : Error stopping K3D cluster: 'NoneType' object has no attribute 'shutdown'
2025-12-10T17:35:41.847 DEBUG --- [ime-Shutdown] localstack.dns.server      : Reverting container DNS config

If I disconnect my VPN and run the same deployment it works perfectly, presumably since it uses my real network adapter but there is no log output for the same connection check to verify this.

Expected Behavior

Ideally LocalStack should perhaps ignore tunnel adapters or provide a mechanism for me to specify which local adapters are acceptable (possibly this is a k3d issue).

How are you starting LocalStack?

With the localstack script

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or `docker-compose.yml

sudo localstack start

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

this is an extract from a bigger script since I deploy multiple clusters and use the aws sdk directly with --endpoint

account="$1"
name="$2"
region="$3"
role="${name}ClusterRole"

policy_doc='{
      "Version":"2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": "eks.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }'

export AWS_ACCESS_KEY_ID=$account

# create iam role for cluster
role_output=$(aws --endpoint-url="${LOCALSTACK_URL}" iam create-role --role-name "$role" --assume-role-policy-document "$policy_doc" --no-cli-pager)
role_arn=$(echo "$role_output" | jq .Role.Arn)
aws --endpoint-url="${LOCALSTACK_URL}" iam attach-role-policy --policy-arn 'arn:aws:iam::aws:policy/AmazonEKSClusterPolicy' --role-name "$role"

# create a vpc
vpc_output=$(aws --endpoint-url="${LOCALSTACK_URL}" ec2 create-vpc --region "$region" --cidr-block "10.2.0.0/16")
vpc_id=$(echo "$vpc_output" | jq -r .Vpc.VpcId)

subnet_output=$(aws --endpoint-url="${LOCALSTACK_URL}" ec2 create-subnet  --region "$region" --vpc-id "$vpc_id" --cidr-block "10.2.1.0/18" --no-cli-pager)
subnet_id=$(echo "$subnet_output"| jq -r .Subnet.SubnetId )

# create the cluster
aws --endpoint-url="http://localhost:4566" eks create-cluster --region "$region" --name "$name" --role-arn="$role" --resources-vpc-config "subnetIds=$subnet_id" --no-cli-pager

Environment

- OS: Pop!_OS 24.04 LTS x86_64
- Kernel: Linux 6.17.9-76061709-generic
- LocalStack CLI: 4.11.1

Anything else?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    aws:eksAmazon Elastic Kubernetes Servicestatus: backlogTriaged but not yet being worked ontype: bugBug report

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions