Add support for IP aliases for pod IPs (GCP alpha feature) #42147

bowei · 2017-02-27T08:36:16Z

Adds support for allocation of pod IPs via IP aliases.

Adds KUBE_GCE_ENABLE_IP_ALIASES flag to the cluster up scripts (`kube-{up,down}.sh`).

KUBE_GCE_ENABLE_IP_ALIASES=true will enable allocation of PodCIDR ips
using the ip alias mechanism rather than using routes. This feature is currently
only available on GCE.

Usage

$ CLUSTER_IP_RANGE=10.100.0.0/16 KUBE_GCE_ENABLE_IP_ALIASES=true bash -x cluster/kube-up.sh

Adds CloudAllocator to the node CIDR allocator (kubernetes-controller manager).

If CIDRAllocatorType is set to CloudCIDRAllocator, then allocation
of CIDR allocation instead is done by the external cloud provider and
the node controller is only responsible for reflecting the allocation
into the node spec.

Splits off the rangeAllocator from the cidr_allocator.go file.
Adds cloudCIDRAllocator, which is used when the cloud provider allocates
the CIDR ranges externally. (GCE support only)
Updates RBAC permission for node controller to include PATCH

k8s-reviewable · 2017-02-27T08:36:23Z

This change is

bowei · 2017-02-27T08:36:55Z

@thockin @freehan

xiangpengzhao · 2017-02-27T12:23:13Z

pkg/controller/node/cloud_cidr_allocator.go

+			glog.V(3).Infof("Node %v has PodCIDR %v", node.Name, podCIDR)
+			return nil
+		}
+		glog.Errorf("PodCIDR cannot be reassigned, node %v spec has %v, but cloud provider has assigned %v",


Should here return error? According to the error info, it seems yes.

But according to the logic below, the PodCIDR should update to CIDR assigned by cloud provider. So IMO here may need to be modified to something like:

glog.V(3).Infof("PodCIDR will be reassigned, node %v spec has %v, but cloud provider has assigned %v", node.Name, node.Spec.PodCIDR, podCIDR)

We do not support reassignment of PodCIDRs once they have been assigned.

Then we should return error here, otherwise PodCIDR will be reassigned to cidr[0] in line #L110, right? Hopefully I understand correctly:)

The range_allocator.go code logs an error, but continues with setting the node spec. This basically copies the current behavior, I'll add a comment to be more clear. This is somewhat of a unhandled state for the system, as the underlying layers will ignore the CIDR update.

Thanks for the explanation!

bowei · 2017-03-01T23:00:27Z

@k8s-bot kubemark e2e test this

roberthbailey · 2017-04-10T07:45:38Z

I only looked at the shell code. By the way, why is this change in 7 commits?

bowei · 2017-04-10T16:47:09Z

@roberthbailey -- commits are grouped in a way to make it easy to review / rebase

bowei · 2017-04-11T00:48:07Z

@k8s-bot test this

k8s-ci-robot · 2017-04-11T02:01:34Z

@bowei: The following test(s) failed:

Test name	Commit	Details	Rerun command
Jenkins GKE smoke e2e	cbff59eeb157c2615d3aa00f41e622a176f25adb	link	`@k8s-bot cvm gke e2e test this`
Jenkins GCI GKE smoke e2e	cbff59eeb157c2615d3aa00f41e622a176f25adb	link	`@k8s-bot gci gke e2e test this`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

KUBE_GCE_ENABLE_IP_ALIASES=true will enable allocation of PodCIDR ips using the ip alias mechanism rather than using routes. NODE_IP_RANGE will control the node instance IP cidr KUBE_GCE_IP_ALIAS_SIZE controls the size of each podCIDR IP_ALIAS_SUBNETWORK controls the name of the subnet created for the cluster

If CIDRAllocatorType is set to `CloudCIDRAllocator`, then allocation of CIDR allocation instead is done by the external cloud provider and the node controller is only responsible for reflecting the allocation into the node spec. - Splits off the rangeAllocator from the cidr_allocator.go file. - Adds cloudCIDRAllocator, which is used when the cloud provider allocates the CIDR ranges externally. (GCE support only) - Updates RBAC permission for node controller to include PATCH

I also sorted the file, it was almost sorted with a few exceptions.

bowei · 2017-04-11T22:07:47Z

@thockin -- there will be follow on commits mostly likely as I get the e2e testing running through in ip alias mode.

thockin · 2017-04-11T22:56:43Z

/lgtm since @roberthbailey is ooo

thockin · 2017-04-11T22:57:28Z

/lgtm

k8s-github-robot · 2017-04-11T22:57:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bowei, thockin

Needs approval from an approver in each of these OWNERS Files:

~~OWNERS~~ [thockin]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

k8s-github-robot · 2017-04-12T05:09:24Z

Automatic merge from submit-queue

Automatic merge from submit-queue Clean up CHANGELOG.md There was extra stuff inside the `release-note` block for #42147 that messed up the formatting for CHANGELOG.md. I edited the PR to fix it, but the CHANGELOG.md had already been pushed.

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Feb 27, 2017

k8s-github-robot assigned lavalamp Feb 27, 2017

xiangpengzhao reviewed Feb 27, 2017

View reviewed changes

bowei force-pushed the ip-alias-2 branch 8 times, most recently from cc18de2 to 3d90228 Compare February 28, 2017 07:49

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 28, 2017

bowei mentioned this pull request Feb 28, 2017

[Proposal] Integrate kube-DNS with external DNS #28453

Closed

bowei force-pushed the ip-alias-2 branch from 3d90228 to cbff59e Compare March 2, 2017 01:21

k8s-github-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Mar 2, 2017

bowei force-pushed the ip-alias-2 branch from cbff59e to d8d9125 Compare March 21, 2017 06:01

k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 21, 2017

bowei force-pushed the ip-alias-2 branch 2 times, most recently from 2a3520e to bc60716 Compare March 21, 2017 20:37

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 24, 2017

bowei force-pushed the ip-alias-2 branch from bc60716 to 4809883 Compare March 27, 2017 21:39

k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 27, 2017

bowei force-pushed the ip-alias-2 branch from 4809883 to 34d9ddf Compare March 27, 2017 23:14

bowei force-pushed the ip-alias-2 branch from 7a999cc to 272885a Compare April 10, 2017 18:55

bowei added 3 commits April 10, 2017 12:05

Add google cloud alpha interfaces to Godep (godep file)

b5247c5

Godep vendor files for google cloud alpha

f51b26e

Add PodCIDRs API for GCE (Google cloud alpha feature)

f5be63e

bowei force-pushed the ip-alias-2 branch from 272885a to f7c8308 Compare April 10, 2017 20:40

roberthbailey self-assigned this Apr 11, 2017

bowei force-pushed the ip-alias-2 branch 2 times, most recently from fa09323 to 96298b8 Compare April 11, 2017 19:30

bowei changed the title ~~IP alias 2~~ Add support for IP aliases for pod IPs (this is a GCP alpha feature) Apr 11, 2017

bowei changed the title ~~Add support for IP aliases for pod IPs (this is a GCP alpha feature)~~ Add support for IP aliases for pod IPs (GCP alpha feature) Apr 11, 2017

bowei force-pushed the ip-alias-2 branch from 96298b8 to 13a8f37 Compare April 11, 2017 20:39

bowei added 3 commits April 11, 2017 14:07

Update known-flags with cidr-allocator-type

091e46e

I also sorted the file, it was almost sorted with a few exceptions.

bowei force-pushed the ip-alias-2 branch from 13a8f37 to 091e46e Compare April 11, 2017 21:08

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 11, 2017

k8s-github-robot merged commit ceccd30 into kubernetes:master Apr 12, 2017

bowei deleted the ip-alias-2 branch April 12, 2017 05:10

enisoc mentioned this pull request Apr 20, 2017

Clean up CHANGELOG.md #44733

Merged

gmarek mentioned this pull request May 5, 2017

Taint based evictions is broken #45342

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for IP aliases for pod IPs (GCP alpha feature) #42147

Add support for IP aliases for pod IPs (GCP alpha feature) #42147

bowei commented Feb 27, 2017 •

edited by enisoc

Loading

k8s-reviewable commented Feb 27, 2017

bowei commented Feb 27, 2017

xiangpengzhao Feb 27, 2017 •

edited

Loading

bowei Feb 27, 2017

xiangpengzhao Feb 28, 2017

bowei Feb 28, 2017

xiangpengzhao Feb 28, 2017

bowei commented Mar 1, 2017

roberthbailey commented Apr 10, 2017

bowei commented Apr 10, 2017

bowei commented Apr 11, 2017

k8s-ci-robot commented Apr 11, 2017 •

edited

Loading

bowei commented Apr 11, 2017

thockin commented Apr 11, 2017

thockin commented Apr 11, 2017

k8s-github-robot commented Apr 11, 2017

k8s-github-robot commented Apr 12, 2017

Add support for IP aliases for pod IPs (GCP alpha feature) #42147

Add support for IP aliases for pod IPs (GCP alpha feature) #42147

Conversation

bowei commented Feb 27, 2017 • edited by enisoc Loading

Adds KUBE_GCE_ENABLE_IP_ALIASES flag to the cluster up scripts (kube-{up,down}.sh).

Usage

Adds CloudAllocator to the node CIDR allocator (kubernetes-controller manager).

k8s-reviewable commented Feb 27, 2017

bowei commented Feb 27, 2017

xiangpengzhao Feb 27, 2017 • edited Loading

Choose a reason for hiding this comment

bowei Feb 27, 2017

Choose a reason for hiding this comment

xiangpengzhao Feb 28, 2017

Choose a reason for hiding this comment

bowei Feb 28, 2017

Choose a reason for hiding this comment

xiangpengzhao Feb 28, 2017

Choose a reason for hiding this comment

bowei commented Mar 1, 2017

roberthbailey commented Apr 10, 2017

bowei commented Apr 10, 2017

bowei commented Apr 11, 2017

k8s-ci-robot commented Apr 11, 2017 • edited Loading

bowei commented Apr 11, 2017

thockin commented Apr 11, 2017

thockin commented Apr 11, 2017

k8s-github-robot commented Apr 11, 2017

k8s-github-robot commented Apr 12, 2017

bowei commented Feb 27, 2017 •

edited by enisoc

Loading

Adds KUBE_GCE_ENABLE_IP_ALIASES flag to the cluster up scripts (`kube-{up,down}.sh`).

xiangpengzhao Feb 27, 2017 •

edited

Loading

k8s-ci-robot commented Apr 11, 2017 •

edited

Loading