More corner cases with iptables proxy #12764

thockin · 2015-08-15T05:39:42Z

Node ports
Set sysctls
Cleanup rules code
Faster sync at startup
Better logging

DNAT back to self doesn't quite work yet because we need to enable hairpin mode on the bridge, and that has to be done interface-by-interface (sigh). Will keep hunting for a better answer there.

This needs SERIOUS e2e testing. I have a very manual battery of tests I have been running, which is a good starting place.

@BenTheElder @ArtfulCoder

BenTheElder · 2015-08-15T05:55:06Z

Read through current last commit (thockin@23a3ab8).
LGTM

k8s-bot · 2015-08-15T05:58:41Z

GCE e2e build/test passed for commit 23a3ab8355eb48317dc2860a394ba76b5001a473.

k8s-bot · 2015-08-16T00:35:26Z

GCE e2e build/test passed for commit be9a28ca34ebf46c76fcb6000f96684dfdb4332a.

k8s-bot · 2015-08-16T00:47:49Z

GCE e2e build/test passed for commit c4e2c83f304966d7042819cd80ab512104856912.

k8s-bot · 2015-08-16T04:36:45Z

GCE e2e build/test passed for commit 392c340c852747e6f372ab9d82aeeb10e1ba4998.

k8s-bot · 2015-08-16T05:25:22Z

GCE e2e build/test passed for commit 35dd764ce77dbcbc0be77f74775c00bbf0770588.

k8s-bot · 2015-08-17T20:41:24Z

GCE e2e build/test passed for commit e93a7340c74cef99f7d78c1c5e5d45e84568ca72.

k8s-bot · 2015-08-17T22:58:54Z

GCE e2e build/test passed for commit b55b1b400d34582c0ccf76cad969c4cccad7f1dd.

ArtfulCoder · 2015-08-17T23:30:12Z

pkg/proxy/iptables/proxier.go

+				args = append(args,
+					"-m", "statistic",
+					"--mode", "random",
+					"--probability", fmt.Sprintf("%f", 1.0/float64(n-i)))


(discussed offline)
we should probably restrict decimal length

Done in next push

On Mon, Aug 17, 2015 at 4:30 PM, Abhi Shah [email protected] wrote:

In pkg/proxy/iptables/proxier.go
#12764 (comment)
:

}

if i < (n - 1) {

// Each rule is a probabilistic match.

args = append(args,

"-m", "statistic",

"--mode", "random",

"--probability", fmt.Sprintf("%f", 1.0/float64(n-i)))

(discussed offline)
we should probably restrict decimal length

—
Reply to this email directly or view it on GitHub
https://github.com/kubernetes/kubernetes/pull/12764/files#r37249463.

thockin · 2015-08-18T03:59:05Z

Issues fixed, new push is up. This needs e2e before it can be activated but I'd like to merge it ASAP.

k8s-bot · 2015-08-18T04:18:46Z

GCE e2e build/test failed for commit 6f34be3.

thockin · 2015-08-18T04:21:45Z

@k8s-bot test this please

k8s-bot · 2015-08-18T04:43:23Z

GCE e2e build/test passed for commit 6f34be3.

ArtfulCoder · 2015-08-18T08:49:28Z

LGTM

More corner cases with iptables proxy

BenTheElder · 2015-08-18T16:15:32Z

Whoo!
On Aug 18, 2015 12:14 PM, "Robert Bailey" [email protected] wrote:

Merged #12764 #12764.

—
Reply to this email directly or view it on GitHub
#12764 (comment).

BenTheElder · 2015-08-18T16:17:33Z

So we still need hairpin mode, anything else?

thockin · 2015-08-18T16:24:34Z

hairpin mode (hairy)
maybe load more kernel modules (just need to understand why)
maybe add "always masq" mode as a flag (easy)
hold node ports open (in progress)

Not sure what else. If you have extra bandwidth, you could do the "always
masq" mode. Is there more to do?

On Tue, Aug 18, 2015 at 9:18 AM, Benjamin Elder [email protected]
wrote:

So we still need hairpin mode, anything else?

—
Reply to this email directly or view it on GitHub
#12764 (comment)
.

BenTheElder · 2015-08-18T16:31:56Z

I'll take a look at the always masq flag this afternoon. As far as I know
that sounds like it, except maybe this:
d2iq-archive/kubernetes-mesos#353 (comment)

On Tue, Aug 18, 2015 at 12:25 PM, Tim Hockin [email protected]
wrote:

hairpin mode (hairy)
maybe load more kernel modules (just need to understand why)
maybe add "always masq" mode as a flag (easy)
hold node ports open (in progress)

Not sure what else. If you have extra bandwidth, you could do the "always
masq" mode. Is there more to do?

On Tue, Aug 18, 2015 at 9:18 AM, Benjamin Elder [email protected]
wrote:

So we still need hairpin mode, anything else?

—
Reply to this email directly or view it on GitHub
<
#12764 (comment)

.

—
Reply to this email directly or view it on GitHub
#12764 (comment)
.

thockin · 2015-08-18T16:35:57Z

yeah, maybe. It's P2 or P3 to me. The other thing we could use is better
e2e testing of this new mode. That's a non-trivial undertaking. I'm happy
to consume more of your time, but I know you're back in school...

On Tue, Aug 18, 2015 at 9:32 AM, Benjamin Elder [email protected]
wrote:

I'll take a look at the always masq flag this afternoon. As far as I know
that sounds like it, except maybe this:

d2iq-archive/kubernetes-mesos#353 (comment)

On Tue, Aug 18, 2015 at 12:25 PM, Tim Hockin [email protected]
wrote:

hairpin mode (hairy)
maybe load more kernel modules (just need to understand why)
maybe add "always masq" mode as a flag (easy)
hold node ports open (in progress)

Not sure what else. If you have extra bandwidth, you could do the "always
masq" mode. Is there more to do?

On Tue, Aug 18, 2015 at 9:18 AM, Benjamin Elder <
[email protected]>
wrote:

So we still need hairpin mode, anything else?

—
Reply to this email directly or view it on GitHub
<

#12764 (comment)

.

—
Reply to this email directly or view it on GitHub
<
#12764 (comment)

.

—
Reply to this email directly or view it on GitHub
#12764 (comment)
.

BenTheElder · 2015-08-18T16:38:51Z

Yeah, I'll see if I can get a bit more done early this week before i get
really busy and GSoC ends. So far it's mostly syllabi and realizing I
need to review a few things. Should have time at least to knock out always
masq mode.

On Tue, Aug 18, 2015 at 12:36 PM, Tim Hockin [email protected]
wrote:

yeah, maybe. It's P2 or P3 to me. The other thing we could use is better
e2e testing of this new mode. That's a non-trivial undertaking. I'm happy
to consume more of your time, but I know you're back in school...

On Tue, Aug 18, 2015 at 9:32 AM, Benjamin Elder [email protected]
wrote:

I'll take a look at the always masq flag this afternoon. As far as I know
that sounds like it, except maybe this:

d2iq-archive/kubernetes-mesos#353 (comment)

On Tue, Aug 18, 2015 at 12:25 PM, Tim Hockin [email protected]
wrote:

hairpin mode (hairy)
maybe load more kernel modules (just need to understand why)
maybe add "always masq" mode as a flag (easy)
hold node ports open (in progress)

Not sure what else. If you have extra bandwidth, you could do the
"always
masq" mode. Is there more to do?

On Tue, Aug 18, 2015 at 9:18 AM, Benjamin Elder <
[email protected]>
wrote:

So we still need hairpin mode, anything else?

—
Reply to this email directly or view it on GitHub
<

#12764 (comment)

.

—
Reply to this email directly or view it on GitHub
<

#12764 (comment)

.

—
Reply to this email directly or view it on GitHub
<
#12764 (comment)

.

—
Reply to this email directly or view it on GitHub
#12764 (comment)
.

googlebot added the cla: yes label Aug 15, 2015

thockin force-pushed the proxy-nodeports branch 2 times, most recently from be9a28c to c4e2c83 Compare August 16, 2015 00:27

thockin mentioned this pull request Aug 16, 2015

External IPs support #12561

Merged

thockin force-pushed the proxy-nodeports branch from 392c340 to 35dd764 Compare August 16, 2015 05:03

mbforbes assigned ArtfulCoder Aug 17, 2015

thockin force-pushed the proxy-nodeports branch from 35dd764 to e93a734 Compare August 17, 2015 20:16

thockin force-pushed the proxy-nodeports branch from e93a734 to b55b1b4 Compare August 17, 2015 22:10

ArtfulCoder reviewed Aug 17, 2015
View reviewed changes

thockin added 9 commits August 17, 2015 20:52

Add nodepoprt chain and link it in, add unused MASQ rule

d14c98f

Clean up iptables rules, add nodeport support

731d5e5

Include protocol in the hash for chain names

d72892d

Clean up logging, make initial sync faster

f1a4857

test for and set route_localnet sysctl

9cf3377

test for and set bridge-nf-call-iptables sysctl

3a5c23d

Make iptables use semver lib

3d30970

Require same min iptables version as -C

7e9c685

Limit float precision to 5 points

6f34be3

thockin force-pushed the proxy-nodeports branch from b55b1b4 to 6f34be3 Compare August 18, 2015 03:58

ArtfulCoder added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 18, 2015

roberthbailey added a commit that referenced this pull request Aug 18, 2015

Merge pull request #12764 from thockin/proxy-nodeports

c45747b

More corner cases with iptables proxy

roberthbailey merged commit c45747b into kubernetes:master Aug 18, 2015

thockin mentioned this pull request Aug 25, 2015

iptables based Kube-proxy should support NodePort #13123

Closed

alexhersh mentioned this pull request Sep 30, 2015

kube-proxy doesn't open ports projectcalico/k8s-exec-plugin#50

Closed

danwinship mentioned this pull request Aug 29, 2019

fix KUBE-MARK-MASQ ownership (kubelet vs kube-proxy) #82125

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

More corner cases with iptables proxy #12764

More corner cases with iptables proxy #12764

thockin commented Aug 15, 2015

BenTheElder commented Aug 15, 2015

k8s-bot commented Aug 15, 2015

k8s-bot commented Aug 16, 2015

k8s-bot commented Aug 16, 2015

k8s-bot commented Aug 16, 2015

k8s-bot commented Aug 16, 2015

k8s-bot commented Aug 17, 2015

k8s-bot commented Aug 17, 2015

ArtfulCoder Aug 17, 2015

thockin Aug 18, 2015

thockin commented Aug 18, 2015

k8s-bot commented Aug 18, 2015

thockin commented Aug 18, 2015

k8s-bot commented Aug 18, 2015

ArtfulCoder commented Aug 18, 2015

BenTheElder commented Aug 18, 2015

BenTheElder commented Aug 18, 2015

thockin commented Aug 18, 2015

BenTheElder commented Aug 18, 2015

thockin commented Aug 18, 2015

BenTheElder commented Aug 18, 2015

More corner cases with iptables proxy #12764

More corner cases with iptables proxy #12764

Conversation

thockin commented Aug 15, 2015

BenTheElder commented Aug 15, 2015

k8s-bot commented Aug 15, 2015

k8s-bot commented Aug 16, 2015

k8s-bot commented Aug 16, 2015

k8s-bot commented Aug 16, 2015

k8s-bot commented Aug 16, 2015

k8s-bot commented Aug 17, 2015

k8s-bot commented Aug 17, 2015

ArtfulCoder Aug 17, 2015

Choose a reason for hiding this comment

thockin Aug 18, 2015

Choose a reason for hiding this comment

thockin commented Aug 18, 2015

k8s-bot commented Aug 18, 2015

thockin commented Aug 18, 2015

k8s-bot commented Aug 18, 2015

ArtfulCoder commented Aug 18, 2015

BenTheElder commented Aug 18, 2015

BenTheElder commented Aug 18, 2015

thockin commented Aug 18, 2015

BenTheElder commented Aug 18, 2015

thockin commented Aug 18, 2015

BenTheElder commented Aug 18, 2015