What type of cluster are you running (GKE, or what instructions did you use to setup your cluster?)

A short term solution might be to try adding --cadvisor-port=0 flag to kubelet and restarting the kubelet service and see if the sensitive information is no longer leaked.

Doing this may the UI and heapster which expect to be able to poll the cadvisor port.

I think @dchen1107 probably knows if there is a longer term solution planned, such as:

• only serve on specified interface. if this is localhost, then it breaks the ui, but maybe that is okay. maybe @lavalamp has thought about this.
• require authentication/authorization to connect to cAdvisor port.

@vmarmol

Self deployed cluster, the kubelet is started via

[Unit]
Description=Kubelet
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
TimeoutStopSec=10
ExecStartPre=/sbin/iptables -A INPUT -p tcp --dport 4194 -j REJECT
ExecStart=/usr/local/bin/kubelet \
--address={{ salt.netinfo.ip() }} \
--api-servers=https://10.200.0.1:6443 \
--auth-path=/etc/kubernetes/kubelet_auth.json \
--cluster-dns=10.100.1.1 \
--config={{ pillar['kubernetes']['config_dir'] }}/manifests \
--hostname-override={{ grains['host'] }} \
--tls-cert-file=/etc/kubernetes/kubelet-tls.crt \
--tls-private-key-file=/etc/kubernetes/kubelet-tls.key
ExecStopPost=/sbin/iptables -D INPUT -p tcp --dport 4194 -j REJECT

Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

We are working on improving the kubelet APIs which will subsume cadvisor APIs. Until then setting cadvisor port to '0' is probably the way to go.

@vishh Is there an issue and target date for doing this?

Filed #12483. The current plan is to complete this for v1.1. cc @dchen1107

@philips Do you mean the read-only kubelet port (10255 by default IIRC)? If so, then we really need to disable this & provide authorisation options on the kubelet to be able to retrieve stats fro the kubelet via the normal port with a restricted token/cert/auth options. We've discussed this for OpenShift (openshift/origin#3020) too.

@philips Sorry I didn't realise that cadvisor was bound to port 4194 as well. Thought this was all embedded inside the kubelet process without exposing any ports.

@vmarmol @vishh Any reason why we start up cadvisor http server at all? https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/cadvisor/cadvisor_linux.go#L99-L103 Seems unnecessary now kubernetes embeds cadvisor & exposes via /stats & /metrics?

/stats does not expose all the containers (/docker-daemon, /kubelet, etc). @mvdan is working on introducing a better stats API in kubelet (#12483) that should help us shut down cadvisor's http server in kubelet.

Couldn't we just add cadvisor as a handler in the kubelet anyway? No need for separate http server & would mean access to raw data is available if anyone wants it.

Indeed that is an option. But we need a better stats API in kubelet anyways for reasons mentioned in #12483.

Absolutely agree but thinking quick win.

+1 on a quick win. We just disabled it by default instead: https://github.com/coreos/coreos-overlay/blob/master/app-admin/kubelet/files/kubelet.service

Which reminds me it would be great to have #12418 so we could make it easy to turn on/off stuff like this w/o rewriting the entire command-line in a service file.

cc @timstclair

@vishh it's not 100% clear to me whether or not #12483 made it safe to use --cadvisor-port=0, could you please clarify? On a fresh k8s 1.6.4 cluster kubelet still exposes cAdvisor to the world:

State   Recv-Q  Send-Q  Local Address:Port  Peer Address:Port
LISTEN  0       128       :::4194              :::*

@philips:

• the file you referenced was deleted as part of coreos/coreos-overlay@fa290a8f
• the documentation @ Kubelet Wrapper Script (CoreOS docs) doesn't mention cAdvisor should be disabled either. Another reason for my question to Vish.

Ahh. That's a bug that the new component config based defaulting scheme introduced.
@mtaufen CadvisorPort being 0 is a valid setting.

I think we can change this. The KubeletConfiguration type is still technically alpha, so it should be ok to make this field a pointer and keep the current default when it's not provided (but allow 0). This won't affect anyone who presently omits the flag and expects port 4194. Theoretically people could be explicitly passing 0 and expecting 4194... I guess... but that's sort of an insane case, and I'd consider that behavior a bug.

Thanks for clarifying guys 👍
The original problem remains though: enabling the cAdvisor port makes the kubelet listen on all interfaces.

/assign