kubeadm's os.Hostname() isn't getting the FQDN the expected way? (causes a fail to join a node in 1.7.0 when hostnameOverride is used) #48617
Description
An interesting issue occurred to me this week, the (v1.7.0) kubeadm init cannot register itself:
Jul 07 18:48:53 kitchen-k8s-minion kubelet[19864]: I0707 18:48:53.169167 19864 kubelet_node_status.go:82] Attempting to register node kitchen-k8s-minion.novalocal
Jul 07 18:48:53 kitchen-k8s-minion kubelet[19864]: E0707 18:48:53.228445 19864 kubelet_node_status.go:106] Unable to register node "kitchen-k8s-minion.novalocal" with API server: nodes "kitchen-k8s-minion.novalocal" is forbidden: node kitchen-k8s-minion cannot modify node kitchen-k8s-minion.novalocal
we are setting the KUBELET_EXTRA_ARGS=--hostname-override=kitchen-k8s-minion.novalocal in /etc/systemd/system/kubelet.service.d/20-extra-args.conf
Checking the hostname:
[kitchen-k8s-minion]# for f in -a -A -b -d -f -i -I -s; do echo -ne "$f:\t"; hostname $f;done
-a: kitchen-k8s-minion
-A: kitchen-k8s-minion
-b: kitchen-k8s-minion
-d: kitchen-k8s-minion
-f: kitchen-k8s-minion.novalocal
-i: kitchen-k8s-minion
-I: kitchen-k8s-minion
-s: kitchen-k8s-minion
[root@kitchen-k8s-minion ~]# cat /proc/sys/kernel/hostname
kitchen-k8s-minion
[root@kitchen-k8s-minion ~]# cat /proc/sys/kernel/domainname
(none)
[root@kitchen-k8s-minion ~]# hostnamectl |head -2
Static hostname: kitchen-k8s-minion
Icon name: computer-vm
Interestingly, mesos does hostname -f to get the FQDN as seen in
whilst the k8s does os.Hostname() only which is not getting the FQDN
kubernetes/pkg/util/node/node.go
Line 48 in 95be540
Trying to see how exactly does Golang make os.Hostname() call:
[kitchen-k8s-minion]# cat app.go
package main
import (
"fmt"
"os"
)
func main() {
nodename, err := os.Hostname()
if err != nil {
fmt.Println("Couldn't determine hostname: %v", err)
}
fmt.Println(nodename)
}
[kitchen-k8s-minion]# docker run --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp golang:1.8 go build -v
[kitchen-k8s-minion]# strace -e openat ./myapp
openat(AT_FDCWD, "/proc/sys/kernel/hostname", O_RDONLY|O_CLOEXEC) = 3
kitchen-k8s-minion
+++ exited with 0 +++
Is this expected?
Should not we rather use hostname -f as seen in the mesos example (mentioned above)?
And it would be nice to have an option for a hostname override during kubeadm init in my opinion..
This might have been also a cause for the issues similar to this one kubernetes/kubeadm#144 where a gentleman would be expecting the FQDN or willing to set the hostname override and the kubeadm should be also respecting it.
Running
- Kubernetes v1.7.0
- [kitchen-k8s-minion] is an openstack instance running CentOS 7 with Linux kernel 3.10.0