Skip to content

Conversation

@bart0sh
Copy link
Contributor

@bart0sh bart0sh commented Oct 19, 2018

What type of PR is this?
/kind bug

What this PR does / why we need it:
Order of Volumes and VolumeMounts in the pod objects created by
kubeadm is undefined as they're represended as maps in the
controlPlaneHostPathMounts struct.

This influences 'kubeadm upgrade' logic in a way that even when
manifest of the component is not changed kubeadm tries to upgrade
it because most of the time current and new pods are not equal
due to the different order of Volumes and VolumeMounts.

For example 'kubeadm apply diff' almost always shows difference
in Volumes and VolumeMounts because of this:

     volumeMounts:
+    - mountPath: /etc/kubernetes/pki
+      name: k8s-certs
+      readOnly: true
     - mountPath: /etc/ssl/certs
       name: ca-certs
+      readOnly: true
+    - mountPath: /etc/pki
+      name: etc-pki
+      readOnly: true
+    - mountPath: /usr/share/ca-certificates
+      name: usr-share-ca-certificates
       readOnly: true
     - mountPath: /etc/ca-certificates
       name: etc-ca-certificates
       readOnly: true
-    - mountPath: /etc/pki
-      name: etc-pki
-      readOnly: true
-    - mountPath: /etc/kubernetes/pki
-      name: k8s-certs
-      readOnly: true
-    - mountPath: /usr/share/ca-certificates
-      name: usr-share-ca-certificates
-      readOnly: true

Sorting Volumes and VolumeMounts should fix this issue and help
to avoid unnecessary upgrades.

Which issue(s) this PR fixes:
Fixes kubernetes/kubeadm#1054

Does this PR introduce a user-facing change?:

kubeadm: fix unnecessary upgrades caused by undefined order of Volumes and VolumeMounts in manifests

Order of Volumes and VolumeMounts in the pod objects created by
kubeadm is undefined as they're represended as maps in the
controlPlaneHostPathMounts struct.

This influences 'kubeadm upgrade' logic in a way that even when
manifest of the component is not changed kubeadm tries to upgrade
it because most of the time current and new pods are not equal
due to the different order of Volumes and VolumeMounts.

For example 'kubeadm apply diff' almost always shows difference
in Volumes and VolumeMounts because of this:
     volumeMounts:
+    - mountPath: /etc/kubernetes/pki
+      name: k8s-certs
+      readOnly: true
     - mountPath: /etc/ssl/certs
       name: ca-certs
+      readOnly: true
+    - mountPath: /etc/pki
+      name: etc-pki
+      readOnly: true
+    - mountPath: /usr/share/ca-certificates
+      name: usr-share-ca-certificates
       readOnly: true
     - mountPath: /etc/ca-certificates
       name: etc-ca-certificates
       readOnly: true
-    - mountPath: /etc/pki
-      name: etc-pki
-      readOnly: true
-    - mountPath: /etc/kubernetes/pki
-      name: k8s-certs
-      readOnly: true
-    - mountPath: /usr/share/ca-certificates
-      name: usr-share-ca-certificates
-      readOnly: true

Sorting Volumes and VolumeMounts should fix this issue and help
to avoid unnecessary upgrades.
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Oct 19, 2018
@k8s-ci-robot k8s-ci-robot added area/kubeadm sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Oct 19, 2018
@bart0sh
Copy link
Contributor Author

bart0sh commented Oct 19, 2018

/cc @neolit123

Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bart0sh thank you for this PR.

i think the PR is pretty self-explanatory and the sorted volumes is something we should definitely have.

one small request: please prefix the kubeadm release notes like so - e.g. kubeadm: note goes here.....
thanks

/lgtm
/assign @timothysc
/assign @liztio
(as per upgrade diff).

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 19, 2018
Copy link
Contributor

@timothysc timothysc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bart0sh, timothysc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 23, 2018
@k8s-ci-robot k8s-ci-robot merged commit 46ec13c into kubernetes:master Oct 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubeadm cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kubeadm upgrade apply failure with same kubernetes version

5 participants