Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix resource quota controller panic #52092

Merged

Conversation

ironcladlou
Copy link
Contributor

@ironcladlou ironcladlou commented Sep 7, 2017

The pod evaluator used by the resource quota controller made direct calls to an
unsafe pod conversion function which mutates the pod argument. With multiple
resource quota controller workers, concurrent processing of the same pod from a
shared informer can result in a panic when the conversion code attempts to write
to a map field in the pod.

Swap out the direct conversion function call to Scheme.ConvertToVersion, which
copies the input before conversion.

Fix a kube-controller-manager crash which can result when `--concurrent-resource-quota-syncs` is >1 and pods exist in the system containing certain alpha/beta annotation keys.

Fixes #52093.

@kubernetes/sig-api-machinery-bugs

/cc @liggitt

The pod evaluator used by the resource quota controller made direct calls to an
unsafe pod conversion function which mutates the pod argument. With multiple
resource quota controller workers, concurrent processing of the same pod from a
shared informer can result in a panic when the conversion code attempts to write
to a map field in the pod.

Swap out the direct conversion function call to Scheme.ConvertToVersion, which
copies the input before conversion.
@k8s-ci-robot k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. kind/bug Categorizes issue or PR as related to a bug. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Sep 7, 2017
@k8s-github-robot k8s-github-robot added do-not-merge/cherry-pick-not-approved Indicates that a PR is not yet approved to merge into a release branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Sep 7, 2017
@liggitt
Copy link
Member

liggitt commented Sep 7, 2017

link to issue?

@liggitt
Copy link
Member

liggitt commented Sep 7, 2017

release note can probably be less detailed

@liggitt
Copy link
Member

liggitt commented Sep 7, 2017

this LGTM

@derekwaynecarr
Copy link
Member

/approve

@k8s-github-robot k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 7, 2017
@derekwaynecarr
Copy link
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 7, 2017
@k8s-github-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: derekwaynecarr, ironcladlou

Associated issue: 52093

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

@ironcladlou
Copy link
Contributor Author

/test pull-kubernetes-e2e-kops-aws

@caesarxuchao
Copy link
Member

lgtm

@liggitt liggitt added this to the v1.7 milestone Sep 8, 2017
@liggitt
Copy link
Member

liggitt commented Sep 8, 2017

@wojtek-t for release approval

@wojtek-t wojtek-t added cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. and removed do-not-merge/cherry-pick-not-approved Indicates that a PR is not yet approved to merge into a release branch. labels Sep 8, 2017
@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot
Copy link

Automatic merge from submit-queue

openshift-merge-robot added a commit to openshift/origin that referenced this pull request Sep 8, 2017
Automatic merge from submit-queue

UPSTREAM: 52092: Fix resource quota controller panic (Drop in 1.8)

The pod evaluator used by the resource quota controller made direct
calls to an unsafe pod conversion function which mutates the pod
argument. With multiple resource quota controller workers, concurrent
processing of the same pod from a shared informer can result in a panic
when the conversion code attempts to write to a map field in the pod.

Swap out the direct conversion function call to Scheme.ConvertToVersion,
which copies the input before conversion.

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1486416
/xref kubernetes/kubernetes#52092
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants