Skip to content

e2e: node: memory manager: wait for pod pending deletion before do the actual test #128894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ffromani wants to merge 1 commit into from
Closed

e2e: node: memory manager: wait for pod pending deletion before do the actual test #128894

ffromani wants to merge 1 commit into from

Conversation

@ffromani
Copy link
Contributor

@ffromani ffromani commented Nov 20, 2024

What type of PR is this?

/kind bug
/kind flake

What this PR does / why we need it:

The metrics tests, memory manager first and foremost, expect a controlled environment on which they run.
This is why they are labeled Serial. The key factor here is the predictability: the test expect
to be in full control of the node state; if any other actor mutates the node state, that makes for a almost guaranteed
false negative.

To reduce the chance to flake, we add an extra check before to run the actual tests: detect pods pending deletion, and
wait for them to be gone.

Which issue(s) this PR fixes:

Fixes #128869

Special notes for your reviewer:

see slack thread: https://kubernetes.slack.com/archives/C0BP8PW9G/p1732023371237679

Does this PR introduce a user-facing change?

NONE

@ffromani
e2e: node: memory manager: wait for pod pending delete
8586d8f 
The metrics tests, memory manager first and foremost,
expect a controlled environment on which they run.
This is why they are labeled Serial.
The key factor here is the predictability: the test expect
to be in full control of the node state; if any other actor
mutates the node state, that makes for a almost guaranteed
false negative.

To reduce the chance to flake, we add an extra check before
to run the actual tests: detect pods pending deletion, and
wait for them to be gone.

Signed-off-by: Francesco Romani <[email protected]>
@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. kind/flake Categorizes issue or PR as related to a flaky test. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Nov 20, 2024
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Nov 20, 2024

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/test sig/node Categorizes an issue or PR as relevant to SIG Node. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Nov 20, 2024
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Nov 20, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ffromani
Once this PR has been reviewed and has the lgtm label, please assign sjenning for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ffromani
Copy link
Contributor Author

ffromani commented Nov 20, 2024

/hold

if we like this direction, I want to update all the other metrics tests

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 20, 2024
@ffromani
Copy link
Contributor Author

ffromani commented Jan 20, 2025

/close

we don't really like this direction after all

@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Jan 20, 2025

@ffromani: Closed this PR.

Details

In response to this:

/close

we don't really like this direction after all

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewsykim andrewsykim Awaiting requested review from andrewsykim

@Random-Liu Random-Liu Awaiting requested review from Random-Liu

Assignees

No one assigned

Labels

area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. kind/bug Categorizes issue or PR as related to a bug. kind/flake Categorizes issue or PR as related to a flaky test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. release-note-none Denotes a PR that doesn't merit a release note. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

SIG Node CI/Test Board
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Failure cluster [20b4f3ec...] Memory Manager Metrics [Serial] [Feature:MemoryManager] when querying /metrics should report zero pinning counters after a fresh restart

2 participants

@ffromani @k8s-ci-robot