Skip to content

Conversation

@feiskyer
Copy link
Member

@feiskyer feiskyer commented Aug 12, 2019

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

/kind bug

What this PR does / why we need it:

Refer Azure/go-autorest#398. In go-autorest SDK https://github.com/Azure/go-autorest/blob/master/autorest/sender.go#L258,
if ARM returns http.StatusTooManyRequests, the sender doesn't increase the retry attempt count,
hence the Azure clients will keep retrying forever until it gets a status code other than 429.

// don't count a 429 against the number of attempts
// so that we continue to retry until it succeeds
if resp == nil || resp.StatusCode != http.StatusTooManyRequests {
    attempt++
}

So this PR explicitly remove http.StatusTooManyRequests from autorest.StatusCodesForRetry.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

  • Need cherry pick to v1.13-v1.15.

Does this PR introduce a user-facing change?:

Fix Azure client requests stuck issues on http.StatusTooManyRequests (HTTP Code 429).

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


/kind bug
/area provider/azure
/priority critial-urgent

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. area/provider/azure Issues or PRs related to azure provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Aug 12, 2019
@feiskyer
Copy link
Member Author

/sig cloud-provider
/assign @andyzhangx

@k8s-ci-robot k8s-ci-robot added sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. approved Indicates a PR has been approved by an approver from all required OWNERS files. area/cloudprovider and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Aug 12, 2019
@feiskyer
Copy link
Member Author

/priority critical-urgent

@k8s-ci-robot k8s-ci-robot added priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Aug 12, 2019
Copy link
Member

@andyzhangx andyzhangx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the PR title could be Stop retry when got StatusTooManyRequests error ?

@feiskyer feiskyer changed the title Fix precondition cancel Fix Azure client requests stuck issues on http.StatusTooManyRequests Aug 12, 2019
Copy link
Member

@andyzhangx andyzhangx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also this PR contains two fixes, would you split? and do you want to cherry pick?

@feiskyer
Copy link
Member Author

feiskyer commented Aug 12, 2019

also this PR contains two fixes, would you split? and do you want to cherry pick?

Yep, it would be cherry-picked to old versions. Let me split the PR into two, cause they would be cherry picked to different versions.

@feiskyer feiskyer force-pushed the fix-precondition-cancel branch from 3d2b157 to 0bcee75 Compare August 12, 2019 08:32
@feiskyer feiskyer force-pushed the fix-precondition-cancel branch from 0bcee75 to e53c57a Compare August 12, 2019 08:42
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 12, 2019
@feiskyer
Copy link
Member Author

/test pull-kubernetes-e2e-aks-engine-azure

Copy link
Member

@andyzhangx andyzhangx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 13, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx, feiskyer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit 54b28ab into kubernetes:master Aug 13, 2019
@k8s-ci-robot k8s-ci-robot added this to the v1.16 milestone Aug 13, 2019
k8s-ci-robot added a commit that referenced this pull request Aug 16, 2019
…79-upstream-release-1.14

Automated cherry pick of #81279: Fix Azure client requests stuck issues on
k8s-ci-robot added a commit that referenced this pull request Aug 16, 2019
…79-upstream-release-1.13

Automated cherry pick of #81279: Fix Azure client requests stuck issues on
k8s-ci-robot added a commit that referenced this pull request Aug 16, 2019
…79-upstream-release-1.15

Automated cherry pick of #81279: Fix Azure client requests stuck issues on
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/cloudprovider area/provider/azure Issues or PRs related to azure provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants