Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Total priority overflow check #45122

Conversation

ravisantoshgudimetla
Copy link
Contributor

@ravisantoshgudimetla ravisantoshgudimetla commented Apr 28, 2017

What this PR does / why we need it:

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #24720

Special notes for your reviewer:
@adohe. I have borrowed some parts of your code in the closed PR and created this one.

Release note:

This fixes the overflow for priorityconfig-  valid range {1, 9223372036854775806}.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Apr 28, 2017
@k8s-reviewable
Copy link

This change is Reviewable

@k8s-ci-robot
Copy link
Contributor

Hi @ravisantoshgudimetla. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with @k8s-bot ok to test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Apr 28, 2017
@k8s-github-robot k8s-github-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Apr 28, 2017
@ravisantoshgudimetla
Copy link
Contributor Author

/cc @jayunit100

@@ -39,3 +39,22 @@ func TestAlgorithmNameValidation(t *testing.T) {
}
}
}

func TestPriorityOverFlow(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also Add a unit test to confirm it doesnt overflow for valid conditions ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about that but I guess that should be part of other test that calls the checkOverFlow function. I don't have strong opinion on this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

definetly should have a test confirming it doesnt always overflow. that would be pretty bad :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack..

return configs, nil
}

// checkOverFlow checks if totalPriority can overflow after adding it with weight.
func checkOverFlow(totalPriority int, weight int) bool {
if totalPriority == MaxInt || weight == MaxInt {
Copy link
Member

@jayunit100 jayunit100 Apr 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • is this redundant to the weight check ? After all if totalPriority overflows, then totalPriority+ weight overflows as well in all cases, right ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once it overflows, no way it can exceed, so == should be fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but why check == if we're planning on checking below?

Copy link
Contributor Author

@ravisantoshgudimetla ravisantoshgudimetla Apr 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, by the time the function is called, one of the variables could have already at MaxInt and after that there is no point in doing an addition as it would throw a buffer overflow, so its kind of

  • Failing early.
  • The computation wouldn't cause an error.

Copy link
Member

@jayunit100 jayunit100 Apr 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Youre right regarding >=, ill delete that to not confuse people :) .

final thought is that IMO if this whole function can be replaced with return (total+weight)-weight != 0, it shouldn't be a function at all, b/c one line of code (excluding old school perl code golf examples) is usually better then 5.

will let you guys decide though...

var totalPriority int
for _, config := range configs {
// Checks for overflow and returns errors.
// TODO: Need to check overflow for weight*10.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

weight*10 is being checked below, right?

Copy link
Contributor Author

@ravisantoshgudimetla ravisantoshgudimetla Apr 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right but we need to have check for * but that's for the calling function.(Means if we have a unit test for getPriorityFunctionConfigs.)


func TestPriorityOverFlow(t *testing.T) {
var a, b int
a = 5000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call a total, and call b weight...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. The only thought that I had was, this could be used anywhere, so did not name variables to reflect weight and total.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually just delete a/b we dont need them do we

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok..

return true
}
sum := totalPriority + weight
return sum-totalPriority != weight
Copy link
Member

@jayunit100 jayunit100 Apr 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this is just return totalPriority+weight-totalPriority !=weight i think thats pretty much always true... ? in this universe ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right but for readability..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha, so its overflow i see whats going on here now

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just make it oneline, and add a comment about how in go we have to test for int overflow additions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha makes sense now

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'd prefer

return (totalPri + weight)-totalPriority != weight
``` since that makes it obvious that your testing the absurd boundary overflow.  but i guess just an opinion. 

return configs, nil
}

// checkOverFlow checks if totalPriority can overflow after adding it with weight.
func checkOverFlow(totalPriority int, weight int) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we should call this function checkIntegerOverflow

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, makes sense

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checkPriorityOverflow ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess, if we want to use this in someother location, its better to generalize the name to checkIntegerOverflow as this function can check for any 2 int's overflow.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if so, I'd like to move it to util and export it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or if the check is simple enough, this func is not necessary. But we need to add unit test for priority config check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check may be simple enough, my only concern is I have to write unit tests for getPriorityFunctionConfigs, which is kind of orthogonal to issue. IMO, the best thing to do is leave this function as is here now and move it to utils once we find other functions that can use this. I am open to other suggestions too.

@jayunit100
Copy link
Member

I think this mostly makes sense, minor comments b/c my head spins when thinking about overflow. Also I don't know the background of the priority normalization goals, so best if someone else finishes the review... @kubernetes/sig-scheduling-pr-reviews .

@k8s-ci-robot k8s-ci-robot added the sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. label Apr 28, 2017
@ravisantoshgudimetla ravisantoshgudimetla changed the title Total priority buffer overflow check Total priority overflow check Apr 28, 2017
@jayunit100
Copy link
Member

@k8s-bot test this please

if totalPriority == MaxInt || weight == MaxInt {
return true
}
sum := totalPriority + weight
Copy link
Member

@k82cn k82cn Apr 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Go, I'm not sure whether sum is still a valid value after overflow; in C, the sum maybe randoms after overflow (e.g. checking by interrupt or others) in some arch.

So I'd prefer to

return weight >= MAX_INT || totalPriority >= MAX_INT || weight >= MAX_INT - totalPriority

or just weight > MAX_INT - totalPriority

(when weight will > MAX_INT?)

Copy link
Contributor Author

@ravisantoshgudimetla ravisantoshgudimetla Apr 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(when weight will > MAX_INT?)

Thats pretty hypothetical case but this logic in general seems concise. I will use this. Thanks @k82cn. Just to be clear, since weight is always +ve.

for _, config := range configs {
// Checks for overflow and returns errors.
// TODO: Need to check overflow for weight*10.
if checkOverFlow(totalPriority, config.Weight*10) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like make 10 as MaxPriority.

@k82cn
Copy link
Member

k82cn commented Apr 28, 2017

the background of the priority normalization goals

My understand is to make priority comparable, e.g. we can not compare 1 cpu and 100Mb imageSize.

@ravisantoshgudimetla ravisantoshgudimetla force-pushed the priority_overflow#24720 branch 2 times, most recently from e73dea4 to d6c6ff8 Compare April 29, 2017 00:38
@ravisantoshgudimetla
Copy link
Contributor Author

@k82cn @jayunit100 , PTAL. I have addressed most of comments except for location of this function. LMK, if you have any other comments.

@@ -87,6 +87,9 @@ var (

const (
DefaultProvider = "DefaultProvider"
MaxUint = ^uint(0)
Copy link
Member

@jayunit100 jayunit100 Apr 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we externalized these or add MaxUint and MaxInt to math.MaxInt math.MaxUInt ? I'm pretty sure we can, and If not, we should add them to a util somewhere.

var totalPriority int
for _, config := range configs {
// Checks for overflow and returns errors.
// TODO: Need to check overflow for weight*MaxPriority.
Copy link
Member

@jayunit100 jayunit100 Apr 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets accomplish this TODO in this PR, ... just add a third arg to your overflow function below, so that it checks everything for us.....

if totalPriority, err := addPrioritySafely(totalPriority, config.Weight, MaxPriority); err != nil {
   return nil, fmt.Errorf("Total priority of priority functions has overflown %v", err)
}

Copy link
Member

@k82cn k82cn Apr 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree that we should handle overflow for weight*MaxPriority instead of TODO.

And I'd suggest to move this check to plugin/pkg/scheduler/api/validation/validation.go; as it also check weight against negative number.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha this simple PR is getting pretty interesting :). Yeah, k82cn's idea is reasonable, if we can move this thing into a generic place (make sure to create corresponding unit test) thats great.

return configs, nil
}

// checkIntegerOverFlow checks if totalPriority can overflow after adding it with weight.
func checkIntegerOverFlow(totalPriority int, weight int) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, sorry, now that I better understand whats going on here, im going to suggest you call this addPrioritySafely ,

  • Check weight*Maxpriority multiplies w/o overflow.
  • then check that totalPriority increments w/o overflow.
  • Return error if fails.
  • Otherwise return the new added on value of totalPriority

That will be super easy to understand. Sorry about changing my mind :).

Copy link
Contributor Author

@ravisantoshgudimetla ravisantoshgudimetla Apr 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jayunit100 , @k82cn, I am fine with making those changes but AFAIU, it seems weights can never overflow, even if it overflows, shouldn't we check them in functions where they were getting updated?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just suggesting that the entire addPriority... operation should handle the TODO as well as the other stuff all in one place, and return a new value representing the result of the operation. That makes it really easy to read, unit test, and so on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you but the overflow can happen at other places as well where weight is getting updated. So, I was suggesting that we should have checks there instead of here, as this is the last step in calculation. (I meant, we need to fail earlier than this.).

@ravisantoshgudimetla
Copy link
Contributor Author

@k82cn, @jayunit100 PTAL. I moved the check to validation.go. Now, it checks for weight*10 overflow as well. The function could have been much more concise but it may effect readability. LMK, your thoughts on it.

@k82cn
Copy link
Member

k82cn commented Apr 30, 2017

sorry for front and rear, here's the diff in my mind (just code style, no functional comments):

  1. define const in scheduler.api, e.g. MaxPriority; as I'd like also to replace
    the magic number (10) in scheduler.
  2. in validation.go, I'd like to only check the range of weight (0 ~ MaxInt/10)
  3. add a release note for the scope/range of weight
  4. add the validation into a new func, named validateSelectedConfigs
  5. add test against getPriorityFunctionConfigs or validateSelectedConfigs instead
    of overflow func

here's a draft diff for your reference: https://gist.github.com/k82cn/ee104da32ae615c0458fdcecc693e918

@ravisantoshgudimetla
Copy link
Contributor Author

@k82cn : Thanks for the feedback. Please see my responses inline.

define const in scheduler.api, e.g. MaxPriority; as I'd like also to replace
the magic number (10) in scheduler.

Reasonable. I can make the change.

in validation.go, I'd like to only check the range of weight (0 ~ MaxInt/10)

The problem here is if you compute MaxInt/10 and multiply it with 10, we will not get MaxInt back.

add a release note for the scope/range of weight

Sure..

add the validation into a new func, named validateSelectedConfigs

TBH, I did not think about this approach as I took some piece of code from the closed PR. But, after looking at the code base, it seems that we are doing a lazy evaluation of priority weights. The idea that I had is to fail fast and early and wanted to change the code(in another PRs) to reflect that. But, if you and others insist, I can leave this as is and write the validateSelectedConfigs func.

add test against getPriorityFunctionConfigs or validateSelectedConfigs instead
of overflow func.

I wanted to avoid writing test cases for getPriorityFunctionConfigs as that is orthogonal to the current set of changes and there is another issue that you are shepherding to improve test suite coverage. Writing test case for validateSelectedConfigs, if we choose the above method seems reasonable.

@k82cn
Copy link
Member

k82cn commented May 1, 2017

in validation.go, I'd like to only check the range of weight (0 ~ MaxInt/10)

The problem here is if you compute MaxInt/10 and multiply it with 10, we will not get MaxInt back.

If so, then MaxInt/10 - 1, that's enough range for weight; the major suggestion is to define the range/scope to be exact number instead of a description.

TBH, I did not think about this approach as I took some piece of code from the closed PR.

What's the gap? it just extra a bit of code into a func.

add test against getPriorityFunctionConfigs or validateSelectedConfigs instead
of overflow func.

I wanted to avoid writing test cases for getPriorityFunctionConfigs as that is orthogonal to the current set of changes and there is another issue that you are shepherding to improve test suite coverage. Writing test case for validateSelectedConfigs, if we choose the above method seems reasonable.

I think we did not cover the major logic in getPriorityFunctionConfigs; IMO, the CheckPriorityOverflow is small enough without unit test.
Regarding code coverage, I'd suggest not to reduce it when new PRs merged; otherwise, that issue will keep open forever :).

@jayunit100
Copy link
Member

jayunit100 commented May 1, 2017

ill defer to k82cn for the rest of the changes, this is a simple PR so don't need too many cooks :).

@grodrigues3
Copy link
Contributor

@k8s-bot ok to test

@k8s-ci-robot k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label May 1, 2017
@timothysc timothysc added this to the v1.7 milestone May 2, 2017
@k82cn
Copy link
Member

k82cn commented May 5, 2017

ping @ravisantoshgudimetla :).

@ravisantoshgudimetla
Copy link
Contributor Author

@k82cn, Apologies for delay. I will update this PR in next 2-3 days.

@k82cn
Copy link
Member

k82cn commented May 6, 2017

@ravisantoshgudimetla , thanks very much :).

@ravisantoshgudimetla
Copy link
Contributor Author

@k82cn, I have addressed most of your comments. LMK, if you want any other changes.

@@ -29,8 +29,8 @@ func ValidatePolicy(policy schedulerapi.Policy) error {
var validationErrors []error

for _, priority := range policy.Priorities {
if priority.Weight <= 0 {
validationErrors = append(validationErrors, fmt.Errorf("Priority %s should have a positive weight applied to it", priority.Name))
if priority.Weight <= 0 || priority.Weight >= schedulerapi.MaxWeight {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer to priority.Weight > schedulerapi.MaxWeight, it'll align with checking against MaxTotalPriority.

@k82cn
Copy link
Member

k82cn commented May 8, 2017

/lgtm

That's great; minor comments.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 8, 2017
@k8s-github-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: k82cn, ravisantoshgudimetla
We suggest the following additional approver: timothysc

Assign the PR to them by writing /assign @timothysc in a comment when ready.

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@timothysc timothysc added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 8, 2017
@k8s-github-robot
Copy link

Automatic merge from submit-queue (batch tested with PRs 44727, 45409, 44968, 45122, 45493)

@k8s-github-robot k8s-github-robot merged commit a3cf8b9 into kubernetes:master May 8, 2017
@ravisantoshgudimetla ravisantoshgudimetla deleted the priority_overflow#24720 branch May 13, 2017 22:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Total priority can overflow
8 participants