Add OOMScoreAdj to the moby API #47950

psaintlaurent · 2024-06-11T16:30:17Z

- What I did
Add OOMScoreAdj to the moby API

- How I did it
I modified the executor, convertor and API to accept OOMScoreAdj

- How to test it
Using the swarmkit PR
moby/swarmkit#3177

Start moby using make BIND_DIR=. DOCKER_DEBUG=1 DELVE_PORT=127.0.0.1:2345:2345 shell
make install
./hack/make.sh run

Use curl to query agains the API to verify the changes or you can use the docker CLI from

thaJeztah

Thanks! I left some comments; can you also add a bullet under the v1.46 changes in the API "version history"? https://github.com/moby/moby/blob/master/docs/api/version-history.md

As this affects the swarm API endpoints, it would be changing multiple endpoints (at least "create" and "update". You can find some prior examples of such changes, e.g. like;

POST /services/create and POST /services/{id}/update now accept Seccomp and AppArmor fields in the ContainerSpec.Privileges object. This allows some configuration of Seccomp and AppArmor in Swarm services.

thaJeztah · 2024-06-11T19:48:42Z

api/common.go

 const (
 	// DefaultVersion of the current REST API.
-	DefaultVersion = "1.46"
+	DefaultVersion = "1.47"


APi 1.46 has not yet been released (but will be with docker v27.0.0) so you can keep it at the current version.

thaJeztah · 2024-06-11T19:49:49Z

client/service_create.go

 	"context"
 	"encoding/json"
 	"fmt"
+	"github.com/opencontainers/go-digest"


looks like an accidental change

thaJeztah · 2024-06-11T19:51:28Z

daemon/cluster/executor/container/container.go

 		Sysctls:        c.spec().Sysctls,
 		CapAdd:         c.spec().CapabilityAdd,
 		CapDrop:        c.spec().CapabilityDrop,


Not for this PR, but looking at this list, I'm wondering if there's other properties that are actually wired up, but that were forgotten to be included in the swagger.

If that's the case, we should fix that 🤔

thaJeztah · 2024-06-11T19:52:41Z

docs/api/v1.47.yaml

See my other comment; we can include these changes in v1.46 (which didn't ship yet); we'll create the v1.46.yaml short before release.

thaJeztah · 2024-06-11T19:59:32Z

This probably also needs to be gated by API version (both for create and update ; similar to

moby/api/server/router/swarm/cluster_routes.go

Lines 227 to 233 in 87794b3

    
           version := httputils.VersionFromContext(ctx) 
        
           if versions.LessThan(version, "1.44") { 
        
           	if service.TaskTemplate.ContainerSpec != nil && service.TaskTemplate.ContainerSpec.Healthcheck != nil { 
        
           		// StartInterval was added in API 1.44 
        
           		service.TaskTemplate.ContainerSpec.Healthcheck.StartInterval = 0 
        
           	} 
        
           }

thaJeztah · 2024-06-13T15:04:26Z

I see you made some updates, but at least one commit is missing a DCO sign-off.

It's probably fine for all commits to be in a single commit; can you squash the commits and add the sign-off? (otherwise CI won't run 😅)

thaJeztah · 2024-06-13T18:43:18Z

docs/api/version-history.md

  `--sysctl` settings for `eth0` will be migrated to `DriverOpts` when possible. 
  This automatic migration will be removed for API versions 1.47 and greater.
+
+* `POST /services/create` now supports OomScoreAdj


Can you mention the update endpoint as well?

I only made modifications for create not update.

I think the same type is used for updating services as well; or at least, I think any property that can be set when creating a service must also be able to be changed (updateable) for the service (and swarmkit will reconcile)

I think @thaJeztah has a point. Look at the most recent PR to extend the POST /services/create endpoint prior to this one: #46386. It affects both create and update. IIRC, when a Swarm service is updated, the Swarm scheduler reconciles by replacing all running tasks with new tasks that follow the updated task template in the service spec.

api/server/router/swarm/cluster_routes.go

api/swagger.yaml

corhere · 2024-06-13T20:05:39Z

api/types/swarm/container.go

 	CapabilityAdd  []string            `json:",omitempty"`
 	CapabilityDrop []string            `json:",omitempty"`
 	Ulimits        []*units.Ulimit     `json:",omitempty"`
+	OomScoreAdj    int64               `json:",omitempty"`


I'm torn. On the one hand, coding style dictates that this field should be spelled OOMScoreAdj on account of "OOM" being an initialism for "out of memory." On the other hand, we have existing precedent in the API for this spelling, with (container.HostConfig).OomScoreAdj.

moby/api/types/container/hostconfig.go

Line 443 in ff5cc18

OomScoreAdj int // Container preference for OOM-killing

I am okay with either spelling. Anyone have a strong opinion one way or the other?

Match the existing API field name. No reason to make Swarmkit any more different than it already is.

Yeah, I hate the Oom, but agree if it's matching the engine API, it's fine.

client/service_create.go

corhere · 2024-06-14T18:56:55Z

api/swagger.yaml

                  Shared: true
                  Size: 0
-                  CreatedAt: "2021-06-28T13:31:01.474619385Z"
+                  CreatedAt: "2021-06-28T13:31:01.464619385Z"


Why did this line change?

That's a typo, I'l be more careful about proofreading.

corhere · 2024-06-14T19:32:40Z

api/server/router/swarm/cluster_routes.go

+	if versions.LessThan(version, "1.46") {
+		if service.TaskTemplate.ContainerSpec != nil && service.TaskTemplate.ContainerSpec.OomScoreAdj != 0 {
+			// OomScoreAdj was added in API 1.46
+			service.TaskTemplate.ContainerSpec.OomScoreAdj = 0
+		}
+	}


Is this function the right place for this code? In #46386 the equivalent change was made in func adjustForAPIVersion, which is called by both func createService and func updateService. (I think above code block for Healthcheck.StartInterval is also misplaced.)

I'm not certain, I'm following the pattern that already exists in that function. I haven't read the entire codebase yet.

I asked @dperny: unfortunately, the pattern you copied was buggy code. I have opened a PR to address that deficiency. #47991

You will need to move the gating of OomScoreAdj by API version into func adjustForAPIVersion so that it applies to both create and update operations. And you will have to update version-history.md to mention that support was added to both the create and update API endpoints.

I read through that code and debugged it manually earlier. It works as intended but I have to infer the intended behavior through existing code patterns as there is no documentation to work off with intended behavior.

Will changing the location make a difference other than for update operations as I only worked on create.

dperny · 2024-06-17T15:32:57Z

api/types/swarm/container.go

 	CapabilityAdd  []string            `json:",omitempty"`
 	CapabilityDrop []string            `json:",omitempty"`
 	Ulimits        []*units.Ulimit     `json:",omitempty"`
+	OomScoreAdj    int64               `json:",omitempty"`


Match the existing API field name. No reason to make Swarmkit any more different than it already is.

dperny · 2024-06-17T15:34:40Z

daemon/cluster/convert/container.go

 		CapabilityAdd:  c.CapabilityAdd,
 		CapabilityDrop: c.CapabilityDrop,
 		Ulimits:        ulimitsToGRPC(c.Ulimits),
+		OomScoreAdj:    c.OomScoreAdj,


This covers containerToGRPC, but you need a matching back-conversion in containerSpecFromGRPC. Without the back-conversion, the client cannot correctly read out the ServiceSpec.

Thanks, this was an IDE issue. Goland just removed code without notice, I have to turn off a few options.

Signed-off-by: plaurent <[email protected]>

thaJeztah · 2024-06-17T20:52:31Z

vendor/github.com/moby/swarmkit/v2/api/specs.pb.go

 	// Ulimits defines the list of ulimits to set in the container. This option
 	// is equivalent to passing --ulimit to docker run.
 	Ulimits []*ContainerSpec_Ulimit `protobuf:"bytes,29,rep,name=ulimits,proto3" json:"ulimits,omitempty"`
+	// OOmScoreAdj defines the relative value used for destroying a container during an OOM


Looks like the comment here doesn't match the casing of the actual field.

This is vendored code, so needs to be fixed in SwarmKit (and definitely not a blocker 😅)

I'll fix that in the morning, thanks!

thaJeztah · 2024-06-17T20:54:29Z

vendor/github.com/moby/swarmkit/v2/api/types.proto

 	// to the container OS's default - generally 60, or the value predefined in
 	// the image; set to -1 to unset a previously set value
 	google.protobuf.Int64Value memory_swappiness = 4;
+


Wondering where the empty lines here originated from, or is this some change in the code generating these? (just curious where the changes came from)

thaJeztah

LGTM

psaintlaurent mentioned this pull request Jun 11, 2024

Add OomScoreAdj to "docker service create" and "docker stack" docker/cli#5145

Merged

3 tasks

thaJeztah requested changes Jun 11, 2024

View reviewed changes

thaJeztah reviewed Jun 13, 2024

View reviewed changes

psaintlaurent force-pushed the ENGINE-903 branch 2 times, most recently from c559d59 to c9897ea Compare June 13, 2024 20:02

corhere requested changes Jun 13, 2024

View reviewed changes

psaintlaurent force-pushed the ENGINE-903 branch 4 times, most recently from ca7d893 to 9f989d6 Compare June 13, 2024 21:02

corhere marked this pull request as draft June 13, 2024 21:58

psaintlaurent marked this pull request as ready for review June 14, 2024 17:04

psaintlaurent marked this pull request as draft June 14, 2024 17:05

psaintlaurent force-pushed the ENGINE-903 branch from 9f989d6 to a0d343a Compare June 14, 2024 18:16

corhere reviewed Jun 14, 2024

View reviewed changes

psaintlaurent force-pushed the ENGINE-903 branch 2 times, most recently from 6f976c7 to 5ff48d0 Compare June 17, 2024 12:45

dperny suggested changes Jun 17, 2024

View reviewed changes

Add OomScoreAdj options

b640582

Signed-off-by: plaurent <[email protected]>

psaintlaurent force-pushed the ENGINE-903 branch from 5ff48d0 to b640582 Compare June 17, 2024 16:36

psaintlaurent marked this pull request as ready for review June 17, 2024 18:49

corhere approved these changes Jun 17, 2024

View reviewed changes

corhere requested a review from thaJeztah June 17, 2024 20:21

thaJeztah reviewed Jun 17, 2024

View reviewed changes

thaJeztah approved these changes Jun 17, 2024

View reviewed changes

thaJeztah merged commit 4ea464d into moby:master Jun 17, 2024

thaJeztah added this to the 27.0.0 milestone Jun 17, 2024

thaJeztah added area/api API kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. impact/api impact/changelog area/swarm labels Jun 17, 2024

This was referenced Jun 18, 2024

vendor: github.com/moby/swarmkit/v2 v2.0.0-20240611172349-ea1a7cec35cb docker/cli#5169

Merged

vendor: github.com/docker/docker 1a1f3cff45ec (master, v27.0-dev) docker/cli#5170

Merged

thaJeztah mentioned this pull request Jul 19, 2024

[27.1 backport] Add OomScoreAdj to "docker service create" and "docker stack" docker/cli#5274

Merged

3 tasks

Add OOMScoreAdj to the moby API #47950

Add OOMScoreAdj to the moby API #47950

Uh oh!

Conversation

psaintlaurent commented Jun 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Jun 11, 2024

Uh oh!

thaJeztah commented Jun 13, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

psaintlaurent Jun 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

psaintlaurent Jun 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

psaintlaurent commented Jun 11, 2024 •

edited

Loading

psaintlaurent Jun 13, 2024 •

edited

Loading

psaintlaurent Jun 16, 2024 •

edited

Loading