[New Scheduler] Manage container creation by KeonHee · Pull Request #5074 · apache/openwhisk

KeonHee · 2021-03-02T16:09:35Z

Description

This module determines which invoker to create the container from.

My changes affect the following components

Types of changes

Bug fix (generally a non-breaking change which closes an issue).
Enhancement or new feature (adds new functionality).
Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

I signed an Apache CLA.
I reviewed the style guides and followed the recommendations (Travis CI will check :).
I added tests to cover my changes.
My changes require further changes to the documentation.
I updated the documentation where necessary.

codecov-io · 2021-03-02T17:02:33Z

Codecov Report

Merging #5074 (9f4708d) into master (cd6fded) will decrease coverage by 6.07%.
The diff coverage is 74.38%.

@@            Coverage Diff             @@
##           master    #5074      +/-   ##
==========================================
- Coverage   81.25%   75.17%   -6.08%     
==========================================
  Files         218      222       +4     
  Lines       10700    11010     +310     
  Branches      450      484      +34     
==========================================
- Hits         8694     8277     -417     
- Misses       2006     2733     +727

Impacted Files	Coverage Δ
.../apache/openwhisk/core/controller/Controller.scala	`54.80% <ø> (ø)`
...enwhisk/core/loadBalancer/InvokerSupervision.scala	`88.81% <ø> (-0.46%)`	⬇️
...che/openwhisk/core/loadBalancer/LoadBalancer.scala	`90.00% <ø> (+13.07%)`	⬆️
...e/loadBalancer/ShardingContainerPoolBalancer.scala	`80.79% <ø> (ø)`
...k/core/containerpool/v2/InvokerHealthManager.scala	`75.00% <ø> (-0.38%)`	⬇️
.../org/apache/openwhisk/core/connector/Message.scala	`64.76% <23.63%> (-14.22%)`	⬇️
.../main/scala/org/apache/openwhisk/core/WarmUp.scala	`64.28% <64.28%> (ø)`
...la/org/apache/openwhisk/common/TransactionId.scala	`94.25% <75.00%> (+0.13%)`	⬆️
...in/scala/org/apache/openwhisk/common/Message.scala	`77.77% <77.77%> (ø)`
...sk/core/scheduler/container/ContainerManager.scala	`86.76% <86.76%> (ø)`
... and 36 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd6fded...9f4708d. Read the comment docs.

KeonHee · 2021-04-06T18:33:17Z

The standalone test continues to fail because an error occurred while pulling openwhisk/action-nodejs-v10:nightly.

$ docker pull openwhisk/action-nodejs-v10:nightly
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

style95 · 2021-04-07T00:55:49Z

The standalone test continues to fail because an error occurred while pulling openwhisk/action-nodejs-v10:nightly.
$ docker pull openwhisk/action-nodejs-v10:nightly
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

I worked around this by pulling such kind images in advance before starting tests.

ningyougang · 2021-04-14T06:06:20Z

+    result = prime * result + tags.hashCode()
+    result = prime * result + dedicatedNamespaces.hashCode()
+    result
+  }


Regarding InvokerResourceMessage's equals and hashCode, it seems we have no need to override these 2 methods.

ningyougang · 2021-04-28T02:48:02Z

  confdir: "{{ config_root_dir }}/registry"

 kafka:
+  topicsPrefix: "{{ kafka_topics_prefix | default('') }}"


Add prefix for kafka topic feature is already implemented in this pr: #5062

style95

LGTM
@KeonHee

Can we add some design documents for this component especially regarding the container creation/deletion process and warmup process here?
https://cwiki.apache.org/confluence/display/OPENWHISK/Component+Design

KeonHee · 2021-04-29T06:23:02Z

@style95 Sure. I will upload it ASAP

KeonHee · 2021-05-06T07:45:38Z

I wrote a document: https://cwiki.apache.org/confluence/display/OPENWHISK/ContainerManager

style95 · 2021-05-06T07:57:23Z

I wrote a document: https://cwiki.apache.org/confluence/display/OPENWHISK/ContainerManager

Can we add a section about warmup?

codecov-commenter · 2021-05-07T02:42:59Z

Codecov Report

❌ Patch coverage is 84.49612% with 40 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.55%. Comparing base (9c445f3) to head (d4a43d6).
⚠️ Report is 206 commits behind head on master.

Files with missing lines	Patch %	Lines
...sk/core/scheduler/container/ContainerManager.scala	86.76%	27 Missing ⚠️
.../main/scala/org/apache/openwhisk/core/WarmUp.scala	64.28%	10 Missing ⚠️
...in/scala/org/apache/openwhisk/common/Message.scala	77.77%	2 Missing ⚠️
...la/org/apache/openwhisk/common/TransactionId.scala	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5074      +/-   ##
==========================================
- Coverage   81.11%   75.55%   -5.56%     
==========================================
  Files         214      225      +11     
  Lines       10913    11423     +510     
  Branches      469      501      +32     
==========================================
- Hits         8852     8631     -221     
- Misses       2061     2792     +731

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

KeonHee added the scheduler label Mar 2, 2021

KeonHee changed the title ~~[WIP] [New Scheduler] Manage container creation~~ [New Scheduler] Manage container creation Mar 22, 2021

KeonHee changed the title ~~[New Scheduler] Manage container creation~~ [WIP] [New Scheduler] Manage container creation Mar 23, 2021

KeonHee changed the title ~~[WIP] [New Scheduler] Manage container creation~~ [New Scheduler] Manage container creation Mar 23, 2021

ningyougang reviewed Apr 14, 2021

View reviewed changes

ningyougang reviewed Apr 28, 2021

View reviewed changes

style95 approved these changes Apr 29, 2021

View reviewed changes

KeonHee added 12 commits May 25, 2021 15:49

Manage container creation

adbe449

Add ContainerManager Test

ac9938d

Add dedicated namespace

82a9437

Remove namespace

020e6fa

Apply scala fmt

ad6c355

Add dedicatedNamespaces filter

0de9b51

Add dedicatedNamespaces test

13a35a2

Move InvokerState to common

c194857

Unify InvokerHealth message

cf1555f

Add configuration for test

5c965d5

Add license header

35262d4

Remove compare InvokerResourceMessage

d4a43d6

style95 merged commit f1829e1 into apache:master May 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[New Scheduler] Manage container creation#5074

[New Scheduler] Manage container creation#5074
style95 merged 12 commits into
apache:masterfrom
KeonHee:container-manager

KeonHee commented Mar 2, 2021 •

edited

Loading

Uh oh!

codecov-io commented Mar 2, 2021 •

edited

Loading

Uh oh!

KeonHee commented Apr 6, 2021

Uh oh!

style95 commented Apr 7, 2021

Uh oh!

ningyougang Apr 14, 2021

Uh oh!

ningyougang Apr 28, 2021 •

edited

Loading

Uh oh!

style95 left a comment

Uh oh!

KeonHee commented Apr 29, 2021

Uh oh!

KeonHee commented May 6, 2021

Uh oh!

style95 commented May 6, 2021

Uh oh!

codecov-commenter commented May 7, 2021 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

KeonHee commented Mar 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

My changes affect the following components

Types of changes

Checklist:

Uh oh!

codecov-io commented Mar 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

KeonHee commented Apr 6, 2021

Uh oh!

style95 commented Apr 7, 2021

Uh oh!

ningyougang Apr 14, 2021

Choose a reason for hiding this comment

Uh oh!

ningyougang Apr 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

style95 left a comment

Choose a reason for hiding this comment

Uh oh!

KeonHee commented Apr 29, 2021

Uh oh!

KeonHee commented May 6, 2021

Uh oh!

style95 commented May 6, 2021

Uh oh!

codecov-commenter commented May 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

KeonHee commented Mar 2, 2021 •

edited

Loading

codecov-io commented Mar 2, 2021 •

edited

Loading

ningyougang Apr 28, 2021 •

edited

Loading

codecov-commenter commented May 7, 2021 •

edited

Loading