Introduce separate Akka dispatchers for CouchDB and Kafka Clients#2956
Introduce separate Akka dispatchers for CouchDB and Kafka Clients#2956bwmcadams wants to merge 4 commits into
Conversation
- Rest of system continues to use the default system dispatcher - Kafka and CouchDB clients each get their own dispatchers * This should prevent all of the akka based stuff from competing constantly for threads * Kafka and Couch can be tuned now based on individual needs - Dispatchers are defined in common `reference.conf`, so config can be overridden by dependents Refs apache#2954
|
PG3 1378 🔵 |
- Rest of system continues to use the default system dispatcher - Kafka and CouchDB clients each get their own dispatchers * This should prevent all of the akka based stuff from competing constantly for threads * Kafka and Couch can be tuned now based on individual needs - Dispatchers are defined in common `reference.conf`, so config can be overridden by dependents Refs apache#2954
markusthoemmes
force-pushed
the
wip-bwm-dispatcher-cleanup
branch
from
1b231ab to
cf3fa50
Compare
|
I took the liberty to rebase this, you'll need to override your local branch. |
| # Max number of threads to cap factor-based parallelism number to | ||
| parallelism-max = 10 | ||
| } | ||
| } |
There was a problem hiding this comment.
Are we at risk to starve this one? Kafka producing can be blocking and polling for sure is blocking although there should never be more than a couple of polls active.
There was a problem hiding this comment.
er, yeah. I think I copied this from docs; it looks like the default for Akka's default dispatchers is 64. I'll move it to that and we can always tweak specifically later.
- At some point we should investigate best default choices for OpenWhisk and fix Refs apache#2954
|
@markusthoemmes Changed to leave fork-join-executor configs at the Akka default, until we can choose better default tuning for the separated dispatchers. |
|
@bwmcadams, is this PR still valid? |
|
@dubeejw the intent is still there; I have to check the upstream changes though. Will update. |
markusthoemmes
added
needs-review
awaits-feedback
|
@bwmcadams I'll close this one for now, please reopen once you get to it. |
3 participants
reference.conf, so config can be overridden by dependentsCloses #2954