-
Notifications
You must be signed in to change notification settings - Fork 308
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cpu usage is high in 3.4.2 but not high in 3.1.1 #584
Comments
I think that the following line causes high cpu usage:
But this line is introduced to resolve batch size exceeded issue. ref: #535 I'm considering how to reduce cpu usage in this line. |
I think that #586 patch can reduce CPU usage with the following configuration: bulk_message_request_threshold -1 Could you check it? |
Gemfile
fluentd conf
It worked well |
Problem
The CPU usage is high, about 75%-100% in 3.4.2 but not high in 3.1.1
Steps to replicate
data flow is here
kafka -> kafka-fluentd-consumer -> fluend v1 -> fluent-plugin-elasticsearch -> elasticsearch
fluentd conf
stackprof
Expected Behavior or What you need to ask
if we use 3.1.1(in other words, without #539)
cpu usage is not high, about 25%
I suspect
size
because it's similar to fluent/fluentd#1801 and komamitsu/fluency#85Any opinion?
Using Fluentd and ES plugin versions
CentOS Linux release 7.6.1810 (Core)
Bare Metal
1.4.2
3.4.2
Gemfile
7.0.1
The text was updated successfully, but these errors were encountered: