Problem

I don't know if it's a plugin or fluentd issue, but fluent-plugin-elasticsearch 3.0.1 included in td-agent 3.3.0 don't seem to honor "chunk_limit_size" buffer setting. After upgrading, fluentd stopped flushing buffer after some time because of this error:

2019-01-29 15:53:40 +0100 [warn]: #0 [elasticsearch] failed to flush the buffer. retry_time=1 next_retry_seconds=2019-01-29 15:54:10 +0100 chunk="580997a747fa17474251b90ace8f29e8" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>"ece", :port=>9200, :scheme=>"http", :user=>"fluent", :password=>"obfuscated"}): [413] Request Content-Length 528883194 exceeds the configured limit of 209715200"

Despite the fact that chunk_limit_size is set to 32MB

BTW, we use elasticsearch enterprise and the proxy is hardcoded to 209715200.

Steps to replicate

Upgrade td-agent to 3.3.0 and send lots of log

Expected Behavior or What you need to ask

Flush log at 32MB max

Using Fluentd and ES plugin versions

• OS version: CentOS 7.6
• VM
• td-agent 3.0.3
• ES plugin 3.0.1
• ECE 2.0, ES 6.5.1