Fluentd Blog

fluent-package v5.0.5 has been released

2024-11-08

Hi users!

We have released fluent-package v5.0.5. fluent-package is a stable distribution package of Fluentd. (successor of td-agent v4)

This is a maintenance release of v5.0.x LTS series. As bundled Ruby was updated to 3.2.6 and Fluentd was updated to 1.16.6. We recommend upgrading to fluent-package v5.0.5!

Changes from fluent-package v5.0.4

In this release, focused on updating bundled ones.

Update ruby to 3.2.6
Update fluentd to 1.16.6
Update bundled gems overall
Solved possibility of DoS through "NoMemoryError"
msi: set GEM_HOME/GEM_PATH in fluentd.bat
Fixed SIGABORT error with fluent-plugin-systemd

Solved possibility of DoS through "NoMemoryError"

In the previous versions of msgpack gem, there was a possibility that cause DoS (Denial Of Service) when crafted message was sent to Fluentd.

When that message was sent, it might cause "NoMemoryError" on Fluentd because there is a case that msgpack try to pre-allocate huge amount of memories.

Above behavior was fixed not to pre-allocate over 32k entries (limit pre-allocated amount of memories) in msgpack 1.7.3 and bundled into fluent-package v5.0.5.

msi: set `GEM_HOME`/`GEM_PATH` in fluentd.bat

In the previous versions, fluentd.bat doesn't set GEMHOME/GEMPATH explicitly.

If users set custom GEM_HOME / GEM_PATH environment variables, the batch file will not work as expected. Therefore, this patch will set the appropriate values in the batch file.

Above bug was also fixed in fluent-package v5.0.5.

Fixed SIGABORT error with fluent-plugin-systemd

In this release, the bundled fluent-plugin-systemd was updated to 1.1.0. That release fixed SIGABORT error with inconsistency of memory allocator handling.

Download

Please see the download page.

About next LTS schedule

We plan to ship the next LTS version of fluent-package v5.0.6 on March, 2025. The content of updates are still in T.B.D.

End of support for td-agent v4, let's migrate to fluent-package

As it was already announced Drop schedule announcement about EOL of Treasure Agent (td-agent) 4, td-agent v4 was reached EOL in Dec, 2023.

After reached EOL, td-agent v4.5.3 on Windows was released because there was a crash bug during startup on Windows. It was backported fix from fluent-package v5 as it is critical in some case. Even though this was a exceptional maintenance release, but there is no change to the fact that we already stopped maintaining td-agent v4.

We strongly recommend migrating from td-agent v4 to fluent-package v5 (LTS). See Upgrade to fluent-package v5

Scheduled support lifecycle announcement about Fluent Package v6

2024-08-29

Hi users!

We had launched fluent-package v5 series last year, recently shipped v5.0.4 in LTS release channel and v5.1.0 in normal release channel.

In this blog article, we explains the planned next major updates - v6.0.0.

When the next LTS (Long Term Support) version available?

In short, we will ship fluent-package v6 in Aug, 2025.

We keeps two release channels as follows:

Normal release
LTS (Long Term Support)

Here is the difference of these channels.

Normal release (6.x series)
- Recommended for early adopters (eager to try new features of Fluentd)
- Rapid release development (planned semi-annually releases, but it may vary.)
- Fluentd will be upgraded occasionally
  - Minor upgrade will be applied. e.g. 1.18 => 1.19 and so on.
  - e.g. v6.0.1 (Fluentd v1.18.1), v6.0.2 (Fluentd v1.18.2), ... v6.1.0 (Fluentd v1.19.0)
- The only latest version will be supported (bug fix will not be backported into older version)
LTS (Long Term Support, 6.0.x series)
- Recommended for enterprise services
- Security and bug fix release only
- Fluentd will be upgraded only in the teeny version and will stick to 1.18.x series. (T.B.D.)
  - Thus, the version number will be 6.0.x. (6.1 series will not be shipped for LTS channel)
  - e.g. v6.0.1 (Fluentd v1.18.1), v6.0.2 (Fluentd v1.18.2), v6.0.3 (Fluentd v1.18.3) ... v6.0.x (Fluentd v1.18.x) and so on.
- Next major version will be fluent-package v7.0.0, and will be shipped in 2027 (T.B.D.).

As for fluent-package v6,

fluent-package v5 LTS will be supported until Dec, 2025.
- We expect users upgrade from v5 to v6 during the grace period.
Until 6.1.0 is released, normal release channel and LTS channel are the same package.
Upgrade from v4 or older version is not supported. Upgrade to v5 first, then upgrade it to v6.

Current releases and future plans:

Release Version	Release Date	End of life
v5.0.x (LTS)	Aug, 2023	Dec, 2025
v6.0.0 (LTS)	Aug, 2025	Dec, 2027 (T.B.D.)
v7.0.0 (LTS)	Aug, 2027 (T.B.D.)	T.B.D.

Happy logging!

Fluentd v1.16.6 has been released

2024-08-20

Hi users!

We have released v1.16.6 on 2024-08-20. ChangeLog is here.

This release is a maintenance release of v1.16 series. In this release, some bugs were fixed with backporting from v1.17.x.

Bug Fixes

`yaml_parser`: `$log_level` notation in YAML was supported

In this release, we added a support for $log_level in YAML config file syntax.

In the previous versions, if you used log_level in YAML configuration, it causes a following warning:

[warn]: #0 'log_level' is deprecated parameter name. use '@log_level' instead.

But @log_level is invalid in YAML format. In YAML format, it should be $log_level as same as similar parameter such as $tag and $type, but not supported before.

Since Fluentd v1.16.6, $log_level has been supported! (In v1.17.x, it was supported since v1.17.1)

fluentd command: Fix `--plugin` (-p) option not to overwrite default value

In this release, we have fixed a bug that fluentd command: --plugin (-p) option could not overwrite default value.

This bug was unexpectedly introduced since v1.16.0.

Now it was reverted to the original behavior.

logger: Fix LoadError with console gem v1.25

In this release, we fixed a LoadError with console gem v1.25 or later.

Since console gem v1.25 or later, it's internal change causes a LoadError in Fluentd. This error breaks existing plugin such as fluent-plugin-prometheus which provides monitoring REST API.

500 response when scraping prometheus metrics in kubernetes with istio service mesh

With supporting console gem v1.25 or later, this issue has solved!

Enjoy logging!

Fluentd v1.17.1 has been released

2024-08-19

Hi users!

We have released v1.17.1 on 2024-08-19. ChangeLog is here.

This release is a new release of v1.17 series. In this release, we added some new options for some plugins and fixed bugs of Parser.

Enhancement

`yaml_parser`: `$log_level` notation in YAML was supported

In this release, we added a support for $log_level in YAML config file syntax.

In the previous versions, if you used log_level in YAML configuration, it causes a following warning:

[warn]: #0 'log_level' is deprecated parameter name. use '@log_level' instead.

But @log_level is invalid in YAML format. In YAML format, it should be $log_level as same as similar parameter such as $tag and $type, but not supported before.

Since Fluentd v1.17.1, $log_level has been supported now!

`out_http`: `compress` option was supported

In this release, we added a new option compress for the out_http plugin.

You can compress HTTP request body by specifying compress gzip. To keep compatibility, the default configuration is set as compress text (no compression).

This feature was contributed by @rockliffelewis. Thanks!

`in_sample`: the default behavior was changed

In this release, we changed the default behavior of in_sample plugin and added a new option reuse_record for it.

The default behavior has changed to copy sample data to avoid the impact of destructive changes by subsequent plugins.

In the previous versions, when in_sample plugin is used with the subsequent filter parser plugin which uses remove_key_name_field, it raises the following error repeatedly.

#0 dump an error event: error_class=ArgumentError error="message does not exist"

This error occurs when key_name and remove_key_name_field removes key from record with destructive change in filter parser. It affects generated sample data after twice or later. (This error is not limited to filter parser plugin. The similar destructive change will be done with the subsequent plugins, this error is reproducible)

The new default behavior (reuse_record false) avoids this problem, but it increases the load when generating large amounts of sample data. You can use this new parameter to have the same performance as before with reuse_record.

Bug Fixes

logger: Fix LoadError with console gem v1.25

In this release, we fixed a LoadError with console gem v1.25 or later.

Since console gem v1.25 or later, it's internal change causes a LoadError in Fluentd. This error breaks existing plugin such as fluent-plugin-prometheus which provides monitoring REST API.

500 response when scraping prometheus metrics in kubernetes with istio service mesh

With supporting console gem v1.25 or later, this issue has solved!

Enjoy logging!

fluent-package v5.1.0 has been released

2024-08-02

Hi users!

We have released fluent-package v5.1.0. fluent-package is a stable distribution package of Fluentd. (successor of td-agent v4)

This is a maintenance release of v5.x series. Bundled ruby version was upgraded to 3.2.5 and fluentd v1.17.0 was shipped!

Changes from fluent-package v5.0.4

In this release, focused on updating bundled ones.

Update fluentd to 1.17.0
Update ruby to 3.2.5
Update bundled gems overall

Update bundled components overall

In LTS version, conservative updating policy is enabled, so updating gem is very limited to bug fix or security fix. In contrast to LTS version, v5.1.0 bundles more recent components because of standard version.

As we already announced in Scheduled support lifecycle announcement about Fluent Package blog article, in normal release channel, we will ship the latest version (v1.17.0) of Fluentd.

So if you want to try using latest version of Fluentd, v5.1.0 is one for you. If you use fluent-package in enterprise services, keep using fluent-package 5.0.x (LTS).

Note:In the previous versions, there is no different in normal release channel and LTS release channel. But from this release, v5.1.x will be shipped in normal release channel. v5.0.x will be shipped in LTS channel.

Download

Please see the download page.

About next LTS schedule

We plan to ship the next LTS version of fluent-package v5.0.5 on Oct, 2024. The content of updates are still in T.B.D.

End of support for td-agent v4, let's migrate to fluent-package

As it was already announced Drop schedule announcement about EOL of Treasure Agent (td-agent) 4, td-agent v4 was reached EOL in Dec, 2023.

We strongly recommend migrating from td-agent v4 to fluent-package v5 (LTS). See Upgrade to fluent-package v5

fluent-package v5.0.4 has been released

2024-07-02

Hi users!

We have released fluent-package v5.0.4. fluent-package is a stable distribution package of Fluentd. (successor of td-agent v4)

This is a maintenance release of v5.0.x LTS series. As bundled Ruby was updated to 3.2.4 and a foolproof mechanism was implemented to prevent launching duplicated Fluentd instances, we recommend upgrading to fluent-package v5.0.4!

Changes from fluent-package v5.0.3

Update ruby to 3.2.4 (#645)
Fixed to prevent launching Fluentd wrongly if the service is already running (#648,#649)
Added support for Ubuntu 24.04 (Noble Numbat)

Note: In this release, dropped CentOS 7 package because of EOL.

Fixed to prevent launching Fluentd wrongly if the service is already running

In this release, a foolproof mechanism was implemented to prevent launching Fluentd wrongly if the service is already running.

As you know, you can check the version of Fluentd with fluentd --version, but there is a case that fluentd -v is executed wrongly to do it.

When already running Fluentd as a service, fluentd -v launches a duplicated Fluentd instance with the same fluentd configuration. If you launch duplicated Fluentd instances, it causes the corruption of processing Fluentd buffer. To prevent such a situation, a foolproof was implemented now.

For example, if Fluentd is running as a service, launching Fluentd causes an error to block it.

Here is the example on Windows:

> fluentd
Error: Can't start duplicate Fluentd instance with the default config.

To start Fluentd, please do one of the following:
  (Caution: Please be careful not to start multiple instances with the same config.)
  - Stop the Fluentd Windows service 'fluentdwinsvc'.
  - Specify the config path explicitly by '-c' ('--config').

Even though if you wrongly launch Fluentd to check version with -v (It should be --version to show version), then it causes the following error.

> fluentd -v
Error: Can't start duplicate Fluentd instance with the default config.

To take the version, please use '--version', not '-v' ('--verbose').

To start Fluentd, please do one of the following:
  (Caution: Please be careful not to start multiple instances with the same config.)
  - Stop the Fluentd Windows service 'fluentdwinsvc'.
  - Specify the config path explicitly by '-c' ('--config')

Note that this foolproof feature is intended to block launching duplicated Fluentd instance, you can explicitly launch Fluentd by specifying a specific option to pass it even though already Fluentd is running as a service.

On Windows:

-c (--config)
--dry-run
--reg-winsvc
--reg-winsvc-fluentdopt
--show-plugin-config

On Linux:

-c (--config)
--dry-run
--show-plugin-config

About next LTS schedule

We plan to ship the next LTS version of fluent-package v5.0.5 on Oct, 2024. The content of updates are still in T.B.D.

End of support for td-agent v4, let's migrate to fluent-package

As it was already announced Drop schedule announcement about EOL of Treasure Agent (td-agent) 4, td-agent v4 was reached EOL in Dec, 2023.

We strongly recommend migrating from td-agent v4 to fluent-package v5 (LTS). See Upgrade to fluent-package v5

Download

Please see the download page.

Fluentd v1.17.0 has been released

2024-04-30

Hi users!

We have released v1.17.0 on 2024-04-30. ChangeLog is here.

This release is a new release of v1.17 series. In this release, we added some new features for some plugins and fixed bugs of Parser.

Enhancement

`in_tail`: Add `glob_policy` option for expanding glob capability of `path` and `exclude_path`

In this release, we added a new option glob_policy for in_tail plugin.

In previous versions, we can use only * in glob patterns for path and exclude_path option.

Example:

path /path/to/*
exclude_path ["/path/to/*.gz", "/path/to/*.zip"]

From this version, we can also use [], ?, and {} in glob patterns depending on the glob_policy option.

Example:

path "[0-1][2-3].log"
glob_policy extended

Please see the document and #4401 for more information.

`out_http`: Support AWS Signature Version 4 authentication

In this release, we added a new option aws_sigv4 for the method setting of out_http plugin.

By using this option, out_http can use AWS Signature Version 4.

For example, this allows out_http to write to Amazon OpenSearch Ingestion.

Please see the document and #4459 for more information.

`out_http`: Add option to reuse connections

In this release, we add a new option reuse_connections for out_http plugin.

This option will improve throughput of out_http.

Please see the document and #4330 for more information.

`in_http`: Recognize CSP reports as JSON data

In this release, we make the data type of the request where the Content-Type is application/csp-report be considered JSON by default.

Now, in_http can receive Content Security Policy's report by default.

Please see #4282 for more information.

Bug Fixes

Make sure parser returns hash

The record data in an event of Fluentd must be a hash object.

Life of a Fluentd event

However, in the previous versions, some parser plugins could return a non-hash object, such as an array. It could cause errors in subsequent processing.

In this release, the following parser plugins have been fixed.

The changes are as follows:

Make sure to return a hash record.
Make sure to accept only a hash or an array of hash.

Here are the details for each case.

Example of changed behavior

Config:

<source>
  @type tcp
  tag test.tcp
  <parse>
    @type json
  </parse>
</source>

<match test.**>
  @type stdout
</match>

Send an array data:

$ netcat 0.0.0.0 5170
[{"k":"v"}, {"k2":"v2"}]

The result before this version:

{datetime} test.tcp: [{"k":"v"},{"k2":"v2"}]

The result after this version:

{datetime} test.tcp: {"k":"v"}
{datetime} test.tcp: {"k2":"v2"}

Example of resolved error

Config:

<source>
  @type tcp
  tag test.tcp
  <parse>
    @type json
    null_empty_string
  </parse>
</source>

<match test.**>
  @type stdout
</match>

Send an array data:

$ netcat 0.0.0.0 5170
[{"k":"v"}, {"k2":"v2"}]

The result before this version:

{datetime} [error]: #0 unexpected error on reading data host="xxx" port=xxx error_class=NoMethodError error="undefined method `each_key' for [{\"k\"=>\"v\"}, {\"k2\"=>\"v2\"}]:Array"

The result after this version:

{datetime} test.tcp: {"k":"v"}
{datetime} test.tcp: {"k2":"v2"}

Remaining problem: filter_parser

In the previous versions, filter_parser could return an array record based on this wrong behavior. From this release, it can not return multiple parsed results anymore and Fluentd outputs a warning log in this case. This behavior should improve in the future.

Here is an example.

<source>
  @type sample
  tag test.array
  sample {"message": "[{\"k\":\"v\"}, {\"k2\":\"v2\"}]"}
</source>

<filter test.**>
  @type parser
  key_name message
  <parse>
    @type json
  </parse>
</filter>

<match test.**>
  @type stdout
</match>

The result before this version:

{datetime} test.array: [{"k":"v"},{"k2":"v2"}]

The result after this version:

{datetime} [warn]: #0 dump an error event: error_class=Fluent::Plugin::Parser::ParserError error="Could not emit the event. The parser returned multiple results, but currently filter_parser plugin only returns the first parsed result. Raw data: '[{\"k\":\"v\"}, {\"k2\":\"v2\"}]'" location=nil tag="test.array" time=xxx record={"k2"=>"v2"}
{datetime} test.array: {"k":"v"}

These are the major changes for this release.

In addition, some performance improvements have been included! Please see ChangeLog for details!

Enjoy logging!

fluent-package v5.0.3 has been released

2024-03-29

Hi users!

We have released fluent-package v5.0.3. fluent-package is a stable distribution package of Fluentd. (successor of td-agent v4)

This is a maintenance release of v5.0.x LTS series. As significant slow starting service and crash issues during startup on Windows were fixed, we recommend upgrading to fluent-package v5.0.3!

Changes from fluent-package v5.0.2

Update fluentd to 1.16.5. See the following blog articles about details.
- Fluentd v1.16.4 has been released
- Fluentd v1.16.5 has been released
Update bundled plugins
- e.g. fluent-diagtool v1.0.5. It supports to collect list of plugins on Windows.
msi: fixed wrong environment path for Fluent Package Prompt (#606)
- It breaks fluent-diagtool behavior to launch fluent-gem correctly.
msi: removed unnecessary path delimiter (#607)
- It doesn't cause any problem yet, but it should treat %~dp0 correctly.
rpm: fixed to take over enabled state of systemd service from td-agent v4 (#613)
deb rpm: fixed to quote target files correctly not to cause migration failures (#615)
msi: added a patch for RubyInstaller to avoid crash on start up (#620)
msi: fixed slow start issue on Windows (#631)

Note: v5.0.3 for Windows msi will be shipped later.

About next LTS schedule

We plan to ship next LTS version of fluent-package v5.0.4 on June, 2024. The content of updates are still in T.B.D.

End of support for td-agent v4, let's migrate to fluent-package

As it was already announced Drop schedule announcement about EOL of Treasure Agent (td-agent) 4, td-agent v4 was reached EOL in Dec, 2023.

We strongly recommend migrating from td-agent v4 to fluent-package v5 (LTS). See Upgrade to fluent-package v5

Download

Please see the download page.

Fluentd v1.16.5 has been released

2024-03-27

Hi users!

We have released v1.16.5 on 2024-03-27. ChangeLog is here.

The previous version v1.16.4 has a serious Buffer bug when it processes large data exceeding chunk size limit. So, we have released v1.16.5 urgently, and have fixed the bug. So, please don't use v1.16.4.

Sorry for troubling.

Bug Fixes

Buffer: Fix emit error of v1.16.4 sometimes failing to process large data exceeding chunk size limit

In the previous version v1.16.4, we fixed a Buffer bug (Please see Fluentd v1.16.4 has been released for details).

There was a race condition problem with that fix, and similar Buffer errors explained in Fluentd v1.16.4 has been released could occur.

emit transaction failed: ...
send an error event stream to @ERROR: ...

These errors are the same as the Buffer error of Fluentd v1.16.4 has been released, but it would be more likely to happen. The cause is a race condition when processing large data exceeding chunk size limit. So, in that case, these errors can occur depending on the timing.

So, please don't use v1.16.4.

See #4447 for more information.

Sorry for troubling.

Fluentd v1.16.4 has been released

2024-03-14

Hi users!

We have released v1.16.4 on 2024-03-14. ChangeLog is here.

In this release, we fixed several bugs and improved performance.

Bug Fixes

Buffer: Fix emit error sometimes caused by failing to process large data exceeding chunk size limit

In previous versions, when Buffer processes large data exceeding the chunk size limit, it sometimes fails and causes an emit error.

It is usually unlikely, but it may occur when Buffer receives data with extremely large records that are distributed unevenly in the data.

If you set @ERROR label, the data is routed to that label, and Fluentd outputs a warning log message: send an error event stream to @ERROR: ....

However, if you don't set @ERROR label, the data is discarded, and Fluentd outputs a warning log message: emit transaction failed: .... In addition, Input plugins that do not consider an emit error may stop working, such as in_windows_eventlog2.

Here is an example of the warning message:

2024-03-22 14:13:35 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="closed stream" location="/path/to/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:82:in `pos'" tag="test"

If you see these warnings, please update Fluentd.

See #4342 for more information.

Others

in_tail: Fix tail watchers in rotate_wait state not being managed. #4334
- This problem should not have any actual negative impact on the operation.
Buffer: Avoid unnecessary log processing. It will improve performance. #4331

Enjoy logging!

Fluentd Blog

fluent-package v5.0.5 has been released

Changes from fluent-package v5.0.4

Solved possibility of DoS through "NoMemoryError"

msi: set GEM_HOME/GEM_PATH in fluentd.bat

Fixed SIGABORT error with fluent-plugin-systemd

Download

About next LTS schedule

End of support for td-agent v4, let's migrate to fluent-package

Scheduled support lifecycle announcement about Fluent Package v6

When the next LTS (Long Term Support) version available?

Fluentd v1.16.6 has been released

Bug Fixes

yaml_parser: $log_level notation in YAML was supported

fluentd command: Fix --plugin (-p) option not to overwrite default value

logger: Fix LoadError with console gem v1.25

Fluentd v1.17.1 has been released

Enhancement

yaml_parser: $log_level notation in YAML was supported

out_http: compress option was supported

in_sample: the default behavior was changed

Bug Fixes

logger: Fix LoadError with console gem v1.25

fluent-package v5.1.0 has been released

Changes from fluent-package v5.0.4

Update bundled components overall

Download

About next LTS schedule

End of support for td-agent v4, let's migrate to fluent-package

fluent-package v5.0.4 has been released

Changes from fluent-package v5.0.3

Fixed to prevent launching Fluentd wrongly if the service is already running

About next LTS schedule

End of support for td-agent v4, let's migrate to fluent-package

Download

Fluentd v1.17.0 has been released

Enhancement

in_tail: Add glob_policy option for expanding glob capability of path and exclude_path

out_http: Support AWS Signature Version 4 authentication

out_http: Add option to reuse connections

in_http: Recognize CSP reports as JSON data

Bug Fixes

Make sure parser returns hash

Example of changed behavior

Example of resolved error

Remaining problem: filter_parser

fluent-package v5.0.3 has been released

Changes from fluent-package v5.0.2

About next LTS schedule

End of support for td-agent v4, let's migrate to fluent-package

Download

Fluentd v1.16.5 has been released

Bug Fixes

Buffer: Fix emit error of v1.16.4 sometimes failing to process large data exceeding chunk size limit

Fluentd v1.16.4 has been released

Bug Fixes

Buffer: Fix emit error sometimes caused by failing to process large data exceeding chunk size limit

Others

msi: set `GEM_HOME`/`GEM_PATH` in fluentd.bat

`yaml_parser`: `$log_level` notation in YAML was supported

fluentd command: Fix `--plugin` (-p) option not to overwrite default value

`yaml_parser`: `$log_level` notation in YAML was supported

`out_http`: `compress` option was supported

`in_sample`: the default behavior was changed

`in_tail`: Add `glob_policy` option for expanding glob capability of `path` and `exclude_path`

`out_http`: Support AWS Signature Version 4 authentication

`out_http`: Add option to reuse connections

`in_http`: Recognize CSP reports as JSON data