Fluentd output plugin to upload/publish event data to Redis storage.
Forked from redis-store which forked from redisstore plugin I'm using this in high traffic production environment
Currently the plugin supports following Redis commands:
- set by
string
type (of the plugin) - lpush/rpush by
list
type - sadd by
set
type - zadd by
zset
type - publish by
publish
type
- plain(as is)
- JSON
- MessagePack
Redis commands require key and value.
For key, the plugin supports either way;
-
Specify a fixed key.
You can do this simply usingkey
option in td-agent configuration file.type redis_store key userdata
-
Lookup a key string in every event data by a lookup path.
If event data have structured data like{ "user": { "name": "Kei" } }
and you want to use each name of user, you can use
key_path
option.type redis_store key_path user.name
With the above data,
Kei
will be a key.
In addition, key_prefix
and key_suffix
are useful in some cases. Both are available either key
and key_path
type redis_store
key_path user.name
key_prefix ouruser.
key_suffix .accesslog
With the previous data, key will be outuser.Kei.accesslog
.
To determine what value in every event data to be srtored, you have two options;
- Store extracted data in event data, by a lookup path with
value_path
option.
It works likekey_path
. - Store whole data.
This is default behavior. To do it, simply omitvalue_path
option.
/usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-redis-store
Key | Type | Required? | Default | Description |
---|---|---|---|---|
host |
string | Optional | 127.0.0.1 | host name of Redis server |
port |
int | Optional | 6379 | port number of Redis server |
password |
string | Optional | password for Redis connection | |
path |
string | Optional | To connect via Unix socket, try '/tmp/redis.sock' | |
db |
int | Optional | 0 | DB number of Redis |
timeout |
float | Optional | 5.0 | connection timeout in seconds |
Key | Type | Default | Description |
---|---|---|---|
key |
string | Fixed key used to store(publish) in Redis | |
key_path |
string | path to lookup for key in the event data | |
key_prefix |
string | prefix of key | |
key_suffix |
string | suffix of key | |
value_path |
string | (whole event data) | path to lookup for value in the event data |
store_type |
string | zset | string /list /set /zset /publish |
format_type |
string | plain | format type for value (plain /json /msgpack ) |
key_expire |
int | -1 | If set, the key will be expired in specified seconds |
flush_interval |
time | 1 | Time interval which events will be flushed to Redis |
Note: either key or key_path is required. |
Key | Type | Default | Description |
---|---|---|---|
type |
string | Fixed key used to store(publish) in Redis | |
No more options than common options. |
Key | Type | Default | Description |
---|---|---|---|
order |
string | asc | asc : rpush, desc : lpush |
prevent_duplicate |
(0,1) | 0 | Prevent duplicated value in one key (list) |
string_tolow |
(0,1) | 0 | value to lowercase |
string_unescape |
(0,1) | 0 | unescape value |
string_unescape_twice |
(0,1) | 0 | if you need another unescape |
escape_html_tag |
(0,1) | 0 | escape html tag |
filter_html_tag |
(0,1) | 0 | replace <>'"& with empty character |
value_length |
(0,1) | list trimming doesn't honor ordering |
No more options than common options.
Key | Type | Default | Description |
---|---|---|---|
score_path |
string | (time of log event) | path to lookup for score in the event data |
value_expire |
int | value expiration in seconds | |
prevent_duplicate |
(0,1) | 0 | Prevent duplicated value in one key (list) |
string_tolow |
(0,1) | 0 | value to lowercase |
string_unescape |
(0,1) | 0 | unescape value |
string_unescape_twice |
(0,1) | 0 | in you need another unescape |
escape_html_tag |
(0,1) | 0 | escape html tag |
filter_html_tag |
(0,1) | 0 | replace <>'"& with empty character |
value_length |
(0,1) | z trimming doesn't honor ordering |
If value_expire
is set, the plugin assumes that the score in the SortedSet is
based on timestamp and it deletes expired members every after new event data arrives.
No more options than common options.
Copyright (c) 2013 moaikids
Copyright (c) 2014 HANAI Tohru
Copyright (c) 2016 Gian Giovani
Apache License, Version 2.0