Fluentd plugin to add Amazon EC2 metadata fields to a event record
| fluent-plugin-ec2-metadata | fluentd | ruby |
|---|---|---|
| >= 0.1.0 | v0.14.x | >= 2.1 |
| 0.0.15 <= | v0.12.x | >= 1.9 |
Use RubyGems:
gem install fluent-plugin-ec2-metadata
Example:
<match foo.**>
@type ec2_metadata
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET/KEY
metadata_refresh_seconds 300 # Optional, default 300 seconds
imdsv2 true # Optional, default false
output_tag ${instance_id}.${tag}
<record>
hostname ${tagset_name}
instance_id ${instance_id}
instance_type ${instance_type}
az ${availability_zone}
private_ip ${private_ip}
vpc_id ${vpc_id}
ami_id ${image_id}
account_id ${account_id}
</record>
</match>
Assume following input is coming:
foo.bar {"message":"hello ec2!"}
then output becomes as below (indented):
i-28b5ee77.foo.bar {
"hostname" : "web0001",
"instance_id" : "i-28b5ee77",
"instance_type" : "m1.large",
"az" : "us-west-1b",
"private_ip : "10.21.34.200",
"vpc_id" : "vpc-25dab194",
"account_id" : "123456789",
"image_id" : "ami-123456",
"message" : "hello ec2!"
}
Or you can use filter version:
<filter foo.**>
@type ec2_metadata
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET/KEY
metadata_refresh_seconds 300 # Optional, default 300 seconds
imdsv2 true # Optional, default false
<record>
hostname ${tagset_name}
instance_id ${instance_id}
instance_type ${instance_type}
private_ip ${private_ip}
az ${availability_zone}
vpc_id ${vpc_id}
ami_id ${image_id}
account_id ${account_id}
</record>
</filter>
The following placeholders are always available:
- ${tag} input tag
- ${tag_parts} input tag splitted by '.'. you can use it like
${tag_parts[0]}or${tag_parts[-1]} - ${instance_id} instance id
- ${instance_type} instance type
- ${availability_zone} availability zone
- ${region} region
- ${private_ip} private ip
- ${mac} MAC address
- ${vpc_id} vpc id
- ${subnet_id} subnet id
- ${account_id} account id
- ${image_id} ami image id
The followings are available when you define aws_key_id and aws_sec_key(or define IAM Policy):
- ${tagset_xxx} EC2 tag (e.g. tagset_name is replaced by the value of Key = Name)
The following is an example for a minimal IAM policy needed to ReadOnlyAccess to EC2.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "elasticloadbalancing:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"cloudwatch:ListMetrics",
"cloudwatch:GetMetricStatistics",
"cloudwatch:Describe*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "autoscaling:Describe*",
"Resource": "*"
}
]
}
Refer to the AWS documentation for example policies. Using IAM roles with a properly configured IAM policy are preferred over embedding access keys on EC2 instances.
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request