Skip to content

Commit 49b87db

Browse files
yamatatsuyamatatsu
authored
feat(iot): add Action to put objects in S3 Buckets (#17307)
I'm trying to implement aws-iot L2 Constructs. This PR is one of steps after following PR: - #16681 (comment) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
1 parent 3570b17 commit 49b87db

File tree

11 files changed

+445
-7
lines changed

11 files changed

+445
-7
lines changed

‎package.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,8 @@
7777
"@aws-cdk/assertions/fs-extra/**",
7878
"@aws-cdk/aws-amplify-alpha/yaml",
7979
"@aws-cdk/aws-amplify-alpha/yaml/**",
80+
"@aws-cdk/aws-iot-actions-alpha/case",
81+
"@aws-cdk/aws-iot-actions-alpha/case/**",
8082
"@aws-cdk/aws-amplify/yaml",
8183
"@aws-cdk/aws-amplify/yaml/**",
8284
"@aws-cdk/aws-codebuild/yaml",
@@ -91,6 +93,8 @@
9193
"@aws-cdk/aws-eks/yaml/**",
9294
"@aws-cdk/aws-events-targets/aws-sdk",
9395
"@aws-cdk/aws-events-targets/aws-sdk/**",
96+
"@aws-cdk/aws-iot-actions/case",
97+
"@aws-cdk/aws-iot-actions/case/**",
9498
"@aws-cdk/aws-s3-deployment/case",
9599
"@aws-cdk/aws-s3-deployment/case/**",
96100
"@aws-cdk/cloud-assembly-schema/jsonschema",

‎packages/@aws-cdk/aws-iot-actions/NOTICE‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,32 @@
11
AWS Cloud Development Kit (AWS CDK)
22
Copyright 2018-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
3+
4+
-------------------------------------------------------------------------------
5+
6+
The AWS CDK includes the following third-party software/licensing:
7+
8+
** case - https://www.npmjs.com/package/case
9+
Copyright (c) 2013 Nathan Bubna
10+
11+
Permission is hereby granted, free of charge, to any person
12+
obtaining a copy of this software and associated documentation
13+
files (the "Software"), to deal in the Software without
14+
restriction, including without limitation the rights to use,
15+
copy, modify, merge, publish, distribute, sublicense, and/or sell
16+
copies of the Software, and to permit persons to whom the
17+
Software is furnished to do so, subject to the following
18+
conditions:
19+
20+
The above copyright notice and this permission notice shall be
21+
included in all copies or substantial portions of the Software.
22+
23+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
24+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
25+
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
26+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
27+
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
28+
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
29+
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
30+
OTHER DEALINGS IN THE SOFTWARE.
31+
32+
----------------

‎packages/@aws-cdk/aws-iot-actions/README.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ supported AWS Services. Instances of these classes should be passed to
2222
Currently supported are:
2323

2424
- Invoke a Lambda function
25+
- Put objects to a S3 bucket
26+
- Put logs to CloudWatch Logs
2527

2628
## Invoke a Lambda function
2729

@@ -49,6 +51,59 @@ new iot.TopicRule(this, 'TopicRule', {
4951
});
5052
```
5153

54+
## Put objects to a S3 bucket
55+
56+
The code snippet below creates an AWS IoT Rule that put objects to a S3 bucket
57+
when it is triggered.
58+
59+
```ts
60+
import * as iot from '@aws-cdk/aws-iot';
61+
import * as actions from '@aws-cdk/aws-iot-actions';
62+
import * as s3 from '@aws-cdk/aws-s3';
63+
64+
const bucket = new s3.Bucket(this, 'MyBucket');
65+
66+
new iot.TopicRule(this, 'TopicRule', {
67+
sql: iot.IotSql.fromStringAsVer20160323("SELECT topic(2) as device_id FROM 'device/+/data'"),
68+
actions: [new actions.S3PutObjectAction(bucket)],
69+
});
70+
```
71+
72+
The property `key` of `S3PutObjectAction` is given the value `${topic()}/${timestamp()}` by default. This `${topic()}`
73+
and `${timestamp()}` is called Substitution templates. For more information see
74+
[this documentation](https://docs.aws.amazon.com/iot/latest/developerguide/iot-substitution-templates.html).
75+
In above sample, `${topic()}` is replaced by a given MQTT topic as `device/001/data`. And `${timestamp()}` is replaced
76+
by the number of the current timestamp in milliseconds as `1636289461203`. So if the MQTT broker receives an MQTT topic
77+
`device/001/data` on `2021-11-07T00:00:00.000Z`, the S3 bucket object will be put to `device/001/data/1636243200000`.
78+
79+
You can also set specific `key` as following:
80+
81+
```ts
82+
new iot.TopicRule(this, 'TopicRule', {
83+
sql: iot.IotSql.fromStringAsVer20160323(
84+
"SELECT topic(2) as device_id, year, month, day FROM 'device/+/data'",
85+
),
86+
actions: [
87+
new actions.S3PutObjectAction(bucket, {
88+
key: '${year}/${month}/${day}/${topic(2)}',
89+
}),
90+
],
91+
});
92+
```
93+
94+
If you wanna set access control to the S3 bucket object, you can specify `accessControl` as following:
95+
96+
```ts
97+
new iot.TopicRule(this, 'TopicRule', {
98+
sql: iot.IotSql.fromStringAsVer20160323("SELECT * FROM 'device/+/data'"),
99+
actions: [
100+
new actions.S3PutObjectAction(bucket, {
101+
accessControl: s3.BucketAccessControl.PUBLIC_READ,
102+
}),
103+
],
104+
});
105+
```
106+
52107
## Put logs to CloudWatch Logs
53108

54109
The code snippet below creates an AWS IoT Rule that put logs to CloudWatch Logs

‎packages/@aws-cdk/aws-iot-actions/lib/cloudwatch-logs-action.ts‎

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,13 @@
11
import * as iam from '@aws-cdk/aws-iam';
22
import * as iot from '@aws-cdk/aws-iot';
33
import * as logs from '@aws-cdk/aws-logs';
4+
import { CommonActionProps } from './common-action-props';
45
import { singletonActionRole } from './private/role';
56

67
/**
78
* Configuration properties of an action for CloudWatch Logs.
89
*/
9-
export interface CloudWatchLogsActionProps {
10-
/**
11-
* The IAM role that allows access to the CloudWatch log group.
12-
*
13-
* @default a new role will be created
14-
*/
15-
readonly role?: iam.IRole;
10+
export interface CloudWatchLogsActionProps extends CommonActionProps {
1611
}
1712

1813
/**

‎packages/@aws-cdk/aws-iot-actions/lib/common-action-props.ts‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
import * as iam from '@aws-cdk/aws-iam';
2+
3+
/**
4+
* Common properties shared by Actions it access to AWS service.
5+
*/
6+
export interface CommonActionProps {
7+
/**
8+
* The IAM role that allows access to AWS service.
9+
*
10+
* @default a new role will be created
11+
*/
12+
readonly role?: iam.IRole;
13+
}

‎packages/@aws-cdk/aws-iot-actions/lib/index.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,4 @@
11
export * from './cloudwatch-logs-action';
2+
export * from './common-action-props';
23
export * from './lambda-function-action';
4+
export * from './s3-put-object-action';

‎packages/@aws-cdk/aws-iot-actions/lib/s3-put-object-action.ts‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
import * as iam from '@aws-cdk/aws-iam';
2+
import * as iot from '@aws-cdk/aws-iot';
3+
import * as s3 from '@aws-cdk/aws-s3';
4+
import { kebab as toKebabCase } from 'case';
5+
import { CommonActionProps } from './common-action-props';
6+
import { singletonActionRole } from './private/role';
7+
8+
/**
9+
* Configuration properties of an action for s3.
10+
*/
11+
export interface S3PutObjectActionProps extends CommonActionProps {
12+
/**
13+
* The Amazon S3 canned ACL that controls access to the object identified by the object key.
14+
* @see https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl
15+
*
16+
* @default None
17+
*/
18+
readonly accessControl?: s3.BucketAccessControl;
19+
20+
/**
21+
* The path to the file where the data is written.
22+
*
23+
* Supports substitution templates.
24+
* @see https://docs.aws.amazon.com/iot/latest/developerguide/iot-substitution-templates.html
25+
*
26+
* @default '${topic()}/${timestamp()}'
27+
*/
28+
readonly key?: string;
29+
}
30+
31+
/**
32+
* The action to write the data from an MQTT message to an Amazon S3 bucket.
33+
*/
34+
export class S3PutObjectAction implements iot.IAction {
35+
private readonly accessControl?: string;
36+
private readonly key?: string;
37+
private readonly role?: iam.IRole;
38+
39+
/**
40+
* @param bucket The Amazon S3 bucket to which to write data.
41+
* @param props Optional properties to not use default
42+
*/
43+
constructor(private readonly bucket: s3.IBucket, props: S3PutObjectActionProps = {}) {
44+
this.accessControl = props.accessControl;
45+
this.key = props.key;
46+
this.role = props.role;
47+
}
48+
49+
bind(rule: iot.ITopicRule): iot.ActionConfig {
50+
const role = this.role ?? singletonActionRole(rule);
51+
role.addToPrincipalPolicy(new iam.PolicyStatement({
52+
actions: ['s3:PutObject'],
53+
resources: [this.bucket.arnForObjects('*')],
54+
}));
55+
56+
return {
57+
configuration: {
58+
s3: {
59+
bucketName: this.bucket.bucketName,
60+
cannedAcl: this.accessControl && toKebabCase(this.accessControl.toString()),
61+
key: this.key ?? '${topic()}/${timestamp()}',
62+
roleArn: role.roleArn,
63+
},
64+
},
65+
};
66+
}
67+
}

‎packages/@aws-cdk/aws-iot-actions/package.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,9 @@
8383
"@aws-cdk/aws-iot": "0.0.0",
8484
"@aws-cdk/aws-lambda": "0.0.0",
8585
"@aws-cdk/aws-logs": "0.0.0",
86+
"@aws-cdk/aws-s3": "0.0.0",
8687
"@aws-cdk/core": "0.0.0",
88+
"case": "1.6.3",
8789
"constructs": "^3.3.69"
8890
},
8991
"homepage": "https://github.com/aws/aws-cdk",
@@ -92,9 +94,13 @@
9294
"@aws-cdk/aws-iot": "0.0.0",
9395
"@aws-cdk/aws-lambda": "0.0.0",
9496
"@aws-cdk/aws-logs": "0.0.0",
97+
"@aws-cdk/aws-s3": "0.0.0",
9598
"@aws-cdk/core": "0.0.0",
9699
"constructs": "^3.3.69"
97100
},
101+
"bundledDependencies": [
102+
"case"
103+
],
98104
"engines": {
99105
"node": ">= 10.13.0 <13 || >=13.7.0"
100106
},

‎packages/@aws-cdk/aws-iot-actions/test/s3-put-object/integ.s3-put-object-action.expected.json‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
{
2+
"Resources": {
3+
"TopicRule40A4EA44": {
4+
"Type": "AWS::IoT::TopicRule",
5+
"Properties": {
6+
"TopicRulePayload": {
7+
"Actions": [
8+
{
9+
"S3": {
10+
"BucketName": {
11+
"Ref": "MyBucketF68F3FF0"
12+
},
13+
"CannedAcl": "bucket-owner-full-control",
14+
"Key": "${year}/${month}/${day}/${topic(2)}",
15+
"RoleArn": {
16+
"Fn::GetAtt": [
17+
"TopicRuleTopicRuleActionRole246C4F77",
18+
"Arn"
19+
]
20+
}
21+
}
22+
}
23+
],
24+
"AwsIotSqlVersion": "2016-03-23",
25+
"Sql": "SELECT topic(2) as device_id, year, month, day FROM 'device/+/data'"
26+
}
27+
}
28+
},
29+
"TopicRuleTopicRuleActionRole246C4F77": {
30+
"Type": "AWS::IAM::Role",
31+
"Properties": {
32+
"AssumeRolePolicyDocument": {
33+
"Statement": [
34+
{
35+
"Action": "sts:AssumeRole",
36+
"Effect": "Allow",
37+
"Principal": {
38+
"Service": "iot.amazonaws.com"
39+
}
40+
}
41+
],
42+
"Version": "2012-10-17"
43+
}
44+
}
45+
},
46+
"TopicRuleTopicRuleActionRoleDefaultPolicy99ADD687": {
47+
"Type": "AWS::IAM::Policy",
48+
"Properties": {
49+
"PolicyDocument": {
50+
"Statement": [
51+
{
52+
"Action": "s3:PutObject",
53+
"Effect": "Allow",
54+
"Resource": {
55+
"Fn::Join": [
56+
"",
57+
[
58+
{
59+
"Fn::GetAtt": [
60+
"MyBucketF68F3FF0",
61+
"Arn"
62+
]
63+
},
64+
"/*"
65+
]
66+
]
67+
}
68+
}
69+
],
70+
"Version": "2012-10-17"
71+
},
72+
"PolicyName": "TopicRuleTopicRuleActionRoleDefaultPolicy99ADD687",
73+
"Roles": [
74+
{
75+
"Ref": "TopicRuleTopicRuleActionRole246C4F77"
76+
}
77+
]
78+
}
79+
},
80+
"MyBucketF68F3FF0": {
81+
"Type": "AWS::S3::Bucket",
82+
"UpdateReplacePolicy": "Delete",
83+
"DeletionPolicy": "Delete"
84+
}
85+
}
86+
}

‎packages/@aws-cdk/aws-iot-actions/test/s3-put-object/integ.s3-put-object-action.ts‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
/// !cdk-integ pragma:ignore-assets
2+
import * as iot from '@aws-cdk/aws-iot';
3+
import * as s3 from '@aws-cdk/aws-s3';
4+
import * as cdk from '@aws-cdk/core';
5+
import * as actions from '../../lib';
6+
7+
const app = new cdk.App();
8+
9+
class TestStack extends cdk.Stack {
10+
constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
11+
super(scope, id, props);
12+
13+
const topicRule = new iot.TopicRule(this, 'TopicRule', {
14+
sql: iot.IotSql.fromStringAsVer20160323(
15+
"SELECT topic(2) as device_id, year, month, day FROM 'device/+/data'",
16+
),
17+
});
18+
19+
const bucket = new s3.Bucket(this, 'MyBucket', {
20+
removalPolicy: cdk.RemovalPolicy.DESTROY,
21+
});
22+
topicRule.addAction(
23+
new actions.S3PutObjectAction(bucket, {
24+
key: '${year}/${month}/${day}/${topic(2)}',
25+
accessControl: s3.BucketAccessControl.BUCKET_OWNER_FULL_CONTROL,
26+
}),
27+
);
28+
}
29+
}
30+
31+
new TestStack(app, 'test-stack');
32+
app.synth();

0 commit comments

Comments
 (0)