aws-kinesisanalytics-flink: Add support to deploy in VPC #21104
Labels
@aws-cdk/aws-kinesisanalytics-flink
Related to kinesisanalytics-flink package
effort/medium
Medium work item – several days of effort
feature-request
A feature should be added or improved.
p1
Comments
anandshah123
added
feature-request
github-actions
bot
added
the
@aws-cdk/aws-kinesisanalytics-flink
otaviomacedo
added
p1
effort/medium
|
+1 @anandshah123 What is the workaround you found to attach the VPC to Flink app ? |
|
Hi @ajdinahmetovic I used CDK custom resource to attach VPC once Flink app is created, |
mergify bot
pushed a commit
that referenced
this issue
Mar 8, 2023
) The Kinesis Data Analytics team added support for [deploying Flink applications in a VPC](https://docs.aws.amazon.com/kinesisanalytics/latest/java/vpc.html). This feature is also available in CloudFormation. Deploying Flink in a VPC allows the application to reach services like Redis and other databases. This PR adds support for configuring `VpcConfigurations` with `vpcSubets` (subnetSelection) and securityGroups following similar patterns for resources like `lambda.Function` that support optional deployment in a VPC. Some design decisions: - Name the subnet selection prop `vpcSubnets`. Some resources call the subnet selection property `subnetSelection` but `vpcSubnets` seemed more popular and is used by the Lambda and ECS modules. - Only support passing an array of security groups. Some resources support adding a single SecurityGroup or SecurityGroupId properties but it appears this [usage is deprecated](https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-lambda/lib/function.ts#L170) in favor of always passing an array of SecurityGroups. - I added a `fromApplicationAttributes` factory that includes `securityGroups`. This seemed like an appropriate time to add this method given there was another property to pass besides ARN and name. However I didn't go down the path of including a role in `fromApplicationAttributes` yet in order to keep this PR focused. - ~~I thought about adding a section to the readme about using VPCs, but I didn't notice a section like that in the [Lambda readme](https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-lambda/README.md) for instance. My current thinking is that the conventions for VPC-bound resources are so consistent it probably doesn't warrant more documentation~~ @aws-cdk-automation did not buy this rational. I'd like to follow-up with a PR to move code into more files as the > 1K lines of code in `application.ts` is getting a little unweildy. I wanted to avoid moving code around in this PR to make it easier to review. Closes #21104. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
|
homakk
pushed a commit
to homakk/aws-cdk
that referenced
this issue
Mar 28, 2023
…#24442) The Kinesis Data Analytics team added support for [deploying Flink applications in a VPC](https://docs.aws.amazon.com/kinesisanalytics/latest/java/vpc.html). This feature is also available in CloudFormation. Deploying Flink in a VPC allows the application to reach services like Redis and other databases. This PR adds support for configuring `VpcConfigurations` with `vpcSubets` (subnetSelection) and securityGroups following similar patterns for resources like `lambda.Function` that support optional deployment in a VPC. Some design decisions: - Name the subnet selection prop `vpcSubnets`. Some resources call the subnet selection property `subnetSelection` but `vpcSubnets` seemed more popular and is used by the Lambda and ECS modules. - Only support passing an array of security groups. Some resources support adding a single SecurityGroup or SecurityGroupId properties but it appears this [usage is deprecated](https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-lambda/lib/function.ts#L170) in favor of always passing an array of SecurityGroups. - I added a `fromApplicationAttributes` factory that includes `securityGroups`. This seemed like an appropriate time to add this method given there was another property to pass besides ARN and name. However I didn't go down the path of including a role in `fromApplicationAttributes` yet in order to keep this PR focused. - ~~I thought about adding a section to the readme about using VPCs, but I didn't notice a section like that in the [Lambda readme](https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-lambda/README.md) for instance. My current thinking is that the conventions for VPC-bound resources are so consistent it probably doesn't warrant more documentation~~ @aws-cdk-automation did not buy this rational. I'd like to follow-up with a PR to move code into more files as the > 1K lines of code in `application.ts` is getting a little unweildy. I wanted to avoid moving code around in this PR to make it easier to review. Closes aws#21104. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the feature
Presently, CDK aws-kinesisanalytics-flink: module class Application doesn't support deploying Apache Flink application in VPC.
We should have Application deployment support for VPC in CDK
Use Case
It is most common use case to run Apache Flink Kinesis Data Analytics application in VPC for connectivity to MSK, Database Sinks like Timestream etc.
Presently workaround is to have CDK custom resource attaching VPC to the Kinesis Data Analytics application.
Proposed Solution
Add VPCConfiguration option just like L1 CfnApplication class in high level L2 Application class.
Potentially also add direct methods or properties like following
or
Other Information
No response
Acknowledgements
CDK version used
1.163.1
Environment details (OS name and version, etc.)
All
The text was updated successfully, but these errors were encountered: