Building a sophisticated CodePipeline with CDK in a Monorepo Setup

Hi. Imagine you started as an AWS DevOps engineer in a small company and your boss wants us to go all-in AWS. Of course, the CICD should be entirely on AWS for automated deployments. All the infrastructure should be put into code. And we are using a monorepo containing a backend and frontend both in TypeScript.

Great. You have been working with GitLab CI or GitHub Actions where you just need to define the Yammer file and it creates on the platform the pipeline for you. On AWS, it's a bit different. Each resource you have to create by yourself. So you decided to use AWS CDK because you can use TypeScript. You can use the same tooling as the developers. You are able to put the code inside their monorepo. And you have also a nice abstraction, which also the developer may understand.

Let's get your hands dirty. So you start by creating a pipeline using this abstraction new pipeline. And each code pipeline contains stages. And a stage contains actions. The actions result and outputs artifact. This artifact can be stored into an S3 bucket. So you import the code from CodeCommit where the monorepo is hosted. And you can use the to create your first stage by using the add stage. And here you create the CodeCommit source action. So this will be triggered whenever something happening on the main branch. And you create a second stage by using the code bit action. And in there, you use a code bit project. The code bit project you can take, you can define each phases. In this case, I use the build phase, where I want to install linting and test unit. And the result of this action can be then stored inside this folder in this case. And now the next stages are the deployment stages. So you want to deploy on different accounts on different AWS account.

So you create the loop over all accounts. And you can also make use of CDKs abstraction. So you can create your own method, in this case, the create pipeline project. So instead of using this code, again, you can actually make it like easier by abstracting that and putting it in your own method, and then just make use of each phase. First, you assume the role, and then you use the CDK deploy command. After the deployment, you can run integration tests. You can also use Canary operations. In the staging and production environment, we have the manual approval. This manual approval action can be merged together with the corporate action. Only if it's not on default, otherwise, we just skip the manual approval action. But each time there's a change on the front end, the back end will also be deployed. To handle that, you need to create multiple pipelines. Now you have a pipeline for the front end and another pipeline for the back end. This could be the code, how it looks like. You could find multiple pipelines, and you can handle it by using environment bio.

But the real question will be when to trigger which pipeline. And the solution for that using EventBridge and AWS Lambda. So you have to know that every AWS service emits an event. In this case, whenever something happening on the code repository, then we want to catch this event. We want to catch it by using EventBridge. And we want to tell the pipeline, hey, use EventBridge for emitting this event. And this event will be then forwarded to a Lambda function. We define the Lambda in here. And with this code, we're making sure we only have one Lambda. Otherwise, we would create multiple Lambdas.

But with this code, we're making sure we only have this Lambda one time. And the Lambda makes use of the code commit get differences. So it checks what file has been changed, and then it triggers the pipeline accordingly. So the inside the Lambda, we define each pipeline. So we create an object where we have the front end, the back end front end stack pipeline and the back end stack pipeline. And we also define when this pipeline should be triggered. And here we define the path of each pipeline.

So in this case, whenever a file changes in the back end folder, or we change something in the back end stack, then the back end stack pipeline should be triggered. We also have the ability to have common folder. So let's say you have like a package, which is shared between the back end and the front end. And that we can use the spread operator in order to trigger all the pipelines. And we make use of the JavaScript SDK for code commit and code pipeline. And the handler uses the get differences command to get all the file changes. And then we loop over each pipeline and loop over each pipeline path to check if the filename is part of the change. And if so, we start to execute the code pipeline.

So what we have seen here is we are able to use CDK to make code pipeline less suck. We can also do something like platform engineering to make use of abstraction so that anytime you create a new pipeline, you can make use of certain methods like the create pipeline project method. We can use event rich to trigger each pipeline accordingly. So we can also say whenever there's a new pull request, we trigger then the pipeline, or there are many other events, you can check it, you can find it online. And with that, you are able to create sophisticated pipeline. Thank you. You'll find all the information in here. You can also find the repository. And thanks for your time.