Build a CI / CD Pipeline on AWS

Anish Mahapatra
9 min readNov 28, 2023

--

We will build a CI/CD Pipeline on AWS in this blog. We have all read in theory about the CI/CD pipeline, wouldn’t it be great if we were actually able to implement it? Let’s begin!

Photo by Growtika on Unsplash

Pre Requisites

You will need the following to do this hands-on tutorial along with me.

  • AWS Account
  • AWS CodeCatalyst Account (linked to AWS ID)

I have used this tutorial as a reference.

1. Create IAM Role

First, we will proceed to create a new Stack in CloudFormation Stack.

We need to apply this stack. Save the below code as a .json file. I have named mine codecatalyst.json

CloudFormation console > Create New Stack > Template > Upload a template
{
"AWSTemplateFormatVersion" : "2010-09-09",

"Description" : "AWS CloudFormation Template IAM_Role_and_Policies: Template to create 2 new service roles for CodeCatalyst - CloudFormation deployment. **WARNING** The `main_branch_IAM_role` provides full access to AWS resources in EC2 and CloudFormation services, as those will be used by sample CloudFormation template mentioned in the blog. Please use this role carefully and delete it when not required.",

"Resources" : {
"MainBranchRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"Service": [
"codecatalyst.amazonaws.com",
"codecatalyst-runner.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"RoleName" : "main_branch_IAM_role",
"Policies": [
{
"PolicyName": "ProvideAmazonEC2FullAccess",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": "ec2:*",
"Effect": "Allow",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "elasticloadbalancing:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "cloudwatch:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "autoscaling:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:AWSServiceName": [
"autoscaling.amazonaws.com",
"ec2scheduled.amazonaws.com",
"elasticloadbalancing.amazonaws.com",
"spot.amazonaws.com",
"spotfleet.amazonaws.com",
"transitgateway.amazonaws.com"
]
}
}
},
{
"Effect": "Allow",
"Action": [
"cognito-idp:DescribeUserPoolClient"
],
"Resource": "*"
}
]
}
},
{
"PolicyName": "ProvideAWSCloudFormationFullAccess",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"cloudformation:*"
],
"Resource": "*"
}
]
}
}
]
}
},
"PRBranchRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"Service": [
"codecatalyst.amazonaws.com",
"codecatalyst-runner.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"RoleName" : "pr_branch_IAM_role",
"Policies": [
{
"PolicyName": "CloudFormation-PR-policy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"cloudformation:Describe*",
"cloudformation:CreateChangeSet",
"cloudformation:DeleteChangeSet",
"cloudformation:ExecuteChangeSet",
"cloudformation:SetStackPolicy",
"cloudformation:ValidateTemplate",
"cloudformation:List*",
"iam:PassRole"
],
"Resource": "*",
"Effect": "Allow"
}
]
}
},
{
"PolicyName": "ProvideEC2ReadOnlyAccess",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "elasticloadbalancing:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"cloudwatch:ListMetrics",
"cloudwatch:GetMetricStatistics",
"cloudwatch:Describe*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "autoscaling:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"sns:ListSubscriptions",
"sns:ListTopics"
],
"Resource": "*"
}
]
}
}
]
}
}
}
}

If you click “view on designer”, you will see the following design.

We have set the region to be us-west-2. Click on next and enter the stack name below.

CodeCatalyst-IAM-roles

The architecture will be as shown below.

Let the rest of the settings remain at default. Just provide the acknowledgement and hit Submit.

The stack is now being created.

Code Catalyst

AWS Code Catalyst is an integrated development environment (IDE) provided by Amazon Web Services (AWS) that aims to simplify the software development workflow. It combines a comprehensive set of tools, services, and resources, enabling developers to efficiently collaborate, build, test, and deploy applications within a unified environment.

Code Catalyst is fully managed by AWS, so developers can focus on development. It provides a cloud-based development environment that allows developers to rapidly iterate on code and improve their development productivity. It also simplifies the cross-account functionalities of continuous integration and delivery (CI/CD) best practices.

Code Catalyst offers a unified software development service that makes it faster to build and deliver software on AWS. You can learn more and get started building in minutes on the AWS Free Tier at the Code Catalyst website.

Log into Code Catalyst

Create a new space with any name. Mine is:

workspaceanish

Verify in AWS

Once the space is created, go to the AWS Accounts tab, click on your account ID, and then click on Manage roles from the AWS Management Console.

Select your workspace

Add IAM Role.

Select the role we created previously.

main_branch_IAM_role

Next, go back to CodeCatalyst and create a new Project and start from scratch.

Start from Scratch

Add Code Repository

Select Source Repository.

Source Reposiory > Add Repository > Create Repository > 3-tier-app

Add the details for the repository as shown below.

Environment

Lastly, we need to set up the AWS environment where our CloudFormation stack will be deployed by automated workflows. This is my non-production environment where my sample CloudFormation template will be deployed.

From the left navigation panel, select CI/CD, Environments, and click on Create Environment. In the Environment details enter following:

  • Environment name: PreProdEnv
  • Environment type: Non-production
  • Description: Pre-production environment to learn, test and experiment with CloudFormation
  • AWS account connection
  • Connection: Select the AWS account ID that will be used in the workflows to deploy this environment

Setting up a Dev Environment

We will use the IDE AWS Cloud9 for our development.

From the left navigation panel, select Code, Dev Environments, and click on Create Dev Environment. In the Create dev environment and open with AWS Cloud9 dialog box select following:

  • Repository: Clone a repository
  • Repository: Select the repo that you want to clone. We created a repo 3-tier-app above, so select same.
  • Branch: Work in existing branch
  • Alias: bootstrap

AWS Cloud9 IDE

You will now see the IDE.

For this blog, I am using a sample template that deploys a VPC with 2 subnets and publicly accessible Amazon EC2 instances that are in an Auto Scaling group behind a Load Balancer form. Feel free to use the same as I will be making changes to this template and run a pull request workflow.

# go to the root folder of the repo and run following

$ cd 3-tier-app/
$ wget https://raw.githubusercontent.com/build-on-aws/ci-cd-iac-aws-cloudformation/main/cloudformation-templates/VPC_AutoScaling_With_Public_IPs.json

# check git status
$ git status

# add changed files, commit and push to the git repo
$ git add . -A
$ git commit -m "Uploading first CloudFormation template"
$ git push

You can verify if the CloudFormation template is successfully updated in the repo — in the CodeCatalyst console, click Code in the left-side navigation menu, then select Source repositories. A new file VPC_AutoScaling_With_Public_IPs.json should be added to the repo.

Set up Workflows to Deply Non-prod environment

Here, we will use the dev environment using yaml.

# go to the root folder of the repo and run following
mkdir -p .codecatalyst/workflows
touch .codecatalyst/workflows/main_branch.yaml

Open this in your IDE as shown in the image below.

3-tier-app > .codecatalyst > worflows > main_branch.yaml

Double click on main_branch.yaml

I will give you the code to paste in the yaml file. Just make sure to replace the AWS ID with your AWS ID, which you can find in the home console.

You can find your AWS ID if you click on your name.

Name: Main_Branch_Workflow
SchemaVersion: "1.0"

# Optional - Set automatic triggers.
Triggers:
- Type: Push
Branches:
- main

# Required - Define action configurations.
Actions:
DeployAWSCloudFormationstack_7c:
Identifier: aws/cfn-deploy@v1
Configuration:
parameter-overrides: SSHLocation=54.10.10.2/32,WebServerInstanceType=t2.micro
capabilities: CAPABILITY_IAM,CAPABILITY_NAMED_IAM,CAPABILITY_AUTO_EXPAND
template: VPC_AutoScaling_With_Public_IPs.json
region: us-west-2
name: PreProdEnvStack
Timeout: 10
Environment:
Connections:
- Role: main_branch_IAM_role
Name: "123456789012" #Insert your AWS ID here (inside the double quotes) and remove the comment after #
Name: PreProdEnv
Inputs:
Sources:
- WorkflowSource

Let’s try out our new workflow

$ clear
$ git add . -A
$ git commit -m "Adding main branch workflow"
$ git push

In your browser, navigate to the CI/CD -> Workflows page. You should see the workflow running:

If you click on the run, you will see the workflow.

If you click on the workflow, you will see the steps for it to be deployed.

It will take some time for CodeCatalyst to go through all the stages and deploy the CloudFormation stack. The stack deployment time depends on the resources defined in the template.

It will succeed in sometime.

Then, go to variables and copy the website.

See your successful output!

When you paste it on the browser, you will get the following:

--

--

Anish Mahapatra

Senior AI & ML Engineer | Fortune 500 | Senior Technical Writer - Google me. Anish Mahapatra | https://www.linkedin.com/in/anishmahapatra/