Build a CI / CD Pipeline on AWS
We will build a CI/CD Pipeline on AWS in this blog. We have all read in theory about the CI/CD pipeline, wouldn’t it be great if we were actually able to implement it? Let’s begin!
Pre Requisites
You will need the following to do this hands-on tutorial along with me.
- AWS Account
- AWS CodeCatalyst Account (linked to AWS ID)
I have used this tutorial as a reference.
1. Create IAM Role
First, we will proceed to create a new Stack in CloudFormation Stack.
We need to apply this stack. Save the below code as a .json file. I have named mine codecatalyst.json
CloudFormation console > Create New Stack > Template > Upload a template{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudFormation Template IAM_Role_and_Policies: Template to create 2 new service roles for CodeCatalyst - CloudFormation deployment. **WARNING** The `main_branch_IAM_role` provides full access to AWS resources in EC2 and CloudFormation services, as those will be used by sample CloudFormation template mentioned in the blog. Please use this role carefully and delete it when not required.",
"Resources" : {
"MainBranchRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"Service": [
"codecatalyst.amazonaws.com",
"codecatalyst-runner.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"RoleName" : "main_branch_IAM_role",
"Policies": [
{
"PolicyName": "ProvideAmazonEC2FullAccess",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": "ec2:*",
"Effect": "Allow",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "elasticloadbalancing:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "cloudwatch:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "autoscaling:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:AWSServiceName": [
"autoscaling.amazonaws.com",
"ec2scheduled.amazonaws.com",
"elasticloadbalancing.amazonaws.com",
"spot.amazonaws.com",
"spotfleet.amazonaws.com",
"transitgateway.amazonaws.com"
]
}
}
},
{
"Effect": "Allow",
"Action": [
"cognito-idp:DescribeUserPoolClient"
],
"Resource": "*"
}
]
}
},
{
"PolicyName": "ProvideAWSCloudFormationFullAccess",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"cloudformation:*"
],
"Resource": "*"
}
]
}
}
]
}
},
"PRBranchRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"Service": [
"codecatalyst.amazonaws.com",
"codecatalyst-runner.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"RoleName" : "pr_branch_IAM_role",
"Policies": [
{
"PolicyName": "CloudFormation-PR-policy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"cloudformation:Describe*",
"cloudformation:CreateChangeSet",
"cloudformation:DeleteChangeSet",
"cloudformation:ExecuteChangeSet",
"cloudformation:SetStackPolicy",
"cloudformation:ValidateTemplate",
"cloudformation:List*",
"iam:PassRole"
],
"Resource": "*",
"Effect": "Allow"
}
]
}
},
{
"PolicyName": "ProvideEC2ReadOnlyAccess",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "elasticloadbalancing:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"cloudwatch:ListMetrics",
"cloudwatch:GetMetricStatistics",
"cloudwatch:Describe*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "autoscaling:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"sns:ListSubscriptions",
"sns:ListTopics"
],
"Resource": "*"
}
]
}
}
]
}
}
}
}
If you click “view on designer”, you will see the following design.
We have set the region to be us-west-2. Click on next and enter the stack name below.
CodeCatalyst-IAM-roles
The architecture will be as shown below.
Let the rest of the settings remain at default. Just provide the acknowledgement and hit Submit.
The stack is now being created.
Code Catalyst
AWS Code Catalyst is an integrated development environment (IDE) provided by Amazon Web Services (AWS) that aims to simplify the software development workflow. It combines a comprehensive set of tools, services, and resources, enabling developers to efficiently collaborate, build, test, and deploy applications within a unified environment.
Code Catalyst is fully managed by AWS, so developers can focus on development. It provides a cloud-based development environment that allows developers to rapidly iterate on code and improve their development productivity. It also simplifies the cross-account functionalities of continuous integration and delivery (CI/CD) best practices.
Code Catalyst offers a unified software development service that makes it faster to build and deliver software on AWS. You can learn more and get started building in minutes on the AWS Free Tier at the Code Catalyst website.
Log into Code Catalyst
Create a new space with any name. Mine is:
workspaceanish
Verify in AWS
Once the space is created, go to the AWS Accounts tab, click on your account ID, and then click on Manage roles from the AWS Management Console.
Select your workspace
Add IAM Role.
Select the role we created previously.
main_branch_IAM_role
Next, go back to CodeCatalyst and create a new Project and start from scratch.
Start from Scratch
Add Code Repository
Select Source Repository.
Source Reposiory > Add Repository > Create Repository > 3-tier-app
Add the details for the repository as shown below.
Environment
Lastly, we need to set up the AWS environment where our CloudFormation stack will be deployed by automated workflows. This is my non-production environment where my sample CloudFormation template will be deployed.
From the left navigation panel, select CI/CD, Environments, and click on Create Environment. In the Environment details enter following:
- Environment name:
PreProdEnv - Environment type:
Non-production - Description:
Pre-production environment to learn, test and experiment with CloudFormation - AWS account connection
Connection: Select the AWS account ID that will be used in the workflows to deploy this environment
Setting up a Dev Environment
We will use the IDE AWS Cloud9 for our development.
From the left navigation panel, select Code, Dev Environments, and click on Create Dev Environment. In the Create dev environment and open with AWS Cloud9 dialog box select following:
- Repository:
Clone a repository - Repository: Select the repo that you want to clone. We created a repo
3-tier-appabove, so select same. - Branch:
Work in existing branch - Alias:
bootstrap
AWS Cloud9 IDE
You will now see the IDE.
For this blog, I am using a sample template that deploys a VPC with 2 subnets and publicly accessible Amazon EC2 instances that are in an Auto Scaling group behind a Load Balancer form. Feel free to use the same as I will be making changes to this template and run a pull request workflow.
# go to the root folder of the repo and run following
$ cd 3-tier-app/
$ wget https://raw.githubusercontent.com/build-on-aws/ci-cd-iac-aws-cloudformation/main/cloudformation-templates/VPC_AutoScaling_With_Public_IPs.json
# check git status
$ git status
# add changed files, commit and push to the git repo
$ git add . -A
$ git commit -m "Uploading first CloudFormation template"
$ git push
You can verify if the CloudFormation template is successfully updated in the repo — in the CodeCatalyst console, click Code in the left-side navigation menu, then select Source repositories. A new file VPC_AutoScaling_With_Public_IPs.json should be added to the repo.
Set up Workflows to Deply Non-prod environment
Here, we will use the dev environment using yaml.
# go to the root folder of the repo and run following
mkdir -p .codecatalyst/workflows
touch .codecatalyst/workflows/main_branch.yamlOpen this in your IDE as shown in the image below.
3-tier-app > .codecatalyst > worflows > main_branch.yaml
Double click on main_branch.yaml
I will give you the code to paste in the yaml file. Just make sure to replace the AWS ID with your AWS ID, which you can find in the home console.
You can find your AWS ID if you click on your name.
Name: Main_Branch_Workflow
SchemaVersion: "1.0"
# Optional - Set automatic triggers.
Triggers:
- Type: Push
Branches:
- main
# Required - Define action configurations.
Actions:
DeployAWSCloudFormationstack_7c:
Identifier: aws/cfn-deploy@v1
Configuration:
parameter-overrides: SSHLocation=54.10.10.2/32,WebServerInstanceType=t2.micro
capabilities: CAPABILITY_IAM,CAPABILITY_NAMED_IAM,CAPABILITY_AUTO_EXPAND
template: VPC_AutoScaling_With_Public_IPs.json
region: us-west-2
name: PreProdEnvStack
Timeout: 10
Environment:
Connections:
- Role: main_branch_IAM_role
Name: "123456789012" #Insert your AWS ID here (inside the double quotes) and remove the comment after #
Name: PreProdEnv
Inputs:
Sources:
- WorkflowSource
Let’s try out our new workflow
$ clear
$ git add . -A
$ git commit -m "Adding main branch workflow"
$ git push
In your browser, navigate to the CI/CD -> Workflows page. You should see the workflow running:
If you click on the run, you will see the workflow.
If you click on the workflow, you will see the steps for it to be deployed.
It will take some time for CodeCatalyst to go through all the stages and deploy the CloudFormation stack. The stack deployment time depends on the resources defined in the template.
It will succeed in sometime.
Then, go to variables and copy the website.
See your successful output!
When you paste it on the browser, you will get the following:
