This is my second article about SAM. I have explained how to create serverless applications using SAMhere, detailing all the essential information about SAM. It would be beneficial to review that article before proceeding with this one.
We will use sam pipeline to deploy the solution.
I had to create one custom template to be able to deploy the solution using only 1 stage. Using the default templates you have to use 2, and I don’t want to do it for this PoC. This is the GitHub code of my custom template
Introduction
We will add CI/CD to our SAM application through the pipeline integration of the AWS SAM CLI.
As we want to add automation to our deployment process and integrate it with the AWS ecosystem, we will use the AWS Developer tools (CodePipeline and CodeBuild). For the Git repository, I will use GitHub.
This is the SAM project code on GitHub that we will use in the article. In the commit history you can find the evolution of the application through the steps explained.
Add CI/CD to a SAM project
We will create the CI/CD pipeline to implement continuous deployment, so when we push new code, the pipeline deploys our resources automatically.
From AWS documentation: AWS SAM provides a set of default pipeline templates for multiple CI/CD systems that encapsulate AWS’s deployment best practices. These default pipeline templates use standard JSON/YAML pipeline configuration formats, and the built-in best practices help perform multi-account and multi-region deployments and verify that pipelines cannot make unintended changes to infrastructure.
We want to create a new pipeline in the AWS CodePipeline resource using the SAM templates.
To generate the pipeline for AWS CodePipeline, we have to perform the following tasks in this order:
Create infrastructure resources
Generate the pipeline configuration
Commit the pipeline configuration to the Git repository
Deploy the pipeline
Connect the Git repository with the CI/CD system
After you’ve generated the starter pipeline configuration and committed it to your Git repository, whenever someone commits a code change to that repository your pipeline will be triggered to deploy the new changes automatically.
The Pipelines that use AWS SAM require certain AWS resources, like an IAM user and roles with necessary permissions, an Amazon S3 bucket, and optionally an Amazon ECR repository. You must have a set of infrastructure resources for each deployment stage of the pipeline.
Then, for each stage we need (dev, test, prod…), we have to run sam pipeline bootstrap. This command will create a CloudFormation stack with the name aws-sam-cli-managed-${stage}-pipeline-resources which the necessary resources that SAM needs.
We will create a single stage with the name test:
1
sam pipeline bootstrap
Click to view command output
A new stack is created for our new stage test.
If you want more than 1 stage you should repeat the sam pipeline bootstrap for the new stage (prod?). In this example I only want 1 stage to simplify.
In our SAM project now we have 1 new file containing our stage information:
Step 2: Generate the pipeline configuration
Two-stage pipeline template
To generate the pipeline configuration, run the command sam pipeline init:
1
sam pipeline init
Click to view command output
I have to stop the execution here.
In the console log below, the following is displayed:
You are using the 2-stage pipeline template
Only 1 stage(s) were detected, fewer than what the template requires: 2.
In any real project you should have at least two stages, so you could use the default AWS template.
However, I don't want to create two stages in my CI/CD pipeline, I am testing a simple SAM project, and I only want ONE.
Unfortunately, you can’t do that with the AWS Quick Start Pipeline Templates so I forked the main AWS project and I created a custom template with only ONE stage.
I had to put my custom template in the root folder because otherwise, the AWS SAM CLI doesn’t work.
One-stage pipeline template
In the next execution, I will select the option Custom Pipeline Template Location and use my updated forked repository to create only one stage in the AWS CodePipeline service.
1
sam pipeline init
Click to view command output
Now we have the new files in our project that CodePipeline will use to deploy our code:
Step 3: Commit the pipeline configuration to Git
This step ensures that your CI/CD system recognizes your pipeline configuration and triggers deployments upon code commits.
Step 4: Deploy the pipeline
For AWS CodePipeline you have to deploy the pipeline running sam deploy -t codepipeline.yaml --stack-name <pipeline-stack-name> --capabilities=CAPABILITY_IAM --region <region-X>
Don’t set the same stack name as your SAM application because doing so will overwrite your application’s stack (and delete your application resources).
1
sam deploy -t codepipeline.yaml --stack-name pipeline-sam-app --capabilities=CAPABILITY_IAM
Click to view command output
A new stack has been created to deploy our AWS CodePipeline:
Now, we need to access AWS CodePipeline to check the execution.
However, the first execution has failed as we can see in the following image:
We can see in the CodePipeline flow that all steps have been created:
Source: integrated with GitHub as we indicated before
UpdatePipeline: the pipeline can update itself
BuildAndPackage: the SAM application is built, packaged, and uploaded (with AWS CodeBuild service)
DeployTest: the SAM application is deployed (with AWS CodeBuild service)
The cause of the error was that the connection between GitHub and AWS must be confirmed after being created:
Step 5: Connect the Git repository with the CI/CD system
If you are using GitHub or Bitbucket, after executing the sam deploy command for your pipeline, you need to complete the pending connection in the Settings/Connection section of the Developer Tools.
In addition, you could store a copy of the CodeStarConnectionArn from the output of the sam deploy command, because you will need it if you want to use AWS CodePipeline with another branch than main.
By accessing the Settings/Connections in the Developer Tools, you can validate that the connection is pending approval:
After activating it, we run the pipeline again (by clicking on the Release change button) and now the pipeline ends correctly.
Update the CI/CD steps in the SAM project
Note that now we have a pipeline that first checks for changes in the pipeline itself and then checks the code and deploys the resources.
Step 1: Update the steps in the pipeline automatically
With this pipeline configuration (with UpdatePipeline step) all the changes that we make in the pipeline will be updated automatically.
We want to test the automatic update of the pipeline if we make some changes.
So, let’s change the pipeline steps to:
delete the UpdatePipeline step
add the UnitTest step
To do this, we have to update the codepipeline.yaml file with the necessary changes:
comment all the UpdatePipeline step code
uncomment all the UnitTest step code
And after that, commit the changes and push them to the repository:
When we push the changes, the UpdatePipeline step executes an update on the pipeline-sam-app stack (in the CloudFormation service) and it will update the pipeline definition.
As we expected, the pipeline has updated itself and now we have UnitTest step but not UpdatePipeline.
Step 2: Update the steps in the pipeline manually
If you remove the UpdatePipeline step, when you push a change to the repository the pipeline won’t be updated, so you have to run manually the update of the pipeline.
To update the pipeline we have to run again the command sam deploy -t codepipeline.yaml --stack-name pipeline-sam-app --capabilities=CAPABILITY_IAM.
Now, we will remove the UnitTest stage.
1
sam deploy -t codepipeline.yaml --stack-name pipeline-sam-app --capabilities=CAPABILITY_IAM
Click to view command output
And the pipeline will be updated:
After all the changes that we made:
Now, we have 3 steps in our CodePipeline:
Source: integrated with GitHub as we indicated before
BuildAndPackage: the SAM application is built, packaged, and uploaded (with AWS CodeBuild service)
DeployTest: the SAM application is deployed (with AWS CodeBuild service)
To summarize the current state:
If we change the SAM application and update the code in our GitHub repository, the pipeline will update the resources in our AWS account.
If we want to update the pipeline itself, we have to update the specific pipeline files in our SAM application, and then run manually the command
What happens if you want to update the pipeline itself? Then, with the recent changes we did, you have to:
modify the codepipeline.yaml file in our SAM application
manually execute the command sam deploy -t codepipeline.yaml --stack-name pipeline-sam-app --capabilities=CAPABILITY_IAM.
Clean up
We have created 3 or 4 stacks in CloudFormation related to SAM, depending if you already had used AWS SAM before (then the aws-sam-cli-managed-default already existed):
aws-sam-cli-managed-default: general SAM resources
sam-app: application code
pipeline-sam-app: CI/CD of the sam-app
aws-sam-cli-managed-test-pipeline-resources: test stage resources
If you want to use AWS SAM in the future you could keep the aws-sam-cli-managed-default stack.
You have several ways to delete your resources:
AWS CloudFormation service
AWS CLI
AWS SAM CLI
If you execute the command sam delete, it only will delete the main stack (sam-app) but not the CI/CD pipeline or the stage resources stack, so you have to specify the pipeline stack in the second execution.
1
sam delete --stack-name pipeline-sam-app
And don’t forget to remove the environment stack resources:
1
sam delete --stack-name aws-sam-cli-managed-test-pipeline-resources
Wrapping It Up
That brings us to the end of our journey of integrating CI/CD into AWS SAM projects. Let’s recap the highlights:
Ease of Deployment with SAM CLI: We utilized the sam pipeline command for a streamlined deployment process. It’s efficient and simplifies the deployment of SAM applications.
Customizing the Pipeline: Demonstrating AWS SAM’s adaptability, we successfully used a custom template for a single-stage pipeline, allowing for tailored deployment strategies.
Deployment Automation: Highlighting the power of AWS tools, we’ve automated our deployments, which simplifies application management and ensures our apps are always current.
What’s next?
Further reading:
SAM + CDK: Interested in how AWS SAM and AWS CDK can work together? I’ve explored this in another article: How to create serverless applications with CDK and SAM. It’s a great next step for those looking to expand their serverless architecture knowledge.
Thank you for reading, and I hope this article has been both informative and useful in setting up your CI/CD pipeline. I look forward to hearing your thoughts and experiences with AWS SAM. Feel free to share them in the comments below. Happy coding!