Post

How to create serverless applications with CDK and SAM

Posted
By Alejandro Lazaro Chueca 12 min read

TLDR

A serverless application is more than just a Lambda Function. It is a combination of Lambda functions, event sources, APIs, databases, and other resources that work together to perform tasks.

Creating serverless resources via the AWS Console is quick and easy, but it’s advisable to use this approach only for testing purposes during the learning phase. After that, and thinking about how to use the resources in a real project, you will need:

  • IaC: to create your resources in a way that allows you to recreate them easily
  • Version control: to track all the code modifications
  • CI/CD: to automate the release process

Does anyone use CloudFormation or Terraform to manage their serverless resources? Possibly but come on, there is a better way!

To manage your serverless resources, the natural way to do it is using the following IaC technologies:

  • SAM (Serverless Application Model) / Serverless Framework: declarative option with templates. It is a specific framework for Serverless applications.
  • CDK / Pulumi: add a level of abstraction and allow you to define the infrastructure with modern programming languages.

In this article, we will review the approach to combining both CDK + SAM.

By the way, CDK + SAM is my preferred approach: you get the best of the 2 options! What do you think? You can share your opinion in the comments!

CDK vs SAM

In the linked articles below, you will find information about CDK and SAM.

What do CDK and SAM have in common?

Both…

  • Are open-source, Apache-licensed software development frameworks
  • Provide Infrastructure as Code (IaC)
  • Use AWS CloudFormation behind the scenes to deploy resources
  • Provide a CLI to build and deploy applications
  • Are well integrated with AWS build pipelines
  • Support component reuse

What are the main differences between CDK and SAM?

 CDKSAM
To declare resourcesUses familiar programming languagesUses JSON or YAML
Dynamic referencesNative language capabilitiesPseudo parameters and logical functions
TestingNot supported natively (you could use SAM)Supported (also debug)
IaC resourcesAllFocus on serverless but you can use CloudFormation to create any AWS resource
ComplexityVery lowMedium, based on CloudFormation
MaintainabilityHigherMedium

Demo: CDK + SAM

From Jan 6, 2022, AWS announced the general availability of AWS SAM CLI support for local testing of AWS CDK applications. It means that you can use SAM over your CDK project to test your resources!. Now 2 years have passed, so this integration is mature!

So… we will use a new CDK project to show how to work the combination of CDK + SAM.

The source code is available here. This repository has several CDK projects but first, we will use the v1-simple

simple-webservice-v1

Prepare to test

With CDK, when you run cdk synth, it will synthesize a stack defined in your app into a CloudFormation template in a json file in the cdk.out folder.

However, SAM uses a yaml template, template.yaml or template.yml, in the root folder.

Also, to test locally, you will need this file created or you will receive an error

1
2
3

> sam local invoke
Error: Template file not found at 
/.../aws-cdk-simple-webservice/template.yml

Therefore, we have to run cdk synth and store the result in one file with the name template.yml.

You have to use --no-staging because it is required for SAM CLI to local debug the source files. It will disable the copy of assets which allows local debugging to reference the original source files

1

cdk synth --no-staging > template.yml

Testing Lambda Functions

Now, you have a template.yml file and can run the SAM command to test your Lambda function.

1
2
3
4
5
6
7
8
9
10

> sam local invoke
Invoking index.handler (nodejs14.x)
Skip pulling image and use local one: public.ecr.aws/sam/emulation-nodejs14.x:rapid-1.46.0-x86_64.

Mounting /Users/alazaroc/Documents/MyProjects/github/aws/cdk/aws-cdk-simple-webservice/v1-simple/functions/simplest-example as /var/task:ro,delegated inside runtime container
START RequestId: 03d4ef7d-47b4-4ad2-a491-d0e5fc797ece Version: $LATEST
2022-04-22T21:00:38.952Z 03d4ef7d-47b4-4ad2-a491-d0e5fc797ece INFO request: {}
END RequestId: 03d4ef7d-47b4-4ad2-a491-d0e5fc797ece
REPORT RequestId: 03d4ef7d-47b4-4ad2-a491-d0e5fc797ece Init Duration: 0.39 ms Duration: 205.98 ms Billed Duration: 206 ms Memory Size: 512 MB Max Memory Used: 512 MB
{"statusCode":200,"headers":{"Content-Type":"text/html"},"body":"You have connected with the Lambda!"}%

The Lambda returns the following body: You have connected with the Lambda!

Testing Lambda Functions with input data

If your Lambda Functions need input data, you can generate it from SAM CLI with the command generate-event

1

sam local generate-event [OPTIONS] COMMAND [ARGS]...

You can use this command to generate sample payloads from different event sources such as S3, API Gateway, SNS, and so on. These payloads contain the information that the event sources send to your Lambda functions.

Alternatively, you can add the input data using the -e option with the invoke command.

1
2
3
4
5
6
7
8

> sam local invoke -e test/events/simple-event.json
Invoking index.handler (nodejs14.x)
Skip pulling image and use local one: public.ecr.aws/sam/emulation-nodejs14.x:rapid-1.46.0-x86_64.

Mounting /Users/alazaroc/Documents/MyProjects/github/aws/cdk/aws-cdk-simple-webservice/v1-simple/functions/simplest-example as /var/task:ro,delegated inside runtime container
} "rawPath": "/test"706Z 992499c1-83c4-408d-966b-2e13f5955cbc INFO input: {
{"statusCode":200,"headers":{"Content-Type":"text/html"},"body":"You have connected with the Lambda!"}END RequestId: 992499c1-83c4-408d-966b-2e13f5955cbc
REPORT RequestId: 992499c1-83c4-408d-966b-2e13f5955cbc Init Duration: 0.89 ms Duration: 244.32 ms Billed Duration: 245 ms Memory Size: 512 MB Max Memory Used: 512 MB

Testing API Gateway

You have to run sam local start-api

1
2
3
4

> sam local start-api
Mounting simplest-lambda at http://127.0.0.1:3000$default [X-AMAZON-APIGATEWAY-ANY-METHOD]
You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2022-04-23 00:03:58  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)

You can access http://127.0.0.1:3000/ to connect with your Lambda Function

testing api-gateway-from-web-v1

If you review your previous console, it will be updated when you access your API Gateway:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

> sam local start-api
default [X-AMAZON-APIGATEWAY-ANY-METHOD]
You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2022-04-23 00:03:58  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)
Invoking index.handler (nodejs14.x)
Skip pulling image and use local one: public.ecr.aws/sam/emulation-nodejs14.x:rapid-1.46.0-x86_64.

Mounting /Users/alazaroc/Documents/MyProjects/github/aws/cdk/aws-cdk-simple-webservice/v1-simple/functions/simplest-example as /var/task:ro,delegated inside runtime container
START RequestId: 8ed4a0a8-18bc-43af-b759-ad0668784351 Version: $LATEST
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Ge    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng} "isBase64Encoded": falsehost"01 +0000",-11ba1c012bf1",
END RequestId: 8ed4a0a8-18bc-43af-b759-ad0668784351
REPORT RequestId: 8ed4a0a8-18bc-43af-b759-ad0668784351 Init Duration: 0.51 ms Duration: 230.55 ms Billed Duration: 231 ms Memory Size: 512 MB Max Memory Used: 512 MB
2022-04-23 00:11:05 127.0.0.1 - - [23/Apr/2022 00:11:05] "GET / HTTP/1.1" 200 -
Invoking index.handler (nodejs14.x)
Skip pulling image and use local one: public.ecr.aws/sam/emulation-nodejs14.x:rapid-1.46.0-x86_64.

Mounting /Users/alazaroc/Documents/MyProjects/github/aws/cdk/aws-cdk-simple-webservice/v1-simple/functions/simplest-example as /var/task:ro,delegated inside runtime container
START RequestId: 5759a74a-40b5-4a7e-8362-eec719ae44a7 Version: $LATEST
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Ge} "isBase64Encoded": falseco"t"01 +0000",-11ba1c012bf1",g+xml,image/*,*/*;q=0.8",
END RequestId: 5759a74a-40b5-4a7e-8362-eec719ae44a7
REPORT RequestId: 5759a74a-40b5-4a7e-8362-eec719ae44a7 Init Duration: 0.50 ms Duration: 238.45 ms Billed Duration: 239 ms Memory Size: 512 MB Max Memory Used: 512 MB
2022-04-23 00:11:06 127.0.0.1 - - [23/Apr/2022 00:11:06] "GET /favicon.ico HTTP/1.1" 200 -

Bonus: Testing DynamoDB

Ok, testing a mocked Lambda Function is the “hello world” example and not very useful, but what about a Lambda Function that connects to a DynamoDB table?

We will update our Lambda Function to store the data in a DynamoDB table, so we will use the v2-dynamodb example in the repository.

This code is based on the pattern defined in the web cdkpatterns as the simple webservice

simple-webservice-v2

Case 1: Testing cloud DynamoDB

When you try to locally test a Lambda Function that stores data in a DynamoDB table, it will automatically attempt to connect to the DynamoDB service of your AWS Account.

So, to test your Account DynamoDB tables, you have to do nothing.

1
2
3
4
5
6
7
8
9
10

> sam local invoke -e test/events/simple-event.json
Invoking index.handler (nodejs14.x)
Skip pulling image and use local one: public.ecr.aws/sam/emulation-nodejs14.x:rapid-1.46.0-x86_64.

Mounting /Users/alazaroc/Documents/MyProjects/github/aws/cdk/aws-cdk-simple-webservice/v2-dynamodb/functions/dynamodb-example as /var/task:ro,delegated inside runtime container
} "rawPath": "/test"492Z 69d093d2-083c-46e7-a318-636ed94d7e47 INFO request: {
2022-04-23T06:38:56.797Z 69d093d2-083c-46e7-a318-636ed94d7e47 ERROR Invoke Error  {"errorType":"ResourceNotFoundException","errorMessage":"Requested resource not found","code":"ResourceNotFoundException","message":"Requested resource not found","time":"2022-04-23T06:38:56.787Z","requestId":"RCJDUN8DRCPPOS3SR3034ETK8VVV4KQNSO5AEMVJF66Q9ASUAAJG","statusCode":400,"retryable":false,"retryDelay":49.42319019990148,"stack":["ResourceNotFoundException: Requested resource not found","    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:52:27)","    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)","    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)","    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)","    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)","    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)","    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10","    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)","    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:688:12)","    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:116:18)"]}
{"errorType":"ResourceNotFoundException","errorMessage":"Requested resource not found","trace":["ResourceNotFoundException: Requested resource not found","    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:52:27)","    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)","    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)","    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)","    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)","    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)","    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10","    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)","    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:688:12)","    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequentialEND RequestId: 69d093d2-083c-46e7-a318-636ed94d7e47
REPORT RequestId: 69d093d2-083c-46e7-a318-636ed94d7e47 Init Duration: 0.98 ms Duration: 928.39 ms Billed Duration: 929 ms Memory Size: 512 MB Max Memory Used: 512 MB
_executor.js:116:18)"]}%

If you don’t deploy your CDK project before attempting to test it, you will get the following ERROR: "errorType":"ResourceNotFoundException","errorMessage":"Requested resource not found"

Of course, you can set your AWS account in your SAM CLI using the profile command.

1
2
3

$ sam local invoke -e test/events/simple-event.json profile test
Invoking index.handler (nodejs14.x)
...

Case 2: Testing local DynamoDB

You may want to test your Lambda function locally instead of connecting to your DynamoDB account, so we will do the following:

  • Download the DynamoDB Docker image
  • Run the DynamoDB Docker image locally
  • Set up DynamoDB: create tables, insert data, and test it
  • Change your Lambda Function code
  • Test DynamoDB locally

Download the DynamoDB Docker image

First, download the DynamoDB Docker image.

1
2
3
4
5
6
7
8
9

> docker pull amazon/dynamodb-local
Using default tag: latest
latest: Pulling from amazon/dynamodb-local
3a461b3ae562: Pull complete
14d349bd5978: Pull complete
3e361eec6409: Pull complete
Digest: sha256:07e740ad576acdcfdc48676f9a153a93a8e35436ea36942d4c14939caeca8851
Status: Downloaded newer image for amazon/dynamodb-local:latest
docker.io/amazon/dynamodb-local:latest

Run the DynamoDB Docker image locally

Next, execute the locally downloaded DynamoDB Docker image.

This terminal tab will be kept running and you will have to open another one.

1
2
3
4
5
6
7
8

> docker run -p 8000:8000 amazon/dynamodb-local
Initializing DynamoDB Local with the following configuration:
Port: 8000
InMemory: true
DbPath: null
SharedDb: false
shouldDelayTransientStatuses: false
CorsParams: *

This command will not persist data in the local DynamoDB.

Create a local DynamoDB table

To create a local DynamoDB table named hits with a path partition key, execute the following command:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

> aws dynamodb create-table --table-name hits --attribute-definitions AttributeName=path,AttributeType=S --key-schema AttributeName=path,KeyType=HASH --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1 --endpoint-url http://localhost:8000
{
    "TableDescription": {
        "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/hits",
        "AttributeDefinitions": [
            {
                "AttributeName": "path",
                "AttributeType": "S"
            }
        ],
        "ProvisionedThroughput": {
            "NumberOfDecreasesToday": 0,
            "WriteCapacityUnits": 1,
            "LastIncreaseDateTime": 0.0,
            "ReadCapacityUnits": 1,
            "LastDecreaseDateTime": 0.0
        },
        "TableSizeBytes": 0,
        "TableName": "hits",
        "TableStatus": "ACTIVE",
        "KeySchema": [
            {
                "KeyType": "HASH",
                "AttributeName": "path"
            }
        ],
        "ItemCount": 0,
        "CreationDateTime": 1650668228.617
    }
}

Add values to our local DynamoDB table

We will add two elements:

  • path: “/test”
  • path: “/hello”
1
2

aws dynamodb put-item --table-name hits --item '{ "path": {"S": "/test"} }' --return-consumed-capacity TOTAL --endpoint-url http://localhost:8000
aws dynamodb put-item --table-name hits --item '{ "path": {"S": "/hello"} }' --return-consumed-capacity TOTAL --endpoint-url http://localhost:8000

Scan your table locally

We check that our table has the created elements:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

> aws dynamodb scan --table-name hits --endpoint-url http://localhost:8000
{
    "Count": 2,
    "Items": [
        {
            "path": {
                "S": "/test"
            }
        },
        {
            "path": {
                "S": "/hello"
            }
        }
    ],
    "ScannedCount": 2,
    "ConsumedCapacity": null
}

Change Lambda Function code

We need to update our Lambda Function code to tell DynamoDB to read from our local DynamoDB service:

You have to use your specific Docker endpoint

1
2
3
4
5
6
7
8

if (process.env.AWS_SAM_LOCAL) {
  // mac
  dynamo.endpoint = new AWS.Endpoint("http://docker.for.mac.localhost:8000/");
  // windows
  // dynamo.endpoint = new AWS.Endpoint("http://docker.for.windows.localhost:8000/");
  // linux
  // dynamo.endpoint = new AWS.Endpoint("http://127.0.0.1:8000");
}

Test DynamoDB locally

In summary, we have:

  • the DynamoDB service locally running
  • a table (hits)
  • 2 elements
    • path=/test
    • path=/hello

Now we are going to test our Lambda Function which will insert data into our local DynamoDB table.

1
2
3
4
5
6
7
8
9
10

> sam local invoke -e test/events/simple-event.json
Invoking index.handler (nodejs14.x)
Skip pulling image and use local one: public.ecr.aws/sam/emulation-nodejs14.x:rapid-1.46.0-x86_64.

Mounting /Users/alazaroc/Documents/MyProjects/github/aws/cdk/aws-cdk-simple-webservice/v2-dynamodb/functions/dynamodb-example as /var/task:ro,delegated inside runtime container
} "rawPath": "/test"036Z eaf85e61-e9a2-4b49-9953-d247f9794fb8 INFO request: {
2022-04-22T23:54:20.130Z eaf85e61-e9a2-4b49-9953-d247f9794fb8 INFO inserted counter for /test
END RequestId: eaf85e61-e9a2-4b49-9953-d247f9794fb8
REPORT RequestId: eaf85e61-e9a2-4b49-9953-d247f9794fb8 Init Duration: 0.48 ms Duration: 697.40 ms Billed Duration: 698 ms Memory Size: 512 MB Max Memory Used: 512 MB
{"statusCode":200,"headers":{"Content-Type":"text/html"},"body":"You have connected with the Lambda and store the data in the DynamoDB table!"}

If we scan the table again, we can review that in the “/test” element will be a new hits column and 2 values:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

> aws dynamodb scan --table-name hits --endpoint-url http://localhost:8000
{
    "Count": 2,
    "Items": [
        {
            "path": {
                "S": "/test"
            },
            "hits": {
                "N": "2"
            }
        },
        {
            "path": {
                "S": "/hello"
            }
        }
    ],
    "ScannedCount": 2,
    "ConsumedCapacity": null
}

If you run your function more times, the value of hits will be updated.

And, of course, you can also test it from API Gateway:

1
2
3
4

> sam local start-api
Mounting dynamodb-lambda at http://127.0.0.1:3000$default [X-AMAZON-APIGATEWAY-ANY-METHOD]
You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2022-04-25 19:53:01  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)

testing api-gateway-from-web-v2

Serverless
how-to cdk sam iac serverless comparative level-300
This post is licensed under CC BY 4.0 by the author.