Comprehensive Guide to Amazon API Gateway Integrations

Posted Feb 10, 2024

By Alejandro Lazaro Chueca 17 min read

This is my second article about Amazon API Gateway. You can find the other one here: Amazon API Gateway exposed: choosing the right endpoint strategy for your API.
Neither of these articles is an introductory piece on Amazon API Gateway.

Introduction

Beyond the commonly known Lambda Function integration, API Gateway offers the option to connect with AWS services, HTTP endpoints, and even private services securely in VPCs. This article aims to unfold the layers of API Gateway integrations, covering:

AWS Lambda Functions: The backbone of serverless APIs.
HTTP Endpoints: Bridging your APIs with the world.
Mock Responses: Simplifying development with simulated responses.
AWS Services: Tightly integrating with the AWS ecosystem.
Private Services through VPC Links: Securely connecting to internal resources.

Lambda Function Integration

Connecting AWS Lambda functions to Amazon API Gateway is a highlight of serverless computing, offering unparalleled scalability and flexibility. This powerful synergy enables developers to deploy serverless applications that dynamically scale to meet user demand, freeing them from the conventional challenges of server maintenance.

How it works

API Gateway can trigger Lambda functions in response to HTTP requests, serving as a highly adaptable interface for any application or backend service. This integration is pivotal for running code in a serverless environment, where operational management and scalability concerns are abstracted away.

Integration options

Lambda integrations come in two flavors, each catering to different needs:

Proxy Integration: Acts as a straightforward pass-through to the Lambda function, passing every detail of the HTTP request (headers, body, query parameters). This method is ideal for rapid API setup, requiring minimal configuration while providing maximum flexibility.
Non-Proxy Integration: Offers granular control over the request and response flow between API Gateway and Lambda. By utilizing Velocity Template Language (VTL), developers can tailor the request transformation before it reaches Lambda and likewise adjust the response before it’s returned to the client. This approach suits cases where specific request/response formats must be adhered to.

Key Advantages

Scalability: Automatically scales with incoming request volume, ensuring performance remains consistent. Cost-Effectiveness: Charges are solely based on the number of requests and the execution time, excluding the cost of the integration itself. Ease of Integration: Simplifies the process of connecting your API to Lambda functions, streamlining deployment.

Visual representation

sequenceDiagram
    participant Client as Client
    participant APIG as API Gateway
    participant Lambda as AWS Lambda
    Client->>+APIG: HTTP Request
    APIG->>+Lambda: Invoke Function
    Lambda-->>-APIG: Function Response
    APIG-->>-Client: HTTP Response

This diagram succinctly illustrates the request-response cycle facilitated by Lambda integration, highlighting the efficiency and simplicity of this approach.

Performance and security

Performance: Benefits from AWS Lambda’s inherent auto-scaling capabilities, offering high performance.
Security: Governed by IAM roles and policies, ensuring robust access control and data protection.

Cost Implications

While there’s no direct charge for integrating Lambda with API Gateway, costs are incurred based on request volume and the duration of Lambda function execution, following AWS Lambda pricing models.

Practical Applications

Data Processing: Ideal for applications requiring real-time data analysis or manipulation.
Real-Time Processing: Supports interactive applications by providing immediate backend responses.
Serverless Backend Services: Enables the development of fully serverless architectures for backend services, from web applications to data APIs.

Implementation example

Consider a scenario where you need to process incoming order data in XML format, but your Lambda function expects JSON. Using a non-proxy integration, you can configure API Gateway to transform the XML into JSON before it reaches the Lambda function. Conversely, if the Lambda function returns JSON, you can set API Gateway to convert this JSON back into XML for the client.

Configuration Code Sample (Terraform)

Below is a simplified example using Terraform to demonstrate setting up a proxy integration with AWS Lambda:

  
resource "aws_api_gateway_rest_api" "MyApi" {
  name        = "LambdaProxyApi"
}

resource "aws_api_gateway_resource" "MyResource" {
  rest_api_id = aws_api_gateway_rest_api.MyApi.id
  parent_id   = aws_api_gateway_rest_api.MyApi.root_resource_id
  path_part   = "{proxy+}"
}

resource "aws_api_gateway_method" "MyLambdaIntegration" {
  rest_api_id   = aws_api_gateway_rest_api.MyApi.id
  resource_id   = aws_api_gateway_resource.MyResource.id
  http_method   = "POST"
  authorization = "NONE"
  integration {
    type              = "AWS_PROXY"
    http_method       = "POST"
    uri               = "arn:aws:apigateway:${var.region}:lambda:path/2015-03-31/functions/${var.lambda_function_arn}/invocations"
    integration_http_method = "POST"
  }
}

This Terraform script sets up an API Gateway with a proxy integration to an AWS Lambda function, illustrating how effortlessly these components can be linked to harness the power of serverless computing.

HTTP Integrations

HTTP integrations enable API Gateway to act as a proxy for HTTP endpoints. This includes integration with HTTP-enabled AWS services, external HTTP APIs, and microservices. This flexibility makes HTTP integrations a cornerstone for extending the functionality of your applications beyond the AWS ecosystem.

API Gateway can act as a robust intermediary, forwarding requests from clients to any HTTP endpoint and then returning the response to the client. This capability is especially valuable for integrating with RESTful APIs across the internet or within your AWS environment, enabling seamless interaction with a wide array of services.

Key Advantages

Flexibility: Connect to any HTTP endpoint, AWS-hosted or externally hosted.
Efficiency: Reduce latency by directly integrating your API with HTTP-supported applications or services.
Generic HTTP Proxy: Uses standard HTTP methods and integrates with services that may not have native API Gateway integrations.

Visual representation

sequenceDiagram
    participant Client as Client
    participant APIG as API Gateway
    participant HTTP as HTTP Endpoint
    Client->>+APIG: HTTP Request
    APIG->>+HTTP: Forward Request
    HTTP-->>-APIG: Endpoint Response
    APIG-->>-Client: HTTP Response

This diagram illustrates the process flow, highlighting how API Gateway serves as an intermediary between clients and HTTP services.

Performance and security

Performance: It depends on the targeted HTTP endpoint, underscoring the importance of choosing responsive services for integration.
Security: Use HTTPS to secure data in transit.

Cost Implications

Integrating with HTTP endpoints incurs no additional fees beyond API Gateway’s standard request pricing. Note, however, that data transfer costs and any charges associated with the external service apply, making it essential to monitor usage to manage expenses effectively.

Practical Applications

HTTP integrations are versatile, supporting a wide range of applications:

Third-party API Integration: Ideal for incorporating functionality from external services, such as social media feeds, payment processing, or weather data.
Microservices Architecture: Facilitates a decoupled architecture by allowing individual microservices, possibly hosted on ECS or EC2, to communicate through API Gateway.
Content Delivery and Management: Streamlines content distribution and management by serving as a gateway to content hosted across different platforms.

Implementation example

Suppose you want to integrate your API Gateway with a third-party weather service API to fetch weather updates. This service provides an HTTP endpoint that accepts GET requests with a city name as a query parameter and returns weather data in JSON format.

Configuration Code Sample (Terraform)

Below is a simplified Terraform script demonstrating how to configure HTTP proxy integration, illustrating the ease with which API Gateway can connect to external services:

  
resource "aws_api_gateway_rest_api" "MyApi" {
  name = "HttpProxyApi"
}

resource "aws_api_gateway_resource" "MyResource" {
  rest_api_id = aws_api_gateway_rest_api.MyApi.id
  parent_id   = aws_api_gateway_rest_api.MyApi.root_resource_id
  path_part   = "weather"
}

resource "aws_api_gateway_method" "MyHttpIntegration" {
  rest_api_id   = aws_api_gateway_rest_api.MyApi.id
  resource_id   = aws_api_gateway_resource.MyResource.id
  http_method   = "GET"
  authorization = "NONE"
  
  integration {
    type                    = "HTTP_PROXY"
    integration_http_method = "GET"
    uri                     = "https://example.com/weather"
  }
}

This example underscores the straightforward nature of setting up HTTP integrations, empowering developers to rapidly extend their APIs’ capabilities.

Mock Integrations

Mock integrations allow you to simulate API behavior without backend integration. With mock integrations, you can return a fixed response to the API caller directly from API Gateway, facilitating the validation of API definitions and client-side development independent of the backend.

Key Advantages

Mock integrations serve multiple purposes, significantly reducing the time and resources typically required for API development:

Fast Prototyping: Quickly mock API responses for client development without the need for a backend.
Testing and Validation: Simplify the process of testing API contracts and response structures, ensuring that API consumers can handle various response types effectively.

Visual representation

sequenceDiagram
    participant Client as Client
    participant APIG as API Gateway
    Client->>+APIG: HTTP Request
    APIG-->>-Client: Mock Response

Performance and security

Performance: Delivers immediate responses, bypassing the latency and processing time associated with backend services.
Security: Leverages API Gateway’s built-in security features, ensuring that mock endpoints are protected according to the same standards as any other API endpoint.

Cost Implications

Mock integrations are cost-effective, incurring charges only based on the number of API calls made, without additional fees for the mock functionality itself. This makes it an economical choice for developing, testing, and iterating on APIs.

Practical Applications

Mock integrations are particularly useful in scenarios such as:

API Prototyping and Testing: Developers can quickly create mock endpoints to test API interactions, client handling of various response types, and error conditions without waiting for backend implementation.
Static Data Responses: Ideal for scenarios where an API needs to return consistent, predictable data for frontend development or testing purposes.
Error Simulation: Facilitates the testing of client applications’ resilience and error handling by simulating API failures or unexpected responses.

Implementation example

Imagine you’re working on a new application feature that requires specific API responses for frontend logic testing. By setting up a mock integration, you can configure API Gateway to return a predetermined JSON response, such as a 200 status code with {“message”: “This is a mock response”}, enabling the frontend team to advance without backend dependencies.

Configuration Code Sample (Terraform)

Below is a Terraform example that demonstrates configuring a mock integration, showcasing the simplicity of setting up mocked responses:

  
resource "aws_api_gateway_rest_api" "MyApi" {
  name = "MockApi"
}

resource "aws_api_gateway_resource" "MyResource" {
  rest_api_id = aws_api_gateway_rest_api.MyApi.id
  parent_id   = aws_api_gateway_rest_api.MyApi.root_resource_id
  path_part   = "mockpath"
}

resource "aws_api_gateway_method" "MyMockIntegration" {
  rest_api_id   = aws_api_gateway_rest_api.MyApi.id
  resource_id   = aws_api_gateway_resource.MyResource.id
  http_method   = "GET"
  authorization = "NONE"
  
  integration {
    type = "MOCK"
    request_templates = {
      "application/json" = "Action=MockResponse&{ \"statusCode\": 200 }"
    }
    integration_responses = [{
      status_code = "200"
      response_templates = {
        "application/json" = "{\"message\": \"This is a mock response\"}"
      }
    }]
  }
}

This script exemplifies how to establish a mock integration within API Gateway using Terraform, enabling developers to simulate API endpoints swiftly.

AWS Service Integrations

AWS Service integrations allow API Gateway to interact directly with other AWS services without writing custom integration code. This integration simplifies architecture by enabling direct access to AWS services, such as putting an item into an Amazon DynamoDB table or publishing a message to an Amazon SNS topic directly through API Gateway.

Key Advantages

The direct connection to AWS services through API Gateway not only simplifies the architecture but also enhances the overall efficiency and scalability of your applications:

Simplicity: Eliminates the need for intermediary services or layers, allowing for a cleaner architecture.
Direct access: Enable APIs to perform actions directly on AWS services.

Difference between AWS Services integration and HTTP integration:
AWS Service Integrations are a subset of what you can achieve with HTTP Integrations but offer a more integrated, AWS-centric approach simplifying the configuration by abstracting the underlying HTTP requests into AWS SDK-like actions directly within API Gateway.

Visual representation

sequenceDiagram
    participant Client as Client
    participant APIG as API Gateway
    participant AWSService as AWS Service
    Client->>+APIG: HTTP Request
    APIG->>+AWSService: Invoke Service Action
    AWSService-->>-APIG: Service Response
    APIG-->>-Client: HTTP Response

This diagram exemplifies how API Gateway acts as a conduit between clients and AWS services, streamlining the request and response flow.

Performance and security

Performance: AWS Service Integrations are built on the robust AWS infrastructure, ensuring high availability and reliability.
Security: Leveraging AWS Identity and Access Management (IAM) roles and permissions, these integrations offer fine-grained security controls, ensuring that only authorized entities can execute specific actions.

Cost Implications

Similar to HTTP integrations, the cost is based on the API Gateway’s request pricing. Additionally, standard operation charges for the integrated AWS service apply.

Practical Applications

The versatility of AWS Service Integrations supports a broad spectrum of use cases:

Direct Database Interaction: Streamline data storage and retrieval by interacting directly with DynamoDB.
Notifications and Messaging: Utilize SNS or SQS for efficient message broadcasting and queueing.
Workflow Orchestration: Leverage AWS Step Functions for complex workflow execution directly from API Gateway.

Implementation example

Imagine a scenario where you need real-time data from a DynamoDB table accessible via your API. Setting up an AWS Service Integration allows API Gateway to directly query DynamoDB, bypassing the need for a Lambda function. This direct approach reduces latency, simplifies the architecture, and can lower costs.

Configuration Code Sample (Terraform)

Below is an enhanced Terraform script that outlines configuring AWS Service Integration for DynamoDB:

  
resource "aws_api_gateway_rest_api" "MyApi" {
  name = "DynamoDBServiceApi"
}

resource "aws_api_gateway_resource" "MyResource" {
  rest_api_id = aws_api_gateway_rest_api.MyApi.id
  parent_id   = aws_api_gateway_rest_api.MyApi.root_resource_id
  path_part   = "data"
}

resource "aws_api_gateway_method" "MyDynamoDBIntegration" {
  rest_api_id   = aws_api_gateway_rest_api.MyApi.id
  resource_id   = aws_api_gateway_resource.MyResource.id
  http_method   = "POST"
  authorization = "NONE"
  
  integration {
    type                    = "AWS"
    integration_http_method = "POST"
    uri                     = "arn:aws:apigateway:${var.region}:dynamodb:action/PutItem"
    credentials             = aws_iam_role.apigateway_dynamodb.arn
  }
}

This script demonstrates how to set up an API Gateway method that directly interacts with DynamoDB, showcasing the efficiency and simplicity of AWS Service Integrations.

AWS Console example

In this case, I will show you how to configure the direct integration between Amazon API Gateway and DynamoDB and Step Functions.

The method execution visual flow is common for both cases:

I will show you only the key configuration for each option, and you will understand how it works.

Direct integration with Step Functions

This is the integration request. Important values here are:

AWS Service: Step Functions
HTTP method: POST
Action name: StartExecution
Execution Role: you have to create one role for API Gateway with permissions to connect to Step Functions

This part is optional, but if you don’t configure it here, you have to specify in each request the ARN of the Step Functions you want to connect with:

Direct integration with DynamoDB

This is the integration request. Important values here are:

AWS Service: DynamoDB
HTTP method: POST
Action name: Scan
Execution Role: you have to create one role for API Gateway with permissions to connect to Step Functions

Important here define the HTTP method as POST. However, your API Gateway method can be a GET method.

This part is optional, but if you don’t configure it here, you have to specify in each request the ARN of the Step Functions you want to connect with:

VPC Link Integrations

VPC Link integrations enable private integrations that allow API Gateway to securely connect with resources within an Amazon Virtual Private Cloud (VPC). This is particularly useful for accessing HTTP endpoints hosted within your VPC without exposing them to the public internet.

Key Advantages

By leveraging VPC Links, you can maintain the security posture of your back-end systems while ensuring seamless connectivity from API Gateway:

Security: Safeguards back-end systems by avoiding public internet exposure, relying instead on the inherent security features of AWS VPC.
Private Access: Facilitates direct access to VPC-contained resources, such as microservices or databases, from your APIs, enhancing internal communication security and efficiency.

Visual representation

sequenceDiagram
    participant Client as Client
    participant APIG as API Gateway
    participant VL as VPC Link
    participant VPCRes as VPC Resource (ECS, ALB, etc.)
    Client->>+APIG: HTTP Request
    APIG->>+VL: Request via VPC Link
    VL->>+VPCRes: Forward Request
    VPCRes-->>-VL: Response
    VL-->>-APIG: Response via VPC Link
    APIG-->>-Client: HTTP Response

This illustration demonstrates the flow of requests and responses facilitated by VPC Links, highlighting the secure, internal pathway within the VPC.

Performance and security

Performance: The effectiveness of VPC Link integrations largely hinges on your VPC’s network setup and the performance of the targeted back-end services.
Security: VPC Links bolster the security of API integrations by ensuring that traffic between API Gateway and VPC resources does not traverse the public internet, utilizing AWS’s robust VPC security mechanisms.

Cost Implications

Pricing includes an hourly charge for each VPC link, plus data processed charges. This is in addition to the API Gateway’s standard request pricing.

Practical Applications

Employing VPC Link is particularly advantageous for:

Securing Private Microservices: Ideal for architectures where microservices deployed on Amazon ECS or EKS within a VPC need to be securely exposed via API Gateway without public internet access.

Implementation example

If you have a microservice running on Amazon ECS that’s exposed via an Application Load Balancer (ALB) within your VPC, you can use VPC Link to create a secure connection between the API Gateway and the ALB. This setup allows API Gateway to route requests to your ECS service without exposing the service to the public internet. To achieve this, create a VPC Link in the API Gateway pointing to your ALB, and then configure your API’s integration request to route through the VPC Link.

Configuration Code Sample (Terraform)

Here’s an enhanced Terraform script that provides a blueprint for setting up a VPC Link integration:

  
resource "aws_apigateway_vpc_link" "MyVPCLink" {
  name        = "MyVPCLink"
  target_arns = [aws_lb.my_alb.arn]
  description = "VPC Link for API Gateway to access resources in VPC"
}

resource "aws_lb" "my_alb" {
  name               = "my-alb"
  internal           = true
  load_balancer_type = "application"
  security_groups    = [aws_security_group.my_sg.id]
  subnets            = aws_subnet.my_subnet.ids
}

resource "aws_security_group" "my_sg" {
  name        = "my-security-group"
  description = "Security group for ALB"
  vpc_id      = aws_vpc.my_vpc.id
}

resource "aws_vpc" "my_vpc" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_subnet" "my_subnet" {
  count                   = 2
  vpc_id                  = aws_vpc.my_vpc.id
  cidr_block              = count.index == 0 ? "10.0.1.0/24" : "10.0.2.0/24"
  availability_zone       = element(split(",", var.availability_zones), count.index)
  map_public_ip_on_launch = false
}

This Terraform configuration outlines the creation of a VPC Link in API Gateway, targeting an Application Load Balancer (ALB) within a VPC, demonstrating the straightforward process of securing your API’s access to internal services.

Conclusion

Throughout this article, we have explored the various integration options available in Amazon API Gateway, each serving distinct purposes and offering unique advantages. From the serverless prowess of Lambda Function integrations to the robust connectivity of HTTP and AWS Service integrations, as well as the secure, private connections facilitated by VPC Links and the simplicity of Mock Responses for development and testing, API Gateway stands as a versatile tool in the AWS ecosystem. Below is a summary table that recaps these integration types, their benefits, use cases, and cost considerations:

Integration Type	Key Advantages	Use Cases	Cost Implications
Lambda Function	Scalability, Cost-Effectiveness, Ease of Integration	Data Processing, Real-Time Processing	Based on request volume and Lambda function execution
HTTP	Flexibility, Efficiency, Generic HTTP Proxy	Third-party APIs, Microservices	API Gateway request pricing plus external service costs
Mock	Fast Prototyping, Testing and Validation	API Prototyping, Testing, Static Data	Based on API call volume
AWS Service	Simplicity, Direct access	Database Interaction, Notifications	API Gateway request pricing plus AWS service operation charges
VPC Link	Security, Private Access	Securing Private Microservices	Hourly charge for each VPC link plus data processed

This table underscores the flexibility and power of API Gateway, illustrating how it can serve a wide range of applications and architectural needs. By carefully selecting the appropriate integration type, developers can optimize their backend services for performance, cost, and security, ensuring their applications are scalable, efficient, and well-integrated within the AWS landscape.

Incorporating these integrations into your architecture not only enhances the capabilities of your applications but also aligns with best practices for cloud-native development, leveraging the full potential of AWS services to create robust, scalable, and secure applications.

Serverless

This post is licensed under CC BY 4.0 by the author.