How open source tools can help you with your code

Introduction

This is the last article of a series of 3 about open source tools and AWS. If you haven’t read them I recommend you to do it.

Getting started with AWS open-source tools: here
Open source tools to analyze your AWS environment: here

The current article is about how open source tools can help you with your code: to generate it (IaC), validate it (policy as code and compliance) and analyze it (static analysis/credentials)

I have created the following sections:

IaC tools
Analysis code: credentials and static analysis
Policy as Code
Compliance

Generate IaC

This section is about open source tools which will help you to generate Infrastructure as Code directly from your existing resources.

former2: generate IaC in many different languages (CloudFormation, CDK, Terraform, ...)
terracognita: generates your infrastructure as code on Terraform configuration
AirIAM: scans existing IAM usage patterns and provides a simple method to migrate IAM configurations into a right-sized Terraform plan

former2

Former2 allows you to generate Infrastructure-as-Code outputs from your existing resources within your AWS account. By making the relevant calls using the AWS JavaScript SDK, Former2 will scan across your infrastructure and present you with the list of resources for you to choose which to generate outputs for.

Supported IaC output:

CloudFormation
Terraform
Troposphere
CDK v1
CDK v2
CDK (Terraform)
Pulumi
Diagram
Raw Output (Debug)

Why use it: Generate IaC from AWS account resources created manually using an extension for your browser. You will configure it through the former2.com website.

Is it popular?: On github: Watch 36; Fork 190; Stars 1.6K

Recently updated?: Yes, two month ago (on Dec 5, 2022)

URL: https://github.com/iann0036/former2

More information: https://former2.com/#section-dashboard

terracognita

Reads from existing public and private cloud providers (reverse Terraform) and generates your infrastructure as code on Terraform configuration.

Terracognita currently imports AWS, GCP, AzureRM and VMware vSphere cloud providers as Terraform (v1.1.9) resource/state.

Why use it: yo are able to create the terraform templates of all your AWS infrastructure with zero effort and this is something wonderful, isn’t it?

Is it popular?: On github: Watch 33; Fork 119; Stars 1.5k

Recently updated? No. Last commit on Aug 25, 2022. In total, 625 commits

URL: https://github.com/cycloidio/terracognita/

AirIAM

AirIAM scans existing IAM usage patterns and provides a simple method to migrate IAM configurations into a right-sized Terraform plan. It identifies unused users, roles, groups, policies and policy attachments and replaces them with a Least Privileges Terraform code modelled to manage AWS IAM.

AirIAM was created to promote immutable and version-controlled IAM management to replace today’s manual and error prone methods.

Why use it: IAM scan tool to detect unused resources (based on Amazon Access Advisor APIs) and the creation of terraform templates of your IAM resources.

Is it popular?: On github: Watch 15; Fork 68; Stars 677

Recently updated? No. Last commit the Aug 2, 2022. In total, 426 commits

URL: https://github.com/bridgecrewio/AirIAM

First time I used this tool the terraform template generation functionality works fine, now now I receive an error and I was not able to use it. However, I think that this tool is useful to find this iam unused resources

Summary: Which tool should I use to generate IaC?

CDK, CloudFormation, Troposphere, Pulumi: former2
Terraform IaC: former2 or terracognita
Specific IAM resources: former2, terracognita or AirIAM

Analyze IaC code: static analysis

Static Code Analysis commonly refers to the running of Static Code Analysis tools that attempt to highlight possible vulnerabilities within ‘static’ (non-running) source code. Information extracted from OWASP website

checkov: Scans cloud infrastructure provisioned to detects security and compliance misconfigurations using graph-based scanning.
KICKS: Find security vulnerabilities, compliance issues, and infrastructure misconfigurations early in the development cycle of your infrastructure-as-code
terrascan: static code analyzer for Infrastructure as Code to scan infrastructure as code for misconfigurations, detect security vulnerabilities and compliance violations
tfsec: static analysis code for Terraform
cfn-nag: looks for patterns in CloudFormation templates that may indicate insecure infrastructure.

All the tools in this section allow you to create custom rules

checkov

Checkov is a static code analysis tool for infrastructure as code (IaC) and also a software composition analysis (SCA) tool for images and open source packages.

It scans cloud infrastructure provisioned to detects security and compliance misconfigurations using graph-based scanning.

Checkov scans these IaC file types:

Terraform (for AWS, GCP, Azure and OCI)
CloudFormation (including AWS SAM)
Azure Resource Manager (ARM)
Serverless framework
Helm charts
Kubernetes
Docker

Some Features:

Over 1000 built-in policies cover security and compliance best practices for AWS, Azure and Google Cloud.
- In AWS there are 177 controls in the framework CKV_AWS. Checkov scans for compliance with common industry standards such as
  - the Center for Internet Security (CIS)
  - and Amazon Web Services (AWS) Foundations Benchmark.
Detects AWS credentials in EC2 user-data, Lambda environment variables and Terraform providers.
Identifies secrets using regular expressions, keywords, and entropy based detection.
Plugins for popular IDEs available (JetBrains, VSCode and Vim). However, activating the extension requires submission of one-time Bridgecrew API Token that can be obtained by creating a new Bridgecrew platform account.

You can create custom policies here more information using Python opr YAML.

Why use it: Includes 177 AWS controls including the Center for Internet Security (CIS) and Amazon Web Services (AWS) Foundations Benchmark, and is able to detect secrets and AWS credentials in the code.
You also can visualize checkov scan output using Bridgecrew platform (free to use with the Community plan - up to 50 resources and small projects - here)

Is it popular?: On github: Watch: 55; Fork: 799; Stars: 5.2k

Recently updated?: Today (in the time I am writing this post)

URL: https://github.com/bridgecrewio/checkov

More information: https://www.checkov.io/1.Welcome/Quick%20Start.html

KICKS

Find security vulnerabilities, compliance issues, and infrastructure misconfigurations early in the development cycle of your infrastructure-as-code.

KICS stands for Keeping Infrastructure as Code Secure and support the following IaC solutions: Terraform, AWS CloudFormation, AWS SAM, AWS CDK, Kubernetes, Docker, Ansible, Helm, Google Deployment Manager, Microsoft ARM, Microsoft Azure Blueprints, OpenAPI 2.0 and 3.0, Pulumi, Crossplane, Knative and Serverless Framework.

KICS is 100% open source is written in Golang using Open Policy Agent (OPA) and it is possible create custom queries to create custom rules (using REGO language).

Why use it: Evaluate IaC to detect vulnerabilities

Is it popular?: On github: Watch: 22; Fork: 224; Stars: 1.4k

URL: https://github.com/Checkmarx/kics

More information: https://www.kics.io/

terrascan

Terrascan is a static code analyzer for Infrastructure as Code. Terrascan allows you to:

Seamlessly scan infrastructure as code for misconfigurations.
Monitor provisioned cloud infrastructure for configuration changes that introduce posture drift, and enables reverting to a secure posture.
Detect security vulnerabilities and compliance violations.
Mitigate risks before provisioning cloud native infrastructure.
Offers flexibility to run locally or integrate with your CI\CD.

Key features

500+ Policies for security best practices
Scanning of Terraform (HCL2), AWS CloudFormation Templates (CFT), Azure Resource Manager (ARM), Kubernetes (JSON/YAML), Helm v3, and Kustomize and Dockerfiles
Integrates with docker image vulnerability scanning for AWS, Azure, GCP, Harbor container registries.

Terrascan policies are written using the Rego policy language, and you can create your own policies

Why use it: More than 300 AWS rules and support CloudFormation and Terraform.

Is it popular?: On github: Watch: 70; Fork: 459; Stars: 3.9k

Recently updated?: Today (in the time I am writing this post)

URL: https://github.com/tenable/terrascan

More information: https://runterrascan.io/docs/

tfsec

The tfsec open source tool provides security analysis of Terraform code and detects potential security issues based on AWS best practices.

The tool, contain checks for more than 30 AWS resources, and can be found here: https://aquasecurity.github.io/tfsec/v1.28.1/checks/aws/

tfsec has the capability to apply user-defined Rego policies. This is a useful feature if your organisation needs to implement custom security policies on top of avoid other misconfigurations and enforcing best practice guidelines. More information here.

Some Features:

Checks for misconfigurations across all major (and some minor) cloud providers
Hundreds of built-in rules
Applies (and embellishes) user-defined Rego policies
Very fast, capable of quickly scanning huge repositories
Plugins for popular IDEs available (JetBrains, VSCode and Vim)

Why use it: static analysis code for terraform with checks in more than 30 AWS resources.
Integration with Github Security alerts:

Is it popular?: On github: Watch: 69; Fork: 485; Stars: 5.6k

Recently updated?: Yes, last month (1318 commits)

URL: https://github.com/aquasecurity/tfsec

More information: https://aquasecurity.github.io/tfsec/v1.28.1/

cfn-nag

The cfn-nag tool looks for patterns in CloudFormation templates that may indicate insecure infrastructure. It will look for:

IAM rules that are too permissive (wildcards)
Security group rules that are too permissive (wildcards)
Access logs that aren’t enabled
Encryption that isn’t enabled
Password literals

The tool contains more than 150 AWS controls.

Why use it: Analyze CloudFormation templates to detect insecure infrastructure

Is it popular?: On github: Watch 35; Fork 199; Stars 1.1k

Recently updated? No. Last commit the Jun 7, 2022. In total, 664 commits

URL: https://github.com/stelligent/cfn_nag

Summary: Which tool should I use to perform a static analysis?

CloudFormation code: checkov, KICKS terrascan, cfn-nag
Terraform code: checkov, KICKS, terrascan or tfsec
Integrate with IDE: checkov or tfsec
Allow create custom rules: checkov, KICKS, terrascan, tfsec

In my blog-backend-infrastructure code available here I had the following errors using these tools:

KICKS: 36 (3 high, 20 medium, 13 low)
terrascan: 9 (1 high, 7 medium, 1 low)
tfsec: N/A (only Terraform code)
cfn-nag: 33

Analyze IaC code: Policy as Code

Policy-as-code is the use of code to define and manage rules and conditions to assure that your Infrastructure will be compliance with that. This is a way to apply preventative governance and compliance (shift left), validating Infrastructure-as-code (IaC) against your organizational best practices for security and compliance.

CloudFormation Guard: policy-as-code evaluation tool for general purpose
OPA: general-purpose policy engine that enables unified, context-aware policy enforcement across the entire stack
Regula: evaluates infrastructure as code files for potential security and compliance violations prior to deployment.

CloudFormation Guard

AWS CloudFormation Guard is an open-source general-purpose policy-as-code evaluation tool. It provides developers with a simple-to-use, yet powerful and expressive domain-specific language (DSL) to define policies and enables developers to validate JSON or YAML formatted structured data with those policies.

Supported: CloudFormation Templates, CloudFormation ChangeSets, Terraform JSON configuration files, Kubernetes configurations, and more.

Why use it: Simple to use, compatible with CloudFormation, Terraform and kubernetes configurations.

Is it popular?: On github: Watch: 30; Fork: 145; Stars: 1.1k

Recently updated?: Last week (in the time I am writing this post)

URL: https://github.com/aws-cloudformation/cloudformation-guard#installation

More information: https://docs.aws.amazon.com/cfn-guard/latest/ug/what-is-guard.html

This is an official aws-cloudformation tool (yet open-source), but you have to create your own rules. I didn’t find the “security best practices” included in the control AWS best practices of Security Hub as rules. However, you have here some examples: https://github.com/aws-cloudformation/cloudformation-guard/tree/main/guard-examples

Open Policy Agent (OPA)

Open Policy Agent (OPA) is an open source, general-purpose policy engine that enables unified, context-aware policy enforcement across the entire stack.

The Open Policy Agent is an open source, general-purpose policy engine that unifies policy enforcement across the stack. OPA provides a high-level declarative language (REGO) that lets you specify policy as code and simple APIs to offload policy decision-making from your software. You can use OPA to enforce policies in microservices, Kubernetes, CI/CD pipelines, API gateways, and more.

OPA ecosystem is hugh here, but in AWS we can use it to analyze our JSON/YAML file in CloudFormation or our Terraform template by creating custom rules for these languages.

OPA is a project of the Cloud Native Computing Foundation (CNCF) landscape.

Why use it: unify policy enforcement across the stack. This example is using Terraform. You have to convert your output Terraform to JSON file and then analyze it with OPA rules:

Is it popular?: On github: Watch: 119; Fork: 1.1k; Stars: 7.6k

Recently updated?: 4 days ago (in the time I am writing this post)

URL: https://github.com/open-policy-agent/opa

More information: https://www.openpolicyagent.org/

You have to create your own OPA rules with REGO language (easy to learn). You have an online playground to test your REGO policies here

Regula

Regula is a tool that evaluates infrastructure as code files for potential AWS, Azure, Google Cloud, and Kubernetes security and compliance violations prior to deployment. Based on Open Policy Agent (OPA) and written in REGO

Regula supports the following file types:

CloudFormation JSON/YAML templates
Terraform source code (.tf or .tf.json format)
Terraform JSON plans
Kubernetes YAML manifests
Azure Resource Manager (ARM) JSON templates (in preview)

This is the list of rules applied (in AWS there are rules for Terraform and CloudFormation): https://regula.dev/rules.html.

You must to know that at this time CloudFormation support 23 controls, and Terraform 114.

Here you can find more information about Writing custom rules.

Why use it: Evaluate IaC to detect vulnerabilities

Is it popular?: On github: Watch: 54; Fork: 776; Stars: 5.1k

URL: https://github.com/fugue/regula

More information: https://regula.dev/

Summary: Which tool should I use to create my custom Policy as Code?

AWS solution: CloudFormation Guard, OPA or regula
Cross-provider solution: OPA or regula (both using REGO language)

Or you can use any of the tools in the static analysis category, because all of them allow you to create custom rules.

Detect credentials in code

These tools can be apply to any git code:

General for git repository:
- git-secrets: Prevents you from committing passwords and other sensitive information to a git repository.
- gitleaks: tool for detecting and preventing hardcoded secrets like passwords, api keys, and tokens in git repos.
Specific of IaC code:
- checkov: static code analysis tool for infrastructure as code (included secrets)
  - This tool is already explained a few lines before so you can get the information from there

git-secrets

Prevents you from committing passwords and other sensitive information to a git repository.

Why use it: Find api keys, passwords, AWS keys in the code.

Is it popular?: On github: Watch: 198; Fork: 1.1k; Stars: 10.8k

Recently updated?: Today (in the time I am writing this post). Commit in total 110

URL: https://github.com/awslabs/git-secrets

gitleaks

Gitleaks is a SAST tool for detecting and preventing hardcoded secrets like passwords, api keys, and tokens in git repos. Gitleaks is an easy-to-use, all-in-one solution for detecting secrets, past or present, in your code.

Why use it: Analyze the code of your IaC infrastructure to detect security and compliance misconfigurations.

Is it popular?: On github: Watch: 142; Fork: 1.1k; Stars: 11.5k

Recently updated?: Today (in the time I am writing this post). Commit in total 896

URL: https://github.com/zricethezav/gitleaks

Others

Compliance
- cloud custodian: rules engine to define policies to enable a well managed, secure and cost optimized cloud infrastructure
Cost of Terraform templates:
- Infracost: estimate cost for Terraform before to deploy

Cloud custodian

Cloud Custodian is a rules engine to define policies to enable a well managed, secure and cost optimized cloud infrastructure in yaml format.

Custodian can be used to manage AWS, Azure, and GCP environments by ensuring real time compliance to security policies (like encryption and access requirements), tag policies, and cost management via garbage collection of unused resources and off-hours resource management.

Custodian policies are written in simple YAML configuration files that enable users to specify policies on a resource type (EC2, ASG, Redshift, CosmosDB, PubSub Topic) and are constructed from a vocabulary of filters and actions.

You can apply to AWS to here

Why use it: manage your rules with yaml policies

Is it popular?: On github: Watch 1.3k; Starts 4.6k

Recently updated?: Yes. The last commit was 3 days ago (4054 commits in total)

URL: https://github.com/cloud-custodian/cloud-custodian/

Many examples available in https://www.cloudcustodian.io/docs/aws/examples/index.html

Infracost

Infracost shows cloud cost estimates for Terraform. It lets DevOps, SRE and engineers see a cost breakdown and understand costs before making changes, either in the terminal or pull requests.

Infracost also has many CI/CD integrations so you can easily post cost estimates in pull requests. This provides your team with a safety net as people can discuss costs as part of the workflow.

The CDK for Terraform is also supported as it can generate Terraform

Why use it: If you are using Terraform, this tool will estimate the cost before to deploy. Do you want to try? (images taken from github) Output of infracost breakdown $infracost$
infracost diff shows diff of monthly costs between current and planned state $infracost-2$
Post cost estimates in pull requests
$infracost-cicd-cost$

Is it popular?: On github: Watch: 63; Fork: 418; Stars: 8.5k

Recently updated?: Yes. Last commit yesterday (in the time I am writing this post). In total 2196 commits

URL: https://github.com/infracost/infracost

More information: https://www.infracost.io/docs/

Next Steps

My arsenal of security tools: https://github.com/toniblyx/my-arsenal-of-aws-security-tools
Improve your handsome security skills: https://github.com/RhinoSecurityLabs/cloudgoat
AWS Security primer: https://cloudonaut.io/aws-security-primer/
AWS Lambda Power tuning (optimize Lambda): https://github.com/alexcasalboni/aws-lambda-power-tuning