Introduction
This is the first part of a series of articles about open source software and AWS, in which I will share those tools that I have tested and that I think you may find useful.
The complete series contains the following:
- Getting started: main open source tools and extend AWS CLI (this article)
- Analyze your AWS environment: focus on the inventory, analysis and security assessment of the AWS environment - here
- How to help you with your code: to generate it (IaC), validate it (policy as code and compliance) and analyze it (static analysis/credentials) - here
What is open-source?
The Open Source Initiative
defines here the following criteria for considering software as open source:
- Free Redistribution
- Source code must be accessible
- The license must allow modifications and derived works
- Integrity of The Author’s Source Code
- No Discrimination Against Persons or Groups
- No Discrimination Against Fields of Endeavour
- Distribution of License
- License Must Not Be Specific to a Product
- License Must Not Restrict Other Software
- License Must Be Technology-Neutral
Open source software is code designed to be accessible to the public:
anyone can view, modify and distribute the code
as they wish.
By the way, both terms are correct: open source and open-source. I could use them interchangeably in this article.
What is the relationship with AWS?
I’m sure you already know that there are many open-source projects related to AWS, too many. This is because AWS is very popular, but also because AWS is very committed to the open source community.
AWS claims that open source is good for everyone and regularly develops open source software and contributes to thousands of open-source communities on GitHub, Apache, and the Linux Foundation. More information can be found here
In this post, I will try to show you some open source projects, since you probably don’t know all of them and you can surely start using some of them.
So, let’s start sharing public code!
Main AWS open-source tools
Let’s start with the most popular AWS open source projects.
You are probably already using some of them and did not realize they are open source. However, I am not going to explain them or give more information, I will just name them here:
- AWS CLI: Amazon Web Services Command Line Interface
- CDK (Cloud Development Kit): Define cloud infrastructure using familiar programming languages
- SAM (Serverless Application Model): Framework for building serverless applications
- AWS Amplify: framework and tools for developing mobile and web applications
- EKS distro: Certified Kubernetes distribution based on and used by Amazon Elastic Kubernetes Service (EKS) to create reliable and secure Kubernetes clusters -https://github.com/aws/eks-distro
- Karpenter: node provisioning project built for Kubernetes
- OpenSearch: A community-driven, search and analytics suite derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2
- Bottlerocket: Linux-based operating system meant for hosting containers
- Firecracker: Virtual machine monitor (VMM) to create and manage microVMs. Firecracker powers the AWS Lambda service.
Extend AWS CLI
My first idea was named this section CLI tools
but all the open source tools listed here are CLI (Command Line Interface) tools, so this section is for the tools that you can use to improve/extend/replace your AWS CLI tool.
- Security
- aws-vault: tool to
securely store and access AWS credentials
in a development environment
- aws-vault: tool to
- Extend AWS CLI
- aws-shell: interactive
productivity booster
for the AWS CLI - awsls: a
list command
for AWS resources - steampipe: Use
SQL to query cloud infrastructure
, SaaS, code, logs, and more - ohmyzsh with the aws plugin: provides completion support for AWS CLI and a few utilities to
manage AWS profiles and display them in the prompt
- aws-shell: interactive
- Logs
- awslogs: a simple command line tool for
querying
groups, streams and eventsfrom Amazon CloudWatch logs
- awslogs: a simple command line tool for
aws-vault
aws-vault
securely store and access AWS credentials in a development environment.
AWS Vault stores IAM credentials in your operating system’s secure keystore and then generates temporary credentials from those to expose to your shell and applications. It’s designed to be complementary to the AWS CLI tools, and is aware of your profiles and configuration in ~/.aws/config.
Is it popular? Yes, GitHub statistics: Watch 119; Fork 725; Stars 7.1k
Recently updated? Yes, 1096 commits, last 2 weeks ago
URL: https://github.com/99designs/aws-vault
In my opinion, this tool is a must-have for securing your credentials.
aws-shell
The interactive productivity booster for the AWS CLI
Is it popular? Yes, GitHub statistics: Watch 230; Fork 755; Stars 6.8k
Recently updated? No, the project seems abandoned (last commit Oct 7, 2020). Total commits 235
URL: https://github.com/awslabs/aws-shell
The tool is not working with AWS CLI v2 (here is the official
Issue
in github), and the project seems to have been abandoned (last commit July 10, 2020).However, if you use v1, the tool is worth it because it contains all the core AWS services.
awsls
A list command for AWS resources. More than 100 AWS resource supported. The goal is to code-generate a list function for every AWS resource that is covered by the Terraform AWS Provider (currently over 500)
Is it popular? Yes, GitHub statistics: Watch 10; Fork 51; Stars 763
Recently updated? No, one year from the last update (Feb 13, 2022), with 91 commits in total.
URL: https://github.com/jckuester/awsls
Although it has not been updated recently, it is worth using for its ability to search multiple accounts and filter using the GREP command.
steampipe
Steampipe is the universal interface to APIs. Use SQL to query cloud infrastructure, SaaS, code, logs, and more.
Is it popular?: On github: Watch 32; Fork 171; Stars 4.6k
Recently updated? Yes, last commit 2 days ago. Total commits 2007
URL: https://github.com/turbot/steampipe
More information (querying aws resources):
- https://steampipe.io/docs/query/overview
- https://aws.amazon.com/blogs/opensource/querying-aws-at-scale-across-apis-regions-and-accounts/
I like this tool! If I want to get specific information in AWS using a CLI tool this is my first choice to do so, I recommend you to try it!
awslogs
awslogs is a simple command line tool for querying groups, streams and events from Amazon CloudWatch logs.
Is it popular?: On github: Watch 61; Fork 326; Stars 4.5k
Recently updated? No, the project seems to have been abandoned. The last commit was July 10, 2020. Total commits 326
URL: https://github.com/jorgebastida/awslogs
The tool is not updated but if you want to query the CloudWatch logs, this is your tool!
ohmyzsh with the aws plugin
If you are using ohmyzsh (framework for managing your zsh configuration), you can add this aws
plugin to extend your CLI.
This plugin provides completion support for AWS CLI and a few utilities to manage AWS profiles and display them in the prompt.
Why use it: Useful if you are using different profiles and ohmyzsh.
Is it popular?: ohmyzsh
is a very popular framework for zsh
(155k stars), and the AWS plugin
is part of this framework.
Recently updated? Yes. The aws plugin was updated 2 weeks ago
URL: https://github.com/ohmyzsh/ohmyzsh/tree/master/plugins/aws
Next steps
As I said in the introduction, there are 2 more articles in this series of open source tools: