The AWS CLI is a tool set that lets you manage your AWS resources. The CLI comes in two versions which, at the time of this writing, are developed concurrently: version 1 and version 2.

Internally, the AWS CLIv1 and v2 are quite different. Version 2 pulls in AWS libraries--libraries which are used across the AWS SDK ecosystem--rather than reinventing the wheel when it comes to common tasks, such as talking to Amazon S3. Running AWS CLIv2 on your operating system of choice requires building and installing these common AWS libraries.

I contributed changes to s2n-tls, aws-lc, the Python runtime, some build tooling, and various other libraries. As a result, the AWS CLIv2 now builds and runs on OpenBSD.

To make installation simple, I've created an OpenBSD port for CLIv2 and its dependencies. Instructions for installing the port are below.

An introduction to Amazon EC2 credentials

When you assign an Identity and Access Management (IAM) role to an Amazon Elastic Compute Cloud (EC2) instance, the short-term credentials for the role are made available via a web service known as the Instance Metadata Service (IMDS). The IMDS provides an HTTP endpoint for retrieving instance metadata such as the instance IP address, AWS Region the instance is running in, the Amazon Machine Image used to launch the instance, and the access key, secret access key, and session token associated with the instance's IAM role. The AWS documentation describes how to retrieve instance role credentials from IMDS. If you've seen or used the http://169.254.169.254 or http://fd00:ec2::254 endpoints, then you've seen/used IMDS.

Retrieval of instance role credentials from IMDS is the mechanism by which the AWS CLI and SDKs learn the credentials belonging to the instance's IAM role without you having to configure anything on the instance. Quoting the IAM documentation:

The AWS SDKs, AWS CLI, and Tools for Windows PowerShell automatically get the credentials from the EC2 Instance Metadata Service (IMDS) and use them.

This is great! It means you can start using the AWS CLI, SDKs, or Tools for Windows PowerShell on an EC2 instance without having to configure any credentials.

However, like most nice things, IMDS can be exploited and used in unintended ways. This blog post will explain how EC2 credentials can be retrieved from IMDS, removed from the EC2 instance, and used outside of EC2. This post will also explain some ways to mitigate this activity.

Passwords suffer from an inherent risk: whoever possess the password inherits the privileges granted by that password. If the possessor is the intended person, then all is good. Otherwise, all is not so good because it means an unintended person has access to the system the password is guarding.

A colleague of mine recently quiped, "'The perimeter' in AWS is actually defined by Identity and Access Management (IAM)." After some reflection, I think my colleague is spot on.

The AWS Cloud Development Kit (CDK) is an "open source software development framework to define your cloud application resources using familiar programming languages". When CDK launched in 2019, I remember reading the announcement and thinking, "Ok, AWS wants their own Terraform-esque tool. No surprise given how popular Terraform is." Months later, my friend and colleague Matt M. was telling me how he was using CDK in a project he was working on and how crazy cool it was.

I finally decided to give CDK a go for one of my projects. Here is what I discovered.

AWS CloudFormation is a service that lets you create a collection of related Amazon Web Services and third-party resources and provision them in an orderly and predictable fashion. A typical access control pattern is to delegate permissions for users to interact with CloudFormation and remove or limit their permissions to provision resources directly. You can grant the AWS CloudFormation service permission to create resources by creating a role that the user passes to CloudFormation when a stack or stack set is created. This can be used to ensure that only pre-authorized services and resources are provisioned in your AWS account. In this post, I show you how to conform to the principle of least privilege while still allowing users to use CloudFormation to create the resources they need.

AWS Serverless Application Model (SAM) is a framework for building serverless applications on AWS. One of the components of SAM is a template specification. SAM templates would look and feel familiar to anyone who has used AWS CloudFormation to define their infrastructure as code, however they are not completely interchangeable. There are multiple reasons why you might want to convert from SAM to native CloudFormation:

• You want to deploy the app using CloudFormation StackSets. SAM uses the AWS::Serverless transform in its templates and transforms are not supported by stack sets.
• You want to deploy the app as part of an AWS Landing Zone (ALZ) account baseline. ALZ uses stack sets as the mechanism to deploy baseline resources and so suffers from the same constraint as the point above.
• Your operating system of choice isn't documented in the SAM installation instructions and you're uncertain how to install from source or doubtful it will work at all (I'm looking at fellow OpenBSD and FreeBSD users here).

This post will show you how to take an existing SAM application and convert it to a CloudFormation template (CFT). As a CFT, the challenges listed above can be avoided.

In a recent customer engagement, Quantiphi, Inc., a member of the Amazon Web Services Partner Network, built a solution capable of pre-processing tens of millions of PDF documents before sending them for inference by a machine learning (ML) model. While the customer's use case--and hence the ML model--was very specific to their needs, the pipeline that does the pre-processing of documents is reusable for a wide array of document processing workloads. This post will walk you through the pre-processing pipeline architecture.

I was recently playing around with the Traffic Mirroring feature in AWS. As a network geek, this is right up my alley because as some colleagues and I used to say, "the wire never lies!". Being able to pick packets off the wire for detailed inspection has saved the day many a time. Until Traffic Mirroring came along, it wasn't possible to do that in an Amazon VPC. Below are my notes and considerations for using this feature.

I recently used AWS DataSync as part of a lab I was building. These are my notes for using DataSync to replicate an Amazon Elastic File System (EFS) share from one region to another.

AWS DataSync is a managed service that enables replication of data between AWS services and from on-prem to AWS. It automates the scheduling of transfer activities, validates copied data, and uses a purpose-built network protocol and multi-threaded architecture to achieve very high efficiency on the wire.

The use case I needed to tackle was replicating an Amazon EFS share in one region to an EFS share in a different region (a one-way replication). (DataSync can also connect to Amazon S3 and Amazon FSx for Windows File Server)