The architecture of the infrastructure-as-code (IaC) tooling you use will determine the level to which your IaC definitions are exposed to bit rot.

This is a maxim I have arrived at after working with multiple IaC tool sets, both professionally and personally, over the last few years. In this blog post, I will explain how I arrived at this maxim by describing three architectural patterns for IaC tools, each with differing levels of risk for bit rot.

The AWS CLI is a tool set that lets you manage your AWS resources. The CLI comes in two versions which, at the time of this writing, are developed concurrently: version 1 and version 2.

Internally, the AWS CLIv1 and v2 are quite different. Version 2 pulls in AWS libraries--libraries which are used across the AWS SDK ecosystem--rather than reinventing the wheel when it comes to common tasks, such as talking to Amazon S3. Running AWS CLIv2 on your operating system of choice requires building and installing these common AWS libraries.

I contributed changes to s2n-tls, aws-lc, the Python runtime, some build tooling, and various other libraries. As a result, the AWS CLIv2 now builds and runs on OpenBSD.

To make installation simple, I've created an OpenBSD port for CLIv2 and its dependencies. Instructions for installing the port are below.

An introduction to Amazon EC2 credentials

When you assign an Identity and Access Management (IAM) role to an Amazon Elastic Compute Cloud (EC2) instance, the short-term credentials for the role are made available via a web service known as the Instance Metadata Service (IMDS). The IMDS provides an HTTP endpoint for retrieving instance metadata such as the instance IP address, AWS Region the instance is running in, the Amazon Machine Image used to launch the instance, and the access key, secret access key, and session token associated with the instance's IAM role. The AWS documentation describes how to retrieve instance role credentials from IMDS. If you've seen or used the http://169.254.169.254 or http://fd00:ec2::254 endpoints, then you've seen/used IMDS.

Retrieval of instance role credentials from IMDS is the mechanism by which the AWS CLI and SDKs learn the credentials belonging to the instance's IAM role without you having to configure anything on the instance. Quoting the IAM documentation:

The AWS SDKs, AWS CLI, and Tools for Windows PowerShell automatically get the credentials from the EC2 Instance Metadata Service (IMDS) and use them.

This is great! It means you can start using the AWS CLI, SDKs, or Tools for Windows PowerShell on an EC2 instance without having to configure any credentials.

However, like most nice things, IMDS can be exploited and used in unintended ways. This blog post will explain how EC2 credentials can be retrieved from IMDS, removed from the EC2 instance, and used outside of EC2. This post will also explain some ways to mitigate this activity.

Passwords suffer from an inherent risk: whoever possess the password inherits the privileges granted by that password. If the possessor is the intended person, then all is good. Otherwise, all is not so good because it means an unintended person has access to the system the password is guarding.

A colleague of mine recently quiped, "'The perimeter' in AWS is actually defined by Identity and Access Management (IAM)." After some reflection, I think my colleague is spot on.

I've written before about how I use MediaWiki for taking notes and as one of my study tools. This has worked well for many years. But a problem started to develop: while I wrote my technical notes in MediaWiki, I wrote my day-to-day notes (books I want to read, notes from podcasts I listen to, and even my weekly planner) in Notion. This meant I had to use different apps for reading/writing in each tool, remember two different markup languages, and couldn't (cleanly) link pieces of content between the two. The final straw was realizing how much more effort I had to expend to maintain my MediaWiki instance; I just didn't have the time or will to keep up with new releases not to mention maintain the server itself.

For these reasons, I decided to move all of my MediaWiki content to Notion and unify all of my notes. But this revealed a new problem: there was no tooling to automate this. So I created my own. Here's how it works.

The AWS Cloud Development Kit (CDK) is an "open source software development framework to define your cloud application resources using familiar programming languages". When CDK launched in 2019, I remember reading the announcement and thinking, "Ok, AWS wants their own Terraform-esque tool. No surprise given how popular Terraform is." Months later, my friend and colleague Matt M. was telling me how he was using CDK in a project he was working on and how crazy cool it was.

I finally decided to give CDK a go for one of my projects. Here is what I discovered.

AWS CloudFormation is a service that lets you create a collection of related Amazon Web Services and third-party resources and provision them in an orderly and predictable fashion. A typical access control pattern is to delegate permissions for users to interact with CloudFormation and remove or limit their permissions to provision resources directly. You can grant the AWS CloudFormation service permission to create resources by creating a role that the user passes to CloudFormation when a stack or stack set is created. This can be used to ensure that only pre-authorized services and resources are provisioned in your AWS account. In this post, I show you how to conform to the principle of least privilege while still allowing users to use CloudFormation to create the resources they need.

AWS Serverless Application Model (SAM) is a framework for building serverless applications on AWS. One of the components of SAM is a template specification. SAM templates would look and feel familiar to anyone who has used AWS CloudFormation to define their infrastructure as code, however they are not completely interchangeable. There are multiple reasons why you might want to convert from SAM to native CloudFormation:

• You want to deploy the app using CloudFormation StackSets. SAM uses the AWS::Serverless transform in its templates and transforms are not supported by stack sets.
• You want to deploy the app as part of an AWS Landing Zone (ALZ) account baseline. ALZ uses stack sets as the mechanism to deploy baseline resources and so suffers from the same constraint as the point above.
• Your operating system of choice isn't documented in the SAM installation instructions and you're uncertain how to install from source or doubtful it will work at all (I'm looking at fellow OpenBSD and FreeBSD users here).

This post will show you how to take an existing SAM application and convert it to a CloudFormation template (CFT). As a CFT, the challenges listed above can be avoided.

In a recent customer engagement, Quantiphi, Inc., a member of the Amazon Web Services Partner Network, built a solution capable of pre-processing tens of millions of PDF documents before sending them for inference by a machine learning (ML) model. While the customer's use case--and hence the ML model--was very specific to their needs, the pipeline that does the pre-processing of documents is reusable for a wide array of document processing workloads. This post will walk you through the pre-processing pipeline architecture.