I’ve just recently started using Hashicorp Packer to automate the provisioning of our master AMIs, the Amazon Machine Images our team uses to build our production and staging environments.

This provision process (or “baking”, as some in the DevOps community prefer to call it) only comprises of a few steps:

  1. Installing core system packages
  2. Configuring users, groups, and file permissions
  3. Injecting our server environment repository into the PATHs and shell profile of all the users on the system.

Packer is great because we can represent all of these steps cleanly in simple scripts, and Packer takes care of all the heavy lifting around spawning a build server, executing our instructions, and building an AMI in EC2.

However, there is one downside: Packer only accepts JSON files as configuration. JSON is a format that is intended to be easily readable by both machines and humans, and is great for the former, but painful as the latter. It also lacks useful features like comments and aliasing. YAML, on the other hand, is much cleaner, supports both comments and aliasing. As a result, I usually turn to YAML to represent my configuration.

Fortunately, Packer accepts configuration via stdin, so we can quite easily define our configuration via YAML, transform it on the fly to the format that Packer expects, and abstract this all away behind a clean set of commands. Lets get started.

Transformation Script

First, we need a script to transform our YAML configuration file into JSON. I chose to use Ruby, as it has both JSON and YAML packages in the standard library.

#!/usr/bin/env ruby

require 'yaml'
require 'json'

configuration = YAML.load($stdin)

Save this file as transform-configuration.rb, and grant it executable permissions:

chmod +x transform-configuration.rb

At this point, we can read our configuration, pass it to this script, and forward it on to Packer, like so:

cat packer-config.yml | ./transform-configuration.rb | packer build -

… but we should make this a little nicer!


Using make, we can hide this behind a convenient command.

This is the Makefile that I’ve been using:


	cat $(INPUT_FILE) | $(TRANSFORMER) | $(PACKER) validate -

	cat $(INPUT_FILE) | $(TRANSFORMER) | $(PACKER) build -

	cat $(INPUT_FILE) | $(TRANSFORMER) | $(PACKER) build -debug -

Now, we can easily run make validate, make debug, and make build, and use Packer with YAML! This is a simple trick, but has greatly improved my happiness with Packer. I’ve been really enjoying using this tool, and it has delivered incredible value to our team, so I’m looking forward to using it more and more, and sharing my thoughts!