Why You Shouldn't Use ^Parameters with CloudFormation Templates

February 7, 2019

^Run-Time (I admit that the title is intended to be clickbait)

CloudFormation has become a popular Infrastructure as Code (IaC) tool for organizations that use AWS exclusively as their cloud provider (whether it should be or not is another whole discussion). The tool features the ability to add parameters to your configuration scripts (written in either YAML or JSON). However, I will argue in this post that you should not use parameters in your (top-level) scripts by enumerating Adam Jacob’s (creator of Chef) 10 principles of IaC as detailed in his chapter in Web Operations and illustrating how the use of parameters either violates or does not add any further value for each of them.

Disclaimers

  • Parameters SHOULD be used to keep secrets and credentials from being committed or pushed to version control systems.

  • CloudFormation, along with other tools such as HashiCorp’s Terraform, allow you to reference existing templates in other templates. Parameters are somewhat useful here, but not in the top-level template. That is to say that it isn’t horrible to have a parameterized template for a general EC2 instance that another template (that takes no parameters) utilizes to provision a cluster of EC2 instances, as long as both are committed to version control. It still wouldn’t be the worst thing to just copy, insert, and modify the existing template into the new template though.

  • If parameters are used to generate new templates that are then committed to a version control system, then they are advantageous and a productive abstraction.

Summary of principles provided by Stephen Nelson-Smith

Modularity

Our services should be small and simple—think at the level of the simplest freestanding, useful component.

Parameters don’t explicitly violate this principle, but they do not philosophically align with the idea of being freestanding. Parameters allow a template to be more general, thus meaning that there is no one file that fully describes the infrastructure configuration for a component. They are reliant on further information being provided at runtime, information that is not necessarily recorded or committed to version control.

Cooperation

Our design should discourage overlap of services, and should encourage other people and services to use our service in a way which fosters continuous improvement of our design and implementation.

A first thought may be that defining similar services from the same script enhances their ability to work together. However, duplicating scripts and making minor changes (or adding to the end of an existing script) allows you to work from the same starting point, while still having a single code-defined source of truth for each service.

Composability

Our services should be like building blocks—we should be able to build complete, complex systems by integrating them.

Parameters are useful for defining building blocks. However, if you don’t bring those building block together in code, then you end up with a library of infrastructure components that have been integrated in various ways, but may be unable to be reproduced in the same manner.

Extensibility

Our services should be easy to modify, enhance, and improve in response to new demands.

Services are easier to modify when we have a version controlled configuration file for them. We can see the current state of the service because it mirrors the most recently committed version of the configuration file. The diffs in our infrastructure should always be reflected in the change or addition of a IaC file.

Flexibility

We should build our services using tools that provide unlimited power to ensure we have the (theoretical) ability to solve even the most complicated of problems.

Problem solving requires three things: knowing where you are, knowing where you want to go, and figuring out how to get there. Configuration files that show our current infrastructure represent the first part. Knowing where we want to go may come from seeing things in our community or doing organic research. Figuring out how to get there is an iterative process. Codified infrastructure allows us to make small, incremental changes, view results, and roll-back if necessary. Staying away from runtime parameters provides the opportunity to see how we got from one place to another.

Repeatability

Our services should produce the same results, in the same way, with the same inputs, every time.

This is a principle that parameters have the potential to severely violate. If you are simply defining the names of various resources, it is unlikely for this to happen. However, if you are modifying ports, Availability Zones, Regions, instance types and sizes, etc., your scripts are very likely to produce different outputs (in fact, this is the whole purpose of parameters!).

Declaration

We should specify our services in terms of what we want it to do, not how we want to do it.

Verbose infrastructure as code does not mean that we are moving closer to how and farther from what. In fact, it takes smaller components and integrates them together so that our defining of what we want to happen inherently shows us how it is going to happen. We say we want a cluster with 3 nodes. Our code says for AWS to spin up 3 EC2 instances. We get into trouble when we have the EC2 instances defined, but we never define the cluster as code. We only say “what we want” at a high level as a parameter in the management console or at the command line, and down the line we won’t know what we have.

Abstraction

We should not worry about the details of the implementation, and think at the level of the component and its function.

I imagine this will be the most contentious point I make in this post, as parameters certainly seem to increase the abstraction of infrastructure provisioning. However, I believe there is a limit to the abstraction that Mr. Jacobs is speaking of. We are controlling computers halfway around the world with configuration templates: I think we are still achieving abstraction even without parameters. And as mentioned in the disclaimers, I am not opposed to achieving abstraction by having top-level templates reference other templates that include parameters.

Idempotence

Our services should only be configured when required—action should only be taken once.

Eliminating parameters naturally encourages more thought than uploading someone elses template file, filling in values, and launching a new CloudFormation Stack does. Knowing what a template specifies (because you wrote it yourself), and committing any modifications to your chosen version control system steers a team toward more consistent infrastructure, and tracks any changes you do have to make.

Convergence

Our services should take responsibility for their own state being in line with policy; over time, the overall system will tend to correctness.

This one is simple: if we don’t have every aspect of a service policy defined, how can they know if their current state is in line? If you are familiar with Kubernetes or any other orchestration platform, you know that we define a Spec and a Status, and a the system compares the two, making sure to take the necessary steps to bring them into a consistent state. Our infrastructure should operate in the same way. Ideally this happens through some sort of automated process, but we can at least ensure that we are capable of comparing manually.

Conclusions

The stated benefit of parameters in any IaC configuration is the efficiency of being able to provision multiple sets of infrastructure from the same script. In reality, it mostly only allows someone in your organization to provision infrastructure without understanding exactly what the underlying pieces are. Copying, pasting, and modifying is not that much harder (if at all) than entering parameter values in a GUI or CLI. There are much fewer benefits than drawbacks.

However, parameters are not the true issue, it is how we choose to use them. As mentioned in the disclaimers, parameters used the right way can be very advantageous. I am simply advocating for creating a IaC library that allows you to provision all of your infrastructure without providing runtime arguments. A good rule of thumb would be that every time you create a new infrastructure configuration in your organization, at least one new configuration file should also be created.

As always, send me a message @HashedDan on Twitter for any questions or comments!