Templating with GeoEngineer

A couple of months ago I joined the Infra team at Coinbase. One of the first projects I worked on was codifying all of our cloud infrastructure. We evaluated a number of different options for how to codify that infrastructure, and ended up using a thin wrapper called GeoEngineer (Geo for short) over Terraform. My colleagues Graham Jenson and Rob Witoff have written and spoken about GeoEngineer previously, and I think they’ve done a good job explaining why we built it and the purpose it serves. Today I’m going to talk about templating with Geo, why templates are useful and how they compare to Terraform modules.

At a high level, Geo is organized around environments. These correspond to the particular AWS account or VPC that you are planning and/or applying your resources into. Additionally, you can further organize your code via projects, which is a logical grouping of resources within an environment. These concepts help you think about where to define your resources and how to group them, but as you add more and more resources, you need additional logic to help keep things simple and maintainable. This is where templates come into play — they let you codify best practices.

For example, as we were migrating towards what we’re calling consolidated login and codifying all of the resources roles, a common pattern was defining a role for some project, letting some Amazon service assume that role, and giving that role some permissions.

role = resource('aws_iam_role', 'role-name') {

  name 'role-name'

  assume_role_policy {

    "Version": "2012-10-17",

    "Statement": [

      {

        "Effect": "Allow",

        "Principal": {

          "Service": "lambda.amazonaws.com"

        },

        "Action": "sts:AssumeRole"

      }

    ]

  }.to_json

}

policy = resource('aws_iam_policy', 'role-name-policy') {

  name 'role-name-policy'

  _policy_file "iam/resources/infra/role-name-policy.json"

}

resource('aws_iam_policy_attachment', 'role-name-attachment') {

  name 'role-name-attachment'

  _policy policy

  roles [role]
}

Sample role for a lambda function. role_with_policies.tf

In most of these cases, the assume_role_policy for each service was the same, just with a different service name, i.e. vpc-flow-logs.amazonaws.com or lambda.amazonaws.com . Additionally, the role-name , policy-name , and attachment-name were similarly the same, or derived from the same base. And the only purpose of the aws_iam_policy_attachment is to connect the two resources we just created, it doesn’t require any additional information. So in order to create those 3 resources, all we really need to know is the role name, the service for the assume role policy, and a list of policy files. It’s a complex implementation with a simple interface.

This allowed us to create a RoleWithPolicies template, which really simplified things.

class RoleWithPolicies < GeoEngineer::Template
  attr_reader :role, :policies
  # parameters are:
  # {
  #   role_name: Name of the role (Required)
  #   service: Name of service for assume role (Optional)
  #   assume_policy: JSON assume role policy (Optional)
  #   policies: { <policy_name>: <path to policy file> } (Required)
  # }
  def initialize(name, project, parameters)
    validate_required_parameters(parameters, %i(role_name service policies))
    validate_not_empty(parameters, %i(policies))
    super(name, project, parameters)
    @role = create_role(parameters)
    @policies = create_policies(parameters)
    create_policy_attachments(@role, @policies)
  end
  
  def create_role(parameters)
    assume_policy = if parameters[:service]
                      _assume_policy(parameters[:service])
                    elsif parameters[:assume_policy]
                      parameters[:assume_policy]
                    end
    resource('aws_iam_role', parameters[:role_name]) {
      name parameters[:role_name]
      assume_role_policy assume_policy
    }
  end
  
  def create_policies(parameters)
    parameters[:policies].map do |(name, policy_file)|
      policy_name = [parameters[:role_name], name].join("::")
      resource('aws_iam_policy', policy_name) {
        name policy_name
        _policy_file policy_file
      }
    end
  end
  
  def create_policy_attachments(role, policies)
    policies.map do |policy|
      resource('aws_iam_policy_attachment', policy.name) {
        name policy.name
        roles [role]
        _policy policy
      }
    end
  end
end

role_with_policies.rb

Using this new template, we can codify the role from earlier as follows:

params = {
  role_name: 'role-name',
  service: 'lambda.amazonaws.com',
  policies: {
    'policy-1': 'iam/policies/role-name-policy-1.json',
    'policy-2': 'iam/policies/role-name-policy-2.json',
    ...
  }
}
env.from_template('role_with_policies', 'role-name', params)

Sample role codified with Geo, demonstrating attaching multiple policies. example_role.rb

The great thing about this particular template is that the more policies you need to attach to a single role, the more lines of code you save. We’ve easily saved hundreds of lines using templates for users, roles, and groups. Another benefit is that the naming ends up being more consistent, which just satisfies my inner OCD.

* * *

Now if you’ve already worked with Terraform quite a bit, you might be thinking: “I can do that same thing using Terraform modules — why should I use Geo?”. And indeed, they do very similar things — Geo just makes it easier to do more while smoothing over some of Terraform’s rough edges. (I’d also like to point out that we’re very excited about the progress demonstrated in Terraform 0.8, and hope that one day we won’t need Geo anymore — but until then, Geo all the way!).

Lets look at another example where we’ve used templates at Coinbase: VPC routing. In our VPCs, we often have one or more route table that needs to be connected to various interfaces like NAT gateways, peering connections, and VPN gateways. With Terraform modules, I was able to create a module for each interface type. For example, with VPN gateways, we wanted 2 routes for each route table attached to the gateway:

variable "cidr_blocks" {
  type    = "list"
  default = ["192.168.1.0/16", "192.168.2.0/16"]
}

variable "virtual_gateway_id" {}

variable "rtb_ids" {
  type = "list"
}

variable "num_route_tables" {
  description = "This should equal the length of the route_table_ids variable. This is a work around because count can't reference a module output."
}

resource "aws_route" "virtual_gateway_first" {
  count                  = "${var.num_route_tables}"
  route_table_id         = "${element(var.rtb_ids, count.index)}"
  destination_cidr_block = "${element(var.cidr_blocks, 0)}"
  gateway_id             = "${var.virtual_gateway_id}"
}

resource "aws_route" "virtual_gateway_second" {
  count                  = "${var.num_route_tables}"
  route_table_id         = "${element(var.rtb_ids, count.index)}"
  destination_cidr_block = "${element(var.cidr_blocks, 1)}"
  gateway_id             = "${var.virtual_gateway_id}"
}

Example Terraform Module: vpn_gateway_routes_module.tf

Great. Now we have a nifty module we can invoke anywhere we’d like via:

module "my_virtual_gateway_routes" {
  source             = "./modules/virtual_gateway_routes"
  virtual_gateway_id = "${aws_vpn_gateway.my_gateway.id}"
  route_table_ids = [
    "${aws_route_table.first.id}",
    "${aws_route_table.second.id}",
    "${aws_route_table.third.id}"
  ]
  num_route_tables = "3"
}

vpn_gateway_routes.tf

Which is certainly better than the raw terraform. However, you’ll notice a couple things:

• You need a new module for each interface type: VPN gateway, NAT gateway, VPC Peering connection, Internet gateway, etc…

• We’ve already got notes about “work arounds” — this particular one relates to restrictions on how module outputs can be used as inputs to other modules

• There’s no validation that the end user provided the data in the correct format

• We have limited functions available for data manipulation

Enter Geo. With Geo, we can achieve the same functionality AND address some of the shortcomings with modules. Let’s see what that same VPN Gateway route module looks like as a Geo template:

class Route < GeoEngineer::Template
  class InvalidInterfaceType < StandardError; end
  VALID_INTERFACE_TYPES = %w(
    virtual_gateway internet_gateway nat_gateway 
    vpc_peering_connection instance network_interface
  ).freeze
  
  attr_reader :routes
  
  # parameters are:
  # {
  #   interface_id: ID of interface resource (Required)
  #   interface_type: type of interface resource (Required)
  #   route_table_ids: [<route_table_id>, ...] (Required)
  #   destination_cidr_blocks: [<cidr_block>, ...] (Required)
  # }
  def initialize(name, project, params)
    validate_required_parameters(
      params,
      %i(interface_id interface_type route_table_ids destination_cidr_blocks)
    )
    validate_interface_type(params[:interface_type])
    super(name, project, params)
    @routes = create_routes(params)
  end
 
  def validate_interface_type(interface_type)
    error_msg = "#{interface_type} is not a valid interface type"
    raise InvalidInterfaceType, error_msg unless VALID_INTERFACE_TYPES.include?(interface_type)
  end
  
  def create_routes(params)
    params[:destination_cidr_blocks].each_with_index.map do |cidr_block, index|
      params[:route_table_ids].map do |route_table_id| 
        route_name = [route_table_id, params[:interface_id], index].join("_")
        resource('aws_route', route_name) {
          route_table_id(route_table_id)
          destination_cidr_block(cidr_block)
          case params[:interface_type]
          when 'virtual_gateway', 'internet_gateway'
            then gateway_id(interface_id)
          when 'nat_gateway'
            then nat_gateway_id(interface_id)
          when 'vpc_peering_connection'
            then vpc_peering_connection_id(interface_id)
          when 'instance'
            then instance_id(interface_id)
          when 'network_interface'
            then network_interface_id(interface_id)
          end
        }
      end
    end.flatten.compact
  end
end

vpn_gateway_routes_template.rb

Which can be used like this:

params = {
  interface_id: "${aws_vpn_gateway.my_gateway.id}",
  interface_type: 'virtual_gateway',
  route_table_ids: [ 
    "${aws_route_table.first.id}",
    "${aws_route_table.second.id}",
    "${aws_route_table.third.id}"
  ],
  destination_cidr_blocks: ["192.168.1.0/16", "192.168.2.0/16"]
}
env.from_template('route', 'my_virtual_gateway_routes', params)

vpn_gateway_routes.rb

With the above Geo example we’ve added validations, cleaner error handling and combined 5 copy and paste modules into a more flexible and powerful tool. Having the full power of the Ruby language means that we can do things that simply aren’t possible with Terraform modules — in this case, switch statements and input validation.

This is a just a brief demonstration of the power of Geo templates to keep your infrastructure code simple and re-usable, and if I’ve piqued your interest, tune in for part 2 for more about template design and best practices.

Resources:

• Geo on Github

• Treating Infrastructure as Code

• Coinbase: Security by Consensus with Coinbase GeoEngineer on AWS

P.S. If any of this sounds interesting, or you’re into Bitcoin — Coinbase is hiring.

Thanks to Rob Witoff