AWS CloudFormation

AWS CloudFormation is a service that helps you model and set up your Amazon Web Services resources so that you can spend less time managing those resources and more time focusing on your applications that run in AWS. You create a template that describes all the AWS resources that you want (like Amazon EC2 instances or Amazon RDS DB instances), and AWS CloudFormation takes care of provisioning and configuring those resources for you. This includes going across multiple regions. The templates make your infrastructure as code.

AWS CloudFormation templates include several major sections, but only the Resources section is required. Some sections in a template can be in any order. However, as you build your template, it might be helpful to use the logical ordering, because values in one section might refer to values from a previous section. For a brief overview of each section of the template, see http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html

An example of a CloudFormation template and the fields.

{
  "AWSTemplateFormatVersion" : "version date",

  "Description" : "JSON string",

  "Metadata" : {
    template metadata
  },

  "Parameters" : {
    set of parameters
  },

  "Mappings" : {
    set of mappings
  },

  "Conditions" : {
    set of conditions
  },

  "Transform" : {
    set of transforms
  },

  "Resources" : {
    set of resources
  },

  "Outputs" : {
    set of outputs
  }
}

CloudFormation templates also allows control over certain behaviors using the following keywords.

DependsOn: Create an explicit dependency that requires a specified resource to be created before another can begin
CreationPolicy: Define a period of time during which AWS CloudFormation will wait for a signal before marking the specific resource as “Create Complete”
UpdatePolicy: Define how resources are replaced when applying changes to an EC2 Auto Scaling Launch Configuration

AWS CloudFormation creates, updates, and deletes resources in parallel to the extent possible. It automatically determines which resources in a template can be parallelized and which have dependencies that require other operations to finish first.

AWS CloudFormation uses references (Ref: LogicalResourceName) to identify implicit dependencies. When you create a VPC and a subnet, for example, the subnet resource includes a reference to the VPC. This indicates to AWS CloudFormation that the VPC must be created before the subnet. Alternatively, if you create eight subnets within a single VPC, the VPC will be created first and the eight subnets will be created in parallel.

Using AWS API’s we can describe infrastructure in code. This means we can setup and configure the AWS infrastructure programatically using various AWS SDKs. CloudFormation is the service that can be used to create, update and delete the infrastructure. Using CloudFormation we can automate the infrastructure management. Some benefits it provide are:

Automating Infrastructure
- Lower Costs
- Improves Quality
- Improves Flexibility
Describing Infrastructure in Code
- Records and easily defines the whole infrastructure

CloudFormation creates a deployment package for the following programming languages/frameworks:

Node.js
Python
Java
C#/.Net Core

For each of the languages above, the deployment package is usually a single .zip file containing all the compiled code. This is dropped into S3 and can be referenced by Lambda.

AWS CodeBuild

The process of building and deploying the packages can be managed using AWS CodeBuild. CodeBuild is built on top of Docker and so enables customizations for the build environments. The CodeBuild service is pay-by-the-minute service. You only pay for the build time. There is a buildspec.yml template file that is used to configure the CodeBuild process.

CloudFormation Template

CloudFormation works using a JSON template. The template describes an instance of the stack, such as the definitions for EC2,S3, DDB etc. A single template can have multiple stacks. The example below shows a template for creating a wordpress environment.

{
	"AWSTemplateFormatVersion": "2010-09-09",
	"Description": "Simple WordPress",
	"Parameters": {
		"VPC": {
			"Description": "The default VPC",
			"Type": "AWS::EC2::VPC::Id"
		},
		"Subnets": {
			"Description": "At least two public subnets from default VPC.",
			"Type": "List"
		}
	},
	"Mappings": {
		"RegionMap": {
			"eu-west-1": {"AMI": "ami-bff32ccc"},
			"ap-southeast-1": {"AMI": "ami-c9b572aa"},
			"ap-southeast-2": {"AMI": "ami-48d38c2b"},
			"eu-central-1": {"AMI": "ami-bc5b48d0"},
			"ap-northeast-2": {"AMI": "ami-249b554a"},
			"ap-northeast-1": {"AMI": "ami-383c1956"},
			"us-east-1": {"AMI": "ami-60b6c60a"},
			"sa-east-1": {"AMI": "ami-6817af04"},
			"us-west-1": {"AMI": "ami-d5ea86b5"},
			"us-west-2": {"AMI": "ami-f0091d91"}
		}
	},
	"Resources": {
		"EC2Instance": {
			"Type": "AWS::EC2::Instance",
			"Properties": {
				"ImageId": {"Fn::FindInMap": ["RegionMap", {"Ref": "AWS::Region"}, "AMI"]},
				"InstanceType": "t2.nano",
				"NetworkInterfaces": [{
					"AssociatePublicIpAddress": "true",
					"DeviceIndex": "0",
					"GroupSet": [{"Ref": "WebserverSecurityGroup"}],
					"SubnetId": {"Fn::Select": ["0", {"Ref": "Subnets"}]}
				}],
				"Tags": [{
					"Key": "Name",
					"Value": "simple-wordpress"
				}],
				"UserData": {"Fn::Base64": {"Fn::Join": ["", [
					"#!/bin/bash -ex\n",
					"yum install -y php php-mysql mysql httpd\n",
					"cd /var/www/html\n",
					"curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar\n",
					"php wp-cli.phar core download\n",
					"php wp-cli.phar core config --dbname=wordpress --dbuser=wordpress --dbpass=wordpress --dbhost=", {"Fn::GetAtt": ["Database", "Endpoint.Address"]}, "\n",
					"php wp-cli.phar core install --url=http://$(curl http://169.254.169.254/latest/meta-data/public-hostname) --title=Globomantics-Blog --admin_user=admin --admin_password=UBeDzefeX3jihRQUkrGwj2qYWnyBcp --admin_email=adam@globomatics.com --skip-email\n",
					"rm wp-cli.phar\n",
					"service httpd start\n"
				]]}}
			}
		},
		"DBSubnetGroup": {
			"Type": "AWS::RDS::DBSubnetGroup",
			"Properties": {
				"DBSubnetGroupDescription": "DB subnet group",
				"SubnetIds": {"Ref": "Subnets"}
			}
		},
		"Database": {
			"Type": "AWS::RDS::DBInstance",
			"Properties": {
				"AllocatedStorage": "5",
				"DBInstanceClass": "db.t2.micro",
				"DBInstanceIdentifier": "simple-wordpress",
				"DBName": "wordpress",
				"Engine": "MySQL",
				"MasterUsername": "wordpress",
				"MasterUserPassword": "wordpress",
				"VPCSecurityGroups": [{"Fn::GetAtt": ["DatabaseSecurityGroup", "GroupId"]}],
				"DBSubnetGroupName": {"Ref": "DBSubnetGroup"},
				"StorageType": "gp2"
			}
		},
		"WebserverSecurityGroup": {
			"Type": "AWS::EC2::SecurityGroup",
			"Properties": {
				"GroupDescription": "simple-wordpress-webserver",
				"VpcId": {"Ref": "VPC"},
				"SecurityGroupIngress": [{
					"CidrIp": "0.0.0.0/0",
					"FromPort": 80,
					"IpProtocol": "tcp",
					"ToPort": 80
				}]
			}
		},
		"DatabaseSecurityGroup": {
			"Type": "AWS::EC2::SecurityGroup",
			"Properties": {
				"GroupDescription": "simple-wordpress-database",
				"VpcId": {"Ref": "VPC"},
				"SecurityGroupIngress": [{
					"IpProtocol": "tcp",
					"FromPort": "3306",
					"ToPort": "3306",
					"SourceSecurityGroupId": {"Ref": "WebserverSecurityGroup"}
				}]
			}
		}
	},
	"Outputs": {
		"WordpressURL": {
			"Description": "The URL of the simple WordPress environment.",
			"Value": {"Fn::Join": ["", ["http://", {"Fn::GetAtt": ["EC2Instance", "PublicIp"]}]]}
		}
	}
}

A CloudFormation template at minimum requires the following:

AWSTempalteFormatVersion
- Example: 2010-09-09 (must use AWS defined version)
Description
- A short description
Resources
- Includes the infrastructure resources

Ideally, the best practice is to have a single template for all environments. The template would have all the configuration definitions for those environments and makes it easier for management since the template is source controlled and easily visible/traceable.

CloudFormation supports almost all the resource types defined in AWS. For example – “AWS::EC2::Instance”

The resource has a ‘type’ field and ‘properties’ field. The properties describes the actual configuration for that resource. For example, for an EC2 Instance we could have the following properties:

ImageId
InstanceType
NetworkInterfaces

CloudFormation is able to automatically resolve dependencies between resources. For example, when launching an EC2 instance it requires a Security Group. When defining the EC2Instance, as long as the Security Group is referenced (using keyword “ref”) CloudFormation is able to automatically resolve the references. Also, the order of the dependencies do not matter as CloudFormation will figure it out.

The AWS CLI can be used to call the cloudformation API. For example:

aws cloudformation create-stack --stack-name sample-stack-nae --template-body file://...

Note: make sure the CLI user has the correct policies defined in its role for CloudFormation.

CloudFormation StackSets

Extends functionality of stacks by enabling you to create, update and elete stacks across multiple accounts and regions in single operation.

Account Management

When using CloudFormation we can manage the permissions using accounts in a couple of different ways.

Same account across different stacks
- Easier to manage
- Difficult to separate permissions/access between the stacks
- Better for smaller teams
Multiple accounts
- Separates permissions/access across stacks
- Resource limits can be controlled per account
- Creates overhead of managing the different accounts
- Better for large teams/organizations
  - AWS Organizations is a new feature geared to help manage multiple accounts

References

AWS SAM (Serverless Application Model)
https://github.com/awslabs/serverless-application-model

AWS Automating CloudFormation
https://app.pluralsight.com/player?course=aws-automating-cloudformation

Account Management

References

Related