Overview of AWS CloudFormation

Using AWS API’s we can describe infrastructure in code. This means we can setup and configure the AWS infrastructure programatically using various AWS SDKs. CloudFormation is the service that can be used to create, update and delete the infrastructure. Using CloudFormation we can automate the infrastructure management. Some benefits it provide are:

  • Automating Infrastructure
    • Lower Costs
    • Improves Quality
    • Improves Flexibility
  • Describing Infrastructure in Code
    • Records and easily defines the whole infrastructure

 

CloudFormation creates a deployment package for the following programming languages/frameworks:

  • Node.js
  • Python
  • Java
  • C#/.Net Core

For each of the languages above, the deployment package is usually a single .zip file containing all the compiled code. This is dropped into S3 and can be referenced by Lambda.

 

AWS CodeBuild

The process of building and deploying the packages can be managed using AWS CodeBuild. CodeBuild is built on top of Docker and so enables customizations for the build environments. The CodeBuild service is pay-by-the-minute service. You only pay for the build time. There is a buildspec.yml template file that is used to configure the CodeBuild process.

 

CloudFormation Template

CloudFormation works using a JSON template. The template describes an instance of the stack, such as the definitions for EC2,S3, DDB etc. A single template can have multiple stacks. The example below shows a template for creating a wordpress environment.

{
	"AWSTemplateFormatVersion": "2010-09-09",
	"Description": "Simple WordPress",
	"Parameters": {
		"VPC": {
			"Description": "The default VPC",
			"Type": "AWS::EC2::VPC::Id"
		},
		"Subnets": {
			"Description": "At least two public subnets from default VPC.",
			"Type": "List"
		}
	},
	"Mappings": {
		"RegionMap": {
			"eu-west-1": {"AMI": "ami-bff32ccc"},
			"ap-southeast-1": {"AMI": "ami-c9b572aa"},
			"ap-southeast-2": {"AMI": "ami-48d38c2b"},
			"eu-central-1": {"AMI": "ami-bc5b48d0"},
			"ap-northeast-2": {"AMI": "ami-249b554a"},
			"ap-northeast-1": {"AMI": "ami-383c1956"},
			"us-east-1": {"AMI": "ami-60b6c60a"},
			"sa-east-1": {"AMI": "ami-6817af04"},
			"us-west-1": {"AMI": "ami-d5ea86b5"},
			"us-west-2": {"AMI": "ami-f0091d91"}
		}
	},
	"Resources": {
		"EC2Instance": {
			"Type": "AWS::EC2::Instance",
			"Properties": {
				"ImageId": {"Fn::FindInMap": ["RegionMap", {"Ref": "AWS::Region"}, "AMI"]},
				"InstanceType": "t2.nano",
				"NetworkInterfaces": [{
					"AssociatePublicIpAddress": "true",
					"DeviceIndex": "0",
					"GroupSet": [{"Ref": "WebserverSecurityGroup"}],
					"SubnetId": {"Fn::Select": ["0", {"Ref": "Subnets"}]}
				}],
				"Tags": [{
					"Key": "Name",
					"Value": "simple-wordpress"
				}],
				"UserData": {"Fn::Base64": {"Fn::Join": ["", [
					"#!/bin/bash -ex\n",
					"yum install -y php php-mysql mysql httpd\n",
					"cd /var/www/html\n",
					"curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar\n",
					"php wp-cli.phar core download\n",
					"php wp-cli.phar core config --dbname=wordpress --dbuser=wordpress --dbpass=wordpress --dbhost=", {"Fn::GetAtt": ["Database", "Endpoint.Address"]}, "\n",
					"php wp-cli.phar core install --url=http://$(curl http://169.254.169.254/latest/meta-data/public-hostname) --title=Globomantics-Blog --admin_user=admin --admin_password=UBeDzefeX3jihRQUkrGwj2qYWnyBcp --admin_email=adam@globomatics.com --skip-email\n",
					"rm wp-cli.phar\n",
					"service httpd start\n"
				]]}}
			}
		},
		"DBSubnetGroup": {
			"Type": "AWS::RDS::DBSubnetGroup",
			"Properties": {
				"DBSubnetGroupDescription": "DB subnet group",
				"SubnetIds": {"Ref": "Subnets"}
			}
		},
		"Database": {
			"Type": "AWS::RDS::DBInstance",
			"Properties": {
				"AllocatedStorage": "5",
				"DBInstanceClass": "db.t2.micro",
				"DBInstanceIdentifier": "simple-wordpress",
				"DBName": "wordpress",
				"Engine": "MySQL",
				"MasterUsername": "wordpress",
				"MasterUserPassword": "wordpress",
				"VPCSecurityGroups": [{"Fn::GetAtt": ["DatabaseSecurityGroup", "GroupId"]}],
				"DBSubnetGroupName": {"Ref": "DBSubnetGroup"},
				"StorageType": "gp2"
			}
		},
		"WebserverSecurityGroup": {
			"Type": "AWS::EC2::SecurityGroup",
			"Properties": {
				"GroupDescription": "simple-wordpress-webserver",
				"VpcId": {"Ref": "VPC"},
				"SecurityGroupIngress": [{
					"CidrIp": "0.0.0.0/0",
					"FromPort": 80,
					"IpProtocol": "tcp",
					"ToPort": 80
				}]
			}
		},
		"DatabaseSecurityGroup": {
			"Type": "AWS::EC2::SecurityGroup",
			"Properties": {
				"GroupDescription": "simple-wordpress-database",
				"VpcId": {"Ref": "VPC"},
				"SecurityGroupIngress": [{
					"IpProtocol": "tcp",
					"FromPort": "3306",
					"ToPort": "3306",
					"SourceSecurityGroupId": {"Ref": "WebserverSecurityGroup"}
				}]
			}
		}
	},
	"Outputs": {
		"WordpressURL": {
			"Description": "The URL of the simple WordPress environment.",
			"Value": {"Fn::Join": ["", ["http://", {"Fn::GetAtt": ["EC2Instance", "PublicIp"]}]]}
		}
	}
}

 

A CloudFormation template at minimum requires the following:

  • AWSTempalteFormatVersion
    • Example: 2010-09-09 (must use AWS defined version)
  • Description
    • A short description
  • Resources
    • Includes the infrastructure resources

Ideally, the best practice is to have a single template for all environments. The template would have all the configuration definitions for those environments and makes it easier for management since the template is source controlled and easily visible/traceable.

CloudFormation supports almost all the resource types defined in AWS. For example – “AWS::EC2::Instance”

The resource has a ‘type’ field and ‘properties’ field. The properties describes the actual configuration for that resource. For example, for an EC2 Instance we could have the following properties:

  • ImageId
  • InstanceType
  • NetworkInterfaces

CloudFormation is able to automatically resolve dependencies between resources. For example, when launching an EC2 instance it requires a Security Group. When defining the EC2Instance, as long as the Security Group is referenced (using keyword “ref”) CloudFormation is able to automatically resolve the references. Also, the order of the dependencies do not matter as CloudFormation will figure it out.

The AWS CLI can be used to call the cloudformation API. For example:

aws cloudformation create-stack --stack-name sample-stack-nae --template-body file://...

Note: make sure the CLI user has the correct policies defined in its role for CloudFormation.

 

Account Management

When using CloudFormation we can manage the permissions using accounts in a couple of different ways.

  • Same account across different stacks
    • Easier to manage
    • Difficult to separate permissions/access between the stacks
    • Better for smaller teams
  • Multiple accounts
    • Separates permissions/access across stacks
    • Resource limits can be controlled per account
    • Creates overhead of managing the different accounts
    • Better for large teams/organizations
      • AWS Organizations is a new feature geared to help manage multiple accounts

 

References

AWS SAM (Serverless Application Model)
https://github.com/awslabs/serverless-application-model

AWS Automating CloudFormation
https://app.pluralsight.com/player?course=aws-automating-cloudformation