AWS EC2

EC2 is the backbone of AWS. Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the Amazon Web Services (AWS) cloud. Using Amazon EC2 eliminates your need to invest in hardware up front, so you can develop and deploy applications faster. You can use Amazon EC2 to launch as many or as few virtual servers as you need, configure security and networking, and manage storage. Amazon EC2 enables you to scale up or down to handle changes in requirements or spikes in popularity, reducing your need to forecast traffic.

Features

  • Virtual computing environments, known as instances
  • Preconfigured templates for your instances, known as Amazon Machine Images (AMIs), that package the bits you need for your server (including the operating system and additional software)
  • Various configurations of CPU, memory, storage, and networking capacity for your instances, known as instance types
  • Secure login information for your instances using key pairs (AWS stores the public key, and you store the private key in a secure place)
  • Storage volumes for temporary data that’s deleted when you stop or terminate your instance, known as instance store volumes
  • Persistent storage volumes for your data using Amazon Elastic Block Store (Amazon EBS), known as Amazon EBS volumes
  • Multiple physical locations for your resources, such as instances and Amazon EBS volumes, known as Regions and Availability Zones
  • A firewall that enables you to specify the protocols, ports, and source IP ranges that can reach your instances using security groups
  • Static IPv4 addresses for dynamic cloud computing, known as Elastic IP addresses
  • Metadata, known as tags, that you can create and assign to your Amazon EC2 resources
  • Virtual networks you can create that are logically isolated from the rest of the AWS cloud, and that you can optionally connect to your own network, known as virtual private clouds (VPCs)
  • By default, AWS has a limit of 20 instances per region. This includes all instances set up on your AWS account. To increase EC2 limits, request a higher limit by providing information about the new limit and regions where it should be applied.

EC2 AMIs can be selected based on:

  • Region
  • Operation System
  • Architecture (32bit / 64bit)
  • Launch Permissions
  • Storage
    • Instance Store (Ephemeral)
    • EBS
  • Virtualization Type

Virtualization Types

AWS EC2 uses Xen Hypervisor (not Hyper V or VMware). Amazon Machine Images (AWS AMI) offers two types of virtualization: Paravirtual (PV) and Hardware Virtual Machine (HVM). Each solution offers its own advantages.

HVM guests are fully virtualized. It means that the VMs running on top of their hypervisors are not aware that they are sharing processing time with other clients on the same hardware. The host should have the capability to emulate underlying hardware for each of its guest machines. This virtualization type provides the ability to run an operating system directly on top of a virtual machine without any modification — as if it were run on the bare-metal hardware. The advantage of this is that HVMs can use hardware extensions which provide very fast access to underlying hardware on the host system.

Paravirtualization, on the other hand, is a lighter form of virtualization. This technique is fast, and provides near native speed in comparison to full virtualization. With Paravirtualization, the guest operating system requires some modification before everything can work. These modifications allow the hypervisor to export a modified version of the underlying hardware to the VMs, allowing them near-native performance. All PV machines running on a hypervisor are basically modified operating systems like Solaris or various Linux distributions.

Bottom line – HVM provides hardware extensions for fast access, PV provides better performance

 

Regions, Availability Zones and Local Zones

Amazon EC2 is hosted in multiple locations world-wide. These locations are composed of Regions, Availability Zones, and Local Zones. Each Region is a separate geographic area. Each Region has multiple, isolated locations known as Availability Zones. Local Zones provide you the ability to place resources, such as compute and storage, in multiple locations closer to your end users. Resources aren’t replicated across Regions unless you specifically choose to do so.

 

AMI

An Amazon Machine Image (AMI) is a template that contains a software configuration (for example, an operating system, an application server, and applications). From an AMI, you launch an instance, which is a copy of the AMI running as a virtual server in the cloud. Note that AMI IDs are different between regions.

 

Instances

Instance is a running instance of an AMI in EC2.

Instance Types

When you launch an instance, the instance type that you specify determines the hardware of the host computer used for your instance. Each instance type offers different compute, memory, and storage capabilities and are grouped in instance families based on these capabilities. Select an instance type based on the requirements of the application or software that you plan to run on your instance. Instance families listed below

  • General Purpose (a, m, t)
    • Ideal for running small and mid-size databases, more memory-hungry data processing tasks, caching fleets, and backend servers for SAP, Microsoft SharePoint, and other enterprise applications.
  • Compute Optimized (c)
    • Ideal for front end fleets for high-traffic web sites, on-demand batch processing, distributed analytics, web servers, video encoding, and high performance science and engineering applications like genome analysis, high-energy physics, or computational fluid dynamics.
  • Memory Optimized (r, u, x, z)
    • In general, if you are running a performance-sensitive database you should first look at this family.
  • Storage Optimized (d, h, i)
    • In particular, we designed these instances to be the best platform for large deployments of NoSQL databases like Cassandra and MongoDB.
  • Accelerated Computing (f, g, p, inf)
    • Many of you are running computational chemistry, rendering, and financial analysis applications on CG1 instances today to take advantage of the speedup you can get from GPGUs.

Amazon EC2 instances typically run on 64-bit virtual Intel processors as specified in the instance type product pages. For more information about the hardware specifications for each Amazon EC2 instance type, see Amazon EC2 Instance Types. However, confusion may result from industry naming conventions for 64-bit CPUs. Chip manufacturer Advanced Micro Devices (AMD) introduced the first commercially successful 64-bit architecture based on the Intel x86 instruction set. Consequently, the architecture is widely referred to as AMD64 regardless of the chip manufacturer. Windows and several Linux distributions follow this practice. This explains why the internal system information on an Ubuntu or Windows EC2 instance displays the CPU architecture as AMD64 even though the instances are running on Intel hardware.

Nitro Instances

The Nitro system is a collection of AWS-built hardware and software components that enable high performance, high availability, and high security. In addition, the Nitro system provides bare metal capabilities that eliminate virtualization overhead and support workloads that require full access to host hardware.

Instance Purchasing Option

Amazon EC2 has the following purchasing options:

  • On-Demand Instances – Pay, by the second, for the instances that you launch.
    • Low cost and flexibility, no up front cost
    • Apps with short term, spiky, unpredictable
  • Savings Plans – Reduce your Amazon EC2 costs by making a commitment to a consistent amount of usage, in USD per hour, for a term of 1 or 3 years.
  • Reserved Instances – Reduce your Amazon EC2 costs by making a commitment to a consistent instance configuration, including instance type and Region, for a term of 1 or 3 years.
    • Applications with steady state or predictable usage
    • Reserved capacity
    • Upfront costs
      • Standard RI (up to 75% off)
      • Convertible RI (up to 54% off)
      • Schedule RI – launch based on a schedule
  • Scheduled Instances – Purchase instances that are always available on the specified recurring schedule, for a one-year term.
  • Spot Instances – Request unused EC2 instances, which can reduce your Amazon EC2 costs significantly.
    • Apps that have flexible start and end times
    • Apps that are only feasible with low compute prices
    • Needs for large amount of compute
    • If you terminate the instance, you pay for that remaining hour
    • If AWS terminates the instance, you do not pay for that last hour
  • Dedicated Hosts – Pay for a physical host that is fully dedicated to running your instances, and bring your existing per-socket, per-core, or per-VM software licenses to reduce costs.
    • Regulatory requirements where multi-tenant is not allowed
    • Licenses requiring single host
    • Can be purchased on-demand
  • Dedicated Instances – Pay, by the hour, for instances that run on single-tenant hardware.
  • Capacity Reservations – Reserve capacity for your EC2 instances in a specific Availability Zone for any duration.

If you require a capacity reservation, purchase Reserved Instances or Capacity Reservations for a specific Availability Zone, or purchase Scheduled Instances. Spot Instances are a cost-effective choice if you can be flexible about when your applications run and if they can be interrupted. Dedicated Hosts or Dedicated Instances can help you address compliance requirements and reduce costs by using your existing server-bound software licenses.

Instance Lifecycle

By working with Amazon EC2 to manage your instances from the moment you launch them through their termination, you ensure that your customers have the best possible experience with the applications or sites that you host on your instances.

The following illustration represents the transitions between instance states. Notice that you can’t stop and start an instance store-backed instance. For more information about instance store-backed instances, see Storage for the Root Device.

 The instance lifecycle
Instance state Description Instance usage billing
pending The instance is preparing to enter the running state. An instance enters the pending state when it launches for the first time, or when it is started after being in the stopped state. Not billed
running The instance is running and ready for use. Billed
stopping The instance is preparing to be stopped or stop-hibernated. Not billed if preparing to stop

Billed if preparing to hibernate

stopped The instance is shut down and cannot be used. The instance can be started at any time. Not billed
shutting-down The instance is preparing to be terminated. Not billed
terminated The instance has been permanently deleted and cannot be started. Not billed

Note

Reserved Instances that applied to terminated instances are billed until the end of their term according to their payment option. For more information, see Reserved Instances

EBS backed volumes and EC2 termination

When an instance is terminated, Amazon Elastic Compute Cloud (Amazon EC2) uses the value of the DeleteOnTermination attribute for each attached EBS volume to determine whether to preserve or delete the volume when the instance is terminated. By default, the DeleteOnTermination attribute for the root volume of an instance is set to true, but it is set to false for all other volume types.

Retirement
An instance is scheduled to be retired when AWS detects the irreparable failure of the underlying hardware hosting the instance. When an instance reaches its scheduled retirement date, it is stopped or terminated by AWS. If your instance root device is an Amazon EBS volume, the instance is stopped, and you can start it again at any time. If your instance root device is an instance store volume, the instance is terminated, and cannot be used again.

Instance Profile

Amazon EC2 uses an instance profile as a container for an IAM role. When you create an IAM role using the IAM console, the console creates an instance profile automatically and gives it the same name as the role to which it corresponds. If you use the Amazon EC2 console to launch an instance with an IAM role or to attach an IAM role to an instance, you choose the role based on a list of instance profile names.

An instance profile can contain only one IAM role. This limit cannot be increased.

Instance Metadata

Instance metadata is data about your instance that you can use to configure or manage the running instance. Instance metadata is divided into categories, for example, host name, events, and security groups.

You can access instance metadata from a running instance using one of the following methods:

  • Instance Metadata Service Version 1 (IMDSv1) – a request/response method
  • Instance Metadata Service Version 2 (IMDSv2) – a session-oriented method

By default, you can use either IMDSv1 or IMDSv2, or both. The instance metadata service distinguishes between IMDSv1 and IMDSv2 requests based on whether, for any given request, either the PUT or GET headers, which are unique to IMDSv2, are present in that request.

[ec2-user ~]$ TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/

 

Networking

Elastic IP

An Elastic IP address is a static IPv4 address designed for dynamic cloud computing. An Elastic IP address is associated with your AWS account. With an Elastic IP address, you can mask the failure of an instance or software by rapidly remapping the address to another instance in your account. It only supports IPv4, no IPv6.

Elastic IP is a great way of keeping a web application available. If an instance happens to go down, a new one can be created and assigned the elastic IP to keep the application online.

Note that Elastic IP requires an Internet Gateway. If the VPC of the EC2 does not have internet access, you will not be able to associate an elastic IP to it.

You can have one Elastic IP (EIP) address associated with a running instance at no charge. If you associate additional EIPs with that instance, you will be charged for each additional EIP associated with that instance per hour on a pro rata basis. Additional EIPs are only available in Amazon VPC. To ensure efficient use of Elastic IP addresses, we impose a small hourly charge when these IP addresses are not associated with a running instance or when they are associated with a stopped instance or unattached network interface.

Enhanced Networking

Enhanced networking provides higher bandwidth, higher packet-per-second (PPS) performance, and consistently lower inter-instance latencies. Support BGP routing

If your packets-per-second rate appears to have reached its ceiling, you should consider moving to enhanced networking, because you have likely reached the upper thresholds of the virtual network interface driver. There are 3 different virtual network adapters

  • VIF
  • Intel 82599 VF
  • Elastic Network Adapter

 

Elastic Load Balancers (ELB)

Elastic Load Balancing distributes incoming application or network traffic across multiple targets, such as Amazon EC2 instances, containers, and IP addresses, in multiple Availability Zones. Elastic Load Balancing scales your load balancer as traffic to your application changes over time. It can automatically scale to the vast majority of workloads.

Note that the load balancers are always running, and therefore always stacking up charges.

Elastic Load Balancing supports three types of load balancers: Application Load Balancers, Network Load Balancers, and Classic Load Balancers.

Application Load Balancer operates at the request level (layer 7), routing traffic to targets – EC2 instances, containers, IP addresses and Lambda functions based on the content of the request. Ideal for advanced load balancing of HTTP and HTTPS traffic, Application Load Balancer provides advanced request routing targeted at delivery of modern application architectures, including microservices and container-based applications. Application Load Balancer simplifies and improves the security of your application, by ensuring that the latest SSL/TLS ciphers and protocols are used at all times.

  • There are no IPs assigned, always referenced by the DNS name

Network Load Balancer operates at the connection level (Layer 4), routing connections to targets – Amazon EC2 instances, microservices, and containers – within Amazon Virtual Private Cloud (Amazon VPC) based on IP protocol data. Ideal for load balancing of both TCP and UDP traffic, Network Load Balancer is capable of handling millions of requests per second while maintaining ultra-low latencies. Network Load Balancer is optimized to handle sudden and volatile traffic patterns while using a single static IP address per Availability Zone. It is integrated with other popular AWS services such as Auto Scaling, Amazon EC2 Container Service (ECS), Amazon CloudFormation and AWS Certificate Manager (ACM).

Classic Load Balancer provides basic load balancing across multiple Amazon EC2 instances and operates at both the request level and connection level. Classic Load Balancer is intended for applications that were built within the EC2-Classic network. We recommend Application Load Balancer for Layer 7 and Network Load Balancer for Layer 4 when using Virtual Private Cloud (VPC).

Application Load Balancer vs Network Load Balancer

The network load balancer just forward requests whereas the application load balancer examines the contents of the HTTP request header to determine where to route the request. So, the application load balancer is performing content based routing.

The other difference between the two is important because network load balancing cannot assure availability of the application. This is because it bases its decisions solely on network and TCP-layer variables and has no awareness of the application at all. Generally a network load balancer will determine “availability” based on the ability of a server to respond to ICMP ping, or to correctly complete the three-way TCP handshake.

ICMP (Internet Control Message Protocol) is an error-reporting protocol network devices like routers use to generate error messages to the source IP address when network problems prevent delivery of IP packets. (ping)

Health Checks

Load balancer will automatically perform health checks on EC2 instances an only route traffic to instances that pass the health check. If the instance fails health check, it will automatically be removed from the load balancer.

Instances are reported as InService or OutofService. The following are types of health checks:

  • A ping
  • A connection attempt
  • A page request

Listeners

Before you start using your Application Load Balancer, you must add one or more listeners. A listener is a process that checks for connection requests, using the protocol and port that you configure. The rules that you define for a listener determine how the load balancer routes requests to its registered targets.

Listeners support the following protocols and ports:

  • Protocols: HTTP, HTTPS
  • Ports: 1-65535

Based on the listener, an elastic load balancer supports the following:

  • Internet Facing
  • Internal Facing
  • HTTP/HTTPs

Note that in the least, if there is only 1 instance behind an ELB, there are still 2 active connections. One to the client its listening on and one to the instance behind.

Sticks Sessions / Session Affinity

Routes users to the particular web server that is managing that session. This can be done using client side cookie or configuration duration parameters that can be set at the load balancer.

Target Groups

Each target group is used to route requests to one or more registered targets. When you create each listener rule, you specify a target group and conditions. When a rule condition is met, traffic is forwarded to the corresponding target group. You can create different target groups for different types of requests. For example, create one target group for general requests and other target groups for requests to the microservices for your application.

When you create a target group, you specify its target type, which determines the type of target you specify when registering targets with this target group. After you create a target group, you cannot change its target type.

The following are the possible target types:

  • instance: The targets are specified by instance ID.
  • ip: The targets are IP addresses.
  • lambda: The target is a Lambda function.

When the target type is ip, you can specify IP addresses from one of the following CIDR blocks:

The subnets of the VPC for the target group

  • 10.0.0.0/8 (RFC 1918)
  • 100.64.0.0/10 (RFC 6598)
  • 172.16.0.0/12 (RFC 1918)
  • 192.168.0.0/16 (RFC 1918)

 

Auto Scaling

Auto Scaling is done with a combination of other AWS services such as:

  • Auto Load Balancer
  • CloudWatch (to monitor and trigger the scaling)
  • Auto Scaler

There are 3 main components to Auto Scaling

  • Auto Scaling Launch Configuration
    • Using the launch configuration, the Auto Scaler will launch specific EC2 instance type, size and configuration.
  • Auto Scaling Group
    • Points to the launch configuration
    • Specifies min, max and desired size of Auto Scaling group
    • May reference an ELB
    • Health Check Type
  • An Auto Scaling Policy
    • Specifies how much to scale in or out
    • One or more policy may be attached to an Auto Scaling Group

Every Auto Scaling group requires minimum size and the launch configuration. A launch configuration must have the configuration name, AMI and instance type. Auto scaling groups may use spot or on demand instances. An auto scaling group can be of one of the following plans

  • Manual Scaling
  • Scheduled Scaling
  • Dynamic Scaling
    • Step Scaling – scales based on set of scaling adjustements
    • Simple Scaling – use step over simple. Simple relies on health check replacement to complete

CloudWatch has different metrics that can be used to trigger the Auto Scaler.

  • CPU
  • Network
  • Queue Size

Use AWS Auto Scaling to automatically scale the following resources:

  • Amazon EC2 Auto Scaling groups: Launch or terminate EC2 instances in an Auto Scaling group.
  • Amazon EC2 Spot Fleet requests: Launch or terminate instances from a Spot Fleet request, or automatically replace instances that get interrupted for price or capacity reasons.
  • Amazon ECS: Adjust the ECS service desired count up or down in response to load variations.
  • Amazon DynamoDB: Enable a DynamoDB table or a global secondary index to increase or decrease its provisioned read and write capacity to handle increases in traffic without throttling.
  • Amazon Aurora: Dynamically adjust the number of Aurora read replicas provisioned for an Aurora DB cluster to handle changes in active connections or workload.

The Load Balancer will pickup new instances that are launched and automatically start routing traffic to them.

Auto Scaling Default Termination Policy

The default termination policy is designed to help ensure that your instances span Availability Zones evenly for high availability.

  1. Determine which Availability Zones have the most instances, and at least one instance that is not protected from scale in.
  2. Determine which instances to terminate so as to align the remaining instances to the allocation strategy for the On-Demand or Spot Instance that is terminating. This only applies to an Auto Scaling group that specifies allocation strategies.For example, after your instances launch, you change the priority order of your preferred instance types. When a scale-in event occurs, Amazon EC2 Auto Scaling tries to gradually shift the On-Demand Instances away from instance types that are lower priority.
  3. Determine whether any of the instances use the oldest launch template or configuration:
    1. [For Auto Scaling groups that use a launch template]Determine whether any of the instances use the oldest launch template unless there are instances that use a launch configuration. Amazon EC2 Auto Scaling terminates instances that use a launch configuration before instances that use a launch template.
    2. [For Auto Scaling groups that use a launch configuration]Determine whether any of the instances use the oldest launch configuration.
  4. After applying all of the above criteria, if there are multiple unprotected instances to terminate, determine which instances are closest to the next billing hour. If there are multiple unprotected instances closest to the next billing hour, terminate one of these instances at random.

Instance Scale-In Protection

To control whether an Auto Scaling group can terminate a particular instance when scaling in, use instance scale-in protection. You can enable the instance scale-in protection setting on an Auto Scaling group or on an individual Auto Scaling instance.

Placement Groups

When you launch a new EC2 instance, the EC2 service attempts to place the instance in such a way that all of your instances are spread out across underlying hardware to minimize correlated failures. You can use placement groups to influence the placement of a group of interdependent instances to meet the needs of your workload. Depending on the type of workload, you can create a placement group using one of the following placement strategies:

  • Cluster – packs instances close together inside an Availability Zone. This strategy enables workloads to achieve the low-latency network performance necessary for tightly-coupled node-to-node communication that is typical of HPC applications.
  • Partition – spreads your instances across logical partitions such that groups of instances in one partition do not share the underlying hardware with groups of instances in different partitions. This strategy is typically used by large distributed and replicated workloads, such as Hadoop, Cassandra, and Kafka.
  • Spread – strictly places a small group of instances across distinct underlying hardware to reduce correlated failures.

A cluster placement group is a logical grouping of instances within a single Availability Zone. A placement group can span peered VPCs in the same Region. The chief benefit of a cluster placement group, in addition to a 10 Gbps flow limit, is the non-blocking, non-oversubscribed, fully bi-sectional nature of the connectivity. In other words, all nodes within the placement group can talk to all other nodes within the placement group at the full line rate of 10 Gbps flows and 100 Gbps aggregate without any slowing due to over-subscription.

Placement groups cannot spread across multiple AZs. Therefore it can become a single-point-of-failure, but this should be used for low latency or high throughput required. Only certain types of instances can be launched in a placement group and AWS recommends using homogenous instance types. You cannot merge placement groups and you cannot move an existing instance into a placement group.

 

Storage Types

All AMIS are categorized as either backed by EBS or by Instance Store.

EBS is available for attached volumes once an instance is launched. Before launch, during configuration, we can attach EBS or Instance Store. For the root volume, we can do EBS or Instance Store (ephemeral).

EC2 Instance Store (Ephemeral Instance)

The storage of EC2s can be ephemeral (attached to the instance) or non-ephemeral. Ephemeral instance stores are only available for certain EC2 instances. It has a fixed capacity and disk type, depending on the EC2 instance type. It provides application-level durability.

Because instance storage is directly attatched to the EC2 hosts, they are capable of much higher IOPS (depending on EC2 instance type). AWS claims over 3 million IOPS for an i3.16xLarge, but currently an io1 provisioned IOPS EBS volume caps out around 32,000.

Note that when using Instance Store the instance cannot be stopped manually. It can only be rebooted or terminated. If instances need to be stopped, you must use EBS. This option is less durable than EBS. Also, Instance Store volumes take a little longer to launch. Finally, Instance Stores do not have data store charges (only compute charges, but remember compute cannot be stopped).

Non-ephemeral storage is done by EBS (Elastic Block Store).

Elastic Block Store (EBS)

EBS are attachable storage that come in SSD and HDD types. It supports encryption, snapshots, provisioned capacity and has independent lifecycle than the EC2 instance. This means even if the EC2 instance is terminated, the storage will exist in EBS. There can be one or many EBS volumes mounted to an EC2 instance. See EBS in storage types for more information.

EBS volumes are replicated within an Availability Zone (AZ) and can easily scale to petabytes of data. Think of volumes as a RAID 1.

Note that EBS volumes dont have a direct way to change encryption state. Therefore if an existing instance is running with an unencrypted volume, the only way to encrypt the data is to create a new encrypted volume and transfer the data over. Or you can take a snapshot of the unencrypted volume, create a new one from it with encryption.

 

More more complete information about EBS – see the storage section.

EBS Root Volumes

Default AMIs cannot have encrypted root volumes. You can use third party tools, such as bitlocker. But if we create a copy of the AMI, then the root volume can be encrypted. Additional volumes can also be encrypted. When an instance is terminated, the EBS root volume is also deleted.

Elastic File Storage

EFS is AWS file storage in the cloud that can be shared. It is a petabyte-scale file system that has elastic capacity (dynamically increase/decreate). It supports NFS v4 protocol and compatible with Linux base but not Windows. Amazon EFS offers two storage classes, Standard and Infrequent Access. The Standard storage class is used to store frequently accessed files. The Infrequent Access (IA) storage class is a lower-cost storage class that’s designed for storing long-lived, infrequently accessed files cost-effectively. See EFS in storage types for more information.

Snapshots

You can back up the data on your Amazon EBS volumes to Amazon S3 by taking point-in-time snapshots. Snapshots are incremental backups, which means that only the blocks on the device that have changed after your most recent snapshot are saved. This minimizes the time required to create the snapshot and saves on storage costs by not duplicating data. When you delete a snapshot, only the data unique to that snapshot is removed. Each snapshot contains all of the information that is needed to restore your data (from the moment when the snapshot was taken) to a new EBS volume.

Snapshots of encrypted volumes are automatically encrypted. Volumes restored from encrypted snapshots are also automatically encrypted. Snapshots can only be shared if unencrypted. They can be shared with other AWS accounts or the public marketplace.

Snapshots can be used to create a backup of critical workloads, such as a large database or a file system that spans across multiple EBS volumes. Multi-volume snapshots allow you to take exact point-in-time, data coordinated, and crash-consistent snapshots across multiple EBS volumes attached to an EC2 instance. You are no longer required to stop your instance or to coordinate between volumes to ensure crash consistency, because snapshots are automatically taken across multiple EBS volumes.

When you delete a snapshot, only the data referenced exclusively by that snapshot is removed. Unique data will not be deleted unless all of the snapshots that reference that data are deleted. Deleting previous snapshots of a volume does not affect your ability to restore volumes from later snapshots of that volume.

Deleting a snapshot of a volume has no effect on the volume. Deleting a volume has no effect on the snapshots made from it. Note that you can’t delete a snapshot of the root device of an EBS volume used by a registered AMI. You must first deregister the AMI before you can delete the snapshot.

Some things to note when working with snapshots:

  • When you create a snapshot for an EBS volume that serves as a root device, you should stop the instance before taking the snapshot.
  • You cannot create snapshots from instances for which hibernation is enabled.
  • You cannot create snapshots from hibernated instances.
  • Although you can take a snapshot of a volume while a previous snapshot of that volume is in the pending status, having multiple pending snapshots of a volume can result in reduced volume performance until the snapshots complete.
  • There is a limit of five pending snapshots for a single gp2, io1, or Magnetic volume, and one pending snapshot for a single st1 or sc1 volume. If you receive a ConcurrentSnapshotLimitExceeded error while trying to create multiple concurrent snapshots of the same volume, wait for one or more of the pending snapshots to complete before creating another snapshot of that volume.
  • When a snapshot is created from a volume with an AWS Marketplace product code, the product code is propagated to the snapshot. 

 

User Data

When you launch an instance in Amazon EC2, you have the option of passing user data to the instance that can be used to perform common automated configuration tasks and even run scripts after the instance starts. You can pass two types of user data to Amazon EC2: shell scripts and cloud-init directives. You can also pass this data into the launch wizard as plain text, as a file (this is useful for launching instances using the command line tools), or as base64-encoded text (for API calls).

If you are familiar with shell scripting, this is the easiest and most complete way to send instructions to an instance at launch. Adding these tasks at boot time adds to the amount of time it takes to boot the instance. You should allow a few minutes of extra time for the tasks to complete before you test that the user script has finished successfully.

By default, user data scripts and cloud-init directives run only during the boot cycle when you first launch an instance. You can update your configuration to ensure that your user data scripts and cloud-init directives run every time you restart your instance. For more information, see How can I execute user data with every restart of my EC2 instance? in the AWS Knowledge Center.

In the example script below, the script creates and configures our web server.

#!/bin/bash
yum update -y
amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2
yum install -y httpd mariadb-server
systemctl start httpd
systemctl enable httpd
usermod -a -G apache ec2-user
chown -R ec2-user:apache /var/www
chmod 2775 /var/www
find /var/www -type d -exec chmod 2775 {} \;
find /var/www -type f -exec chmod 0664 {} \;
echo "" > /var/www/html/phpinfo.php

 

RTO and RPO

Recovery Time Objective is how long it takes to recover a system to get it back online. It is measured in units of time (seconds, minutes, hours). Recovery Point Objectives is how much data is recovered (or lost). It is measured in units data size (MB, GB) or time (seconds, minutes, hours).

The RTO/RPO objectives define how you should manage your EC2s, such as how often snapshots should be taken and defining a process for bringing up systems from those snapshots.

 

Security

When an EC2 needs to interact with other AWS services or resources, it is best practice to use roles instead of creating new credentials.

Don’t create an IAM user and pass the user’s credentials to the application or embed the credentials in the application. Instead, create an IAM role that you attach to the EC2 instance to give temporary security credentials to applications running on the instance. When an application uses these credentials in AWS, it can perform all of the operations that are allowed by the policies attached to the role. For details, see Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances.

EC2 Key Pairs

Amazon AWS uses keys to encrypt and decrypt login information. At the basic level, a sender uses a public key to encrypt data, which its receiver then decrypts using another private key. These two keys, public and private, are known as a key pair.

You need a key pair to be able to connect to your instances. The way this works on Linux and Windows instances is different.

The difference between Linux and Windows instances is that Linux instances do not have a password already set and you must use the key pair to log in to Linux instances. On the other hand, on Windows instances, you need the key pair to decrypt the administrator password. Using the decrypted password, you can use RDP and then connect to your Windows instance.

Administrator password is encrypted with public key of key pair and private key is required to decrypt the password. While log in to instance as the administrator with decrypted password.

 

Pricing

The following should be considered for estimating cost on EC2

  • Clock hours of server time
  • Machine configuration
  • Purchase type (On Demand, Reserved, Spot)
  • Number of instances
  • Load balancing
  • Detailed monitoring
  • Auto scaling
  • Elastic IP addresses
  • Operating system and other software packages
  • Storage – if using EBS. Instance Store/ephemeral is free or part of the instance cost.

Reserved and Spot Instances

RI can be up to 75% cheaper. There are 3 RI types:

  • Standard RI
  • Convertible RI – can be converted to other types
  • Scheduled RI – pay full, partial, monthly

Spot instances share compute capacity with others in AWS and can be up to 30 – 45% cheaper. But must work on schedule with other AWS users.

Another way to save EC2 costs is by hibernating or stopping your instances. Only run them when needed.

 

Amazon Certificate Manager

AWS Certificate Manager is a service that lets you easily provision, manage, and deploy public and private Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with AWS services and your internal connected resources. SSL/TLS certs provisioned through ACM are free.

ACM also handles cert renewals, however the certs cannot be downloaded. They can only be used with AWS services such as ELB (elastic load balancers), CloudFront and APIs on API Gateway.

ACM can support up to 10 domains per certificate. The domain must be verified before ACM will allow the cert generation. Emails will be sent to common administrator email addresses to each domain, of which it must be responded within 72 hours.

To assign a cert in ELB, change it on the Listeners tab. There is an option there to select the cert from ACM

 

Workspaces

Amazon WorkSpaces enables you to provision virtual, cloud-based Microsoft Windows or Amazon Linux desktops for your users, known as WorkSpaces. Amazon WorkSpaces eliminates the need to procure and deploy hardware or install complex software. You can quickly add or remove users as your needs change. Users can access their virtual desktops (VDI) from multiple devices or web browsers.

Features

  • Windows and Linux (can do Windows 7 as well)
  • Workspaces can be personalized by the users (can also push out policies)
  • By default users have admin access in workspace
  • Workspaces are persistent
  • The data on the data drive (d:\) is backed up
  • No AWS account for users to access Workspaces

 

ECS

Amazon Elastic Container Service (Amazon ECS) is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster. You can host your cluster on a serverless infrastructure that is managed by Amazon ECS by launching your services or tasks using the Fargate launch type. For more control you can host your tasks on a cluster of Amazon Elastic Compute Cloud (Amazon EC2) instances that you manage by using the EC2 launch type.

The following diagram shows the architecture of an Amazon ECS environment using the Fargate launch type:

To deploy applications on Amazon ECS, your application components must be architected to run in containers. A Docker container is a standardized unit of software development, containing everything that your software application needs to run: code, runtime, system tools, system libraries, etc. Containers are created from a read-only template called an image.

Images are typically built from a Dockerfile, a plain text file that specifies all of the components that are included in the container. These images are then stored in a registry from which they can be downloaded and run on your cluster.

View following post for complete description and sample project.

Load Balanced and Auto Scaling containerized app with AWS ECS