A detailed look at AWS EC2
EC2 stands for Elastic Compute Cloud. The service provides virtual servers in the cloud. The service as a whole is extremely scalable, meaning that it can grow with your business and can help reduce the amount of traffic forecasting required as it can scale up and down to cope with traffic demands.
By default, all EC2 instances are launched with a private IP address. This enables them to communicate with other instances within the same VPC. We can choose to launch our instances with a Public IP address too – whether this happens by default when you launch an instance will depend on your VPC / Subnet settings.
The default VPC and subnets are configured to give a private IP by default
When you launch an instance, you can add your own bash scripts under the advanced details section. For example:
Yum update -y
Yum install -y httpd
Service httpd start
This would mean that your instance would be launched with HTTPD (Apache Web Server) already installed. You can view the scripts that ran on instance launch by using a curl command in the terminal: Curl http://169.254.169.254/latest/user-data
You can also view the metadata using: Curl http://169.254.169.254/latest/meta-data
When you wish to connect to your instance via SSH, you’ll need to have your public + private keypairs. These can be created and downloaded in the final step of instance creation. Once created, highlight the instance and press the connect button. You’ll then be prompted to update the permissions on the keypair file and given a connect script.
There are a few primary purchasing options for EC2 instances. They are described below:
|On Demand||The on-demand instances can be provisioned or terminated at any time you choose. You can choose to launch any instance type at any time. You are only charged while the instance is running.||High||High|
|Reserved||Reserved instances are cheaper than on-demand instances. This is because you commit to paying for a reserved instance for one or three years. Whether you use the instance or not, you are still liable for the costs.||Medium||Medium|
|Spot||Allow you to bid on instances that you want. For example, if you would usually pay $3 an hour for a specific EC2 instance, you could bid $1 per hour. If the price in the marketplace drops to your desired cost, you will be automatically provisioned the instance.
As soon as the cost rises above your bid again, you will immediately lose your instance. You cannot guarantee you will have the instance for any specified length of time.
Amazon Machine Image
Amazon Machine Images are pre-configured packages that provide the necessary information for an instance to launch. This information includes the operating system, preinstalled software packages (like NGinx web server), the EBS (HDD) mapping and any other settings that are defined in the AMI.
There are three types of AMI:
- Community AMI’s – these are free to use but are generally limited to just operating system choice
- Marketplace AMI’s – these are more complex and include the operating system and other supporting packages (often licensed software). These are AMI’s which you must pay to use
- My AMI’s – these are the AMI’s you create and save on your AWS account
AMI’s can be used with auto scaling to launch pre-configured instances to meet demand or for disaster recovery purposes.
AWS uses two types of virtualization:
- HVM – this runs directly on the virtual machine as if it were running directly on the underlying hardware. It can take advantage of hardware extensions like enhanced networking.
- PV – can run on hardware that does not have explicit support for virtualization. However, it cannot take advantage of hardware extensions. HVM is the preferred option of virtualization in the AWS environment.
|T2||General Purpose||Burstable Performance|
|M3||General Purposes||Nice Balance|
|C3/C4||Compute Optimized||High Traffic Web Servers|
|D2||Storage Optimized||Large Scale Data Warehouses|
|I2||Storage Optimized||Large Scale Data Warehouses|
|G2||GPU Optimized||Machine Learning / High Perf DB|
|P2||GPU Optimized||Machine Learning / High Perf DB|
|R3/R4||Memory Optimized||Databases / Large Enterprise Apps|
|X1||Memory Optimized||Databases / Large Enterprise Apps|
EBS (Elastic Back Store) Volumes
Elastic Block Store (EBS) is block level storage on EC2 instances. It’s Network Attached Storage (NAS) and can be moved between instances with ease. It’s highly available and reliable storage and can be attached to any running EC2 instance in the same Availability Zone and can only be attached to a single EC2 instance at any given time.
EBS volumes can persist independently from the life of the instance. To do this, you’ll need to uncheck the box that says ‘delete on termination’ during the creation of your EC2 instance.
The achievable IOPS & storage space on each varies across different storage types:
|General Purpose||Ideal for test environments or small database instances. These achieve 3 IOPS per GB of provisioned storage. These drives do have burstable performance.
For example, when you see 100/3000 IOPS when choosing your storage medium, this means you’ll have 100 IOPS, burstable to 3000 for short periods of time. Disk sizes are between 1GiB and 16GiB in size.
|Provisioned IOPS||Ideal for mission critical applications which require sustained performance. You can provision up to 20,000 IOPS. Disk sizes are between 4GiB and 16 GiB|
|Magnetic||Ideal for applications where performance is not at all important. These are extremely low cost but rarely utilized.|
Even volumes with provisioned IOPS may not provide the performance you expect. To achieve optimal performance, you should utilize EBS optimized instances which provide additional dedicated throughput for your EBS volume.
EC2 instances must have a root volume. This isn’t necessarily an EBS however, by default it is. You can add additional volumes to your EC2 instance and can move volumes between instances in the same availability zone.
You can use snapshots to backup or duplicate your EBS volumes. You can restore a backup by creating a new EBS volume, using the snapshot as a template. However, if you are to create an EBS volume from a snapshot, you must be aware that you will have a degraded performance (of up to 50%) until all the storage blocks on the EBS have been read – you can do this manually to achieve ‘normal’ performance levels.
Facts about snapshots;
- Snapshots are only for EBS volumes and do not apply to instance store volumes
- They’re incremental in nature – they only store the changes since the last snapshot
- If the ‘original’ snapshot is deleted, all data is still accessible, even though they’re incremental in nature
- Frequent snapshots increase durability and are therefore recommended for any environment
- Snapshots can degrade performance while they’re being taken. So if possible, these should be scheduled for non-peak hours
Instance Store Volumes
Unlike EBS, instance store volumes are physically attached to the host hardware running the instance. These drives hold ephemeral data – that means, the data on the drive exists for only the life of the instance. When the instance is stopped, shut down or terminated the data is erased. However, if the instance is rebooted, the data remains.
Security groups work on the instance level. They’re relatively similar to Network Access Control Lists in the sense that you can configure rules to allow certain types of traffic.
There are noticeable differences between a NACL and a security group:
- You can apply one or more security groups to an instance
- Security groups only support allow rules
- Security groups are stateful
- All rules are evaluated before a decision is made
Note, that all inbound traffic is denied and all outbound traffic is allowed by default.
What does it mean to be stateful? Well, let’s say you allow inbound SSH traffic to your instance and that traffic requires a response from your server but you do not have SSH allowed in the outbound rules of your security group – it will still allow you to send the response. This is not true with NACL’s.
The default security group has all inbound and outbound traffic allowed by default.
Note that all traffic is denied, unless there is an explicit allow rule.
An elastic IP is a static public IPv4 address.You can attach an elastic IP to an instance, even if the instance only has a private IP address. If an instance does have a public IP address, the elastic IP will replace its default public IP address.
Elastic IP’s can be used to mask the failure of an instance. You do this by removing the IP from the failed instance & quickly remapping it to another, working instance in your account.
Placement groups are AWS’ answer to a big cloud problem. Let’s say, you need 10 instances and they all need to have extremely quick connectivity to one another? If it were in your own data centre, you’d put them right next to one another.
AWS allows you to do the same. You can request that all your instances are put as physically close to one another as possible & you can utilize the low-latency 10Gbps link between those instances.
Now, all the instances you add to a placement group must be part of the same Availability Zone (AZ) and they MUST have enhanced networking (you can choose this at instance type selection).
- If an instance in a placement group is stopped, it will remain part of the placement group once it is restarted
- It’s suggested that you launch all your required instances into a placement group in a single request. This improves the chances of your machines being physically close to one another
- All instances in the placement group should ideally be the same instance type
- Instances that aren’t launched into a placement group can’t be added into one
- Placement groups cannot be merged together, cannot span multiple availability zones but can be connected
- Placement group names must be unique in your own AWS account
It is possible that if you add instances to your placement group at a later date, you’ll get an ‘insufficient capacity’ error. This can usually be resolved by stopping all instances in the group & starting them again.