AWS Disaster Recovery

AWS Disaster recovery methods

a) Backup and Restore

b) Pilot light

c) Warm Standby

d) Multisite

Backup and Restore

Data is backed up to tape and sent to off site regularly

Pilot light

Minimal version of the service is always running in the cloud.

Quicker recovery than backup and restore method

Warm Standby

Scaled down version of a fully functional environment

Multisite

Active Active Configuration

Linux AMI Virtualization types

Linux Amazon Machine Images use one of two types of virtualization:

a) Paravirtual (PV)

b) Hardware virtual machine (HVM).

The main difference between PV and HVM AMIs is the way in which they boot and whether they can take advantage of special hardware extensions (CPU, network, and storage) for better performance.

For the best performance, we recommend that you use current generation instance types and HVM AMIs when you launch your instances.

HVM AMIs

HVM AMIs are presented with a fully virtualized set of hardware and boot by executing the master boot record of the root block device of your image. This virtualization type provides the ability to run an operating system directly on top of a virtual machine without any modification, as if it were run on the bare-metal hardware. 

HVM guests can take advantage of hardware extensions that provide fast access to the underlying hardware on the host system. HVM AMIs can take advantage of enhanced networking and GPU processing. (In order to pass through instructions to specialized network and GPU devices, the OS needs to be able to have access to the native hardware platform; HVM virtualization provides this access)

All current generation instance types support HVM AMIs. The CC2, CR1, HI1, and HS1 previous generation instance types support HVM AMIs.

To find an HVM AMI, verify that the virtualization type of the AMI is set to HVM, using the console or the describe-images command.

PV AMIs

PV AMIs boot with a special boot loader called PV-GRUB, which starts the boot cycle and then chain loads the kernel specified in the menu.lst (menu.lst -> ./grub.conf) file on your image. Paravirtual hosts cannot take advantage of special hardware extensions such as enhanced networking or GPU processing. 

The C3 and M3 current generation instance types support PV AMIs. The C1, HI1, HS1, M1, M2, and T1 previous generation instance types support PV AMIs.

To find a PV AMI, verify that the virtualization type of the AMI is set to paravirtual, using the console or the describe-images command.

PV on HVM

Paravirtual guests traditionally performed better with storage and network operations than HVM guests because they could leverage special drivers for I/O that avoided the overhead of emulating network and disk hardware, whereas HVM guests had to translate these instructions to emulated hardware. 

Now these PV drivers are available for HVM guests, so operating systems that cannot be ported to run in a paravirtualized environment (such as Windows) can still see performance advantages in storage and network I/O by using them. With these PV on HVM drivers, HVM guests can get the same, or better, performance than paravirtual guests.

AMI Launch Permissions

Public - Launch permission to all

Explicit - Launch permission to special AWS accounts

Implicit - Only owner can launch

How are you charged AMI Launch Permissions

With Instance store AMI every time you customize  and create new one all the parts are stored in Amazon S3 for each AMI. So storage footprint for each customized AMI is full size of AMI.

For EBS backed AMI each time you customize and create new one only the changes are stored  for subsequent AMI. resulting in low storage.

We can convert instance store AMI to EBS backed AMI but lot of steps need to be followed.

Copying of AMI is not chargeable.

launch pemrission, user defined tags or s3 permission or not copied from source AMI  o new AMI.

Making AMI public will take some time and available only to that region.

You can create boot strap script to  customize your AMI  to install software on startup

Difference Between NACL and Security Group

Network Access control lists are applicable at the subnet level, so any instance in the subnet with an associated NACL will follow rules of NACL. That's not the case with security groups, security groups has to be assigned explicitly to the instance.

By default your default vpc, will have a default Network Access Control List which would allow all traffic , both inbound and outbound. If you want to restrict access at subnet level it's a good practice to create a custom NACL and edit it with the rules of your choice and while editing you could also edit subnet associations , where you associate a subnet with this custom NACL, now any instance in this subnet will start following NACL rules

NACLs are stateless unlike security groups. Security groups are statefull ,if you add an inbound rule say for port 80, it is automatically allowed out, meaning outbound rule for that particular port need not be explicitly added. But in NACLs you need to provide explicit inbound and outbound rules

In security groups you cannot deny traffic from a particular instance, by default everything is denied. You can set rules only to allow. Where as in NACLs you can set rules both to deny and allow. There by denying and allowing only the instances of your choice. You get to take more granular decisions.

Security groups evaluate all the rules in them before allowing a traffic . NACLs do it in the number order, from top to bottom like, if your rule #0 says allow all HTTP traffic and your rule #100 says don't allow HTTP traffic from ip address 10.0.2.156 , it's not gonna work, because rule #0 has already allowed traffic. So it's good practice to have your deny rules first in NACL and followed by allow rules. AWS best practice is to number your rules in increment of 100s in NACL.

Accessing internet from Private Subnet

To Access internet from Private subnet. We have two methods

a)            By using NAT Gateway

b)          By Using NAT Instance

By Using NAT Instance

1) Launch the NAT Instance using ( AMI to be used : Amazon NAT AMI ) in public Subnet

2) Launch the instance in private subnet

3) Assign EIP to the NAT Instance

4) Disable Source/Destination Check for the instance (Select Instance -> Actions -> Networking -> Uncheck Source/Destination check )

5) Update the IPTables of NAT table like below (i.e) POSTROUTING to MASQUERADE

# iptables -t nat -A POSTROUTING -o eth0 -s <privae subnet>  -j MASQUERADE

Modify the Security Group of NAT instance inbound rule to all or the port from PVT subnet

Modify the Route table of the PVT subnet for 0.0.0.0/0 to point to NAT Instance

cat <<EOF | sudo tee /etc/sysctl.d/custom_nat_tuning.conf

# for large instance types, allow keeping track of more

# connections (requires enough RAM)

net.ipv4.netfilter.ip_conntrack_max=262144

EOF

sudo sysctl -p /etc/sysctl.d/custom_nat_tuning.con

/usr/local/sbin/configure-pat.sh

By Using NAT Gateway

1) Launch the NAT Gateway in public Subnet. 

2) Launch the instance in private subnet

3) Assign EIP to the NAT Gateway

4) Modify the Route table of the PVT subnet for 0.0.0.0/0 to point to NAT Gateway

Displayed error	Reason	Remedial steps
Subnet has insufficient free addresses to create this NAT gateway	The subnet you specified does not have any free private IP addresses. The NAT gateway requires a network interface with a private IP address allocated from the subnet's range.	You can check how many IP addresses are available in your subnet by going to the Subnets page in the Amazon VPC console, and viewing the Available IPs field in the details pane for your subnet. To create free IP addresses in your subnet, you can delete unused network interfaces, or terminate instances that you do not require.
Network vpc-xxxxxxxx has no Internet gateway attached	A NAT gateway must be created in a VPC with an Internet gateway.	Create and attach an Internet gateway to your VPC. For more information, see Attaching an Internet Gateway.
Elastic IP address eipalloc-xxxxxxxxcould not be associated with this NAT gateway	The Elastic IP address that you specified does not exist or could not be found.	Check the allocation ID of the Elastic IP address to ensure that you entered it correctly. Ensure that you have specified an Elastic IP address that's in the same region in which you're creating the NAT gateway.
Elastic IP address eipalloc-xxxxxxxxis already associated	The Elastic IP address that you specified is already associated with another resource, and cannot be associated with the NAT gateway.	You can check which resource is associated with the Elastic IP address by going to the Elastic IPs page in the Amazon VPC console, and viewing the values specified for the instance ID or network interface ID. If you do not require the Elastic IP address for that resource, you can disassociate it. Alternatively, allocate a new Elastic IP address to your account. For more information, see Working with Elastic IP Addresses.
Network interface eni-xxxxxxxx, created and used internally by this NAT gateway is in an invalid state. Please try again.	There was a problem creating or using the network interface for the NAT gateway.	You cannot fix this error. Try creating a NAT gateway again.

Comparison of NAT Instances and NAT Gateways

The following is a high-level summary of the differences between NAT instances and NAT gateways.

Attribute	NAT gateway	NAT instance
Availability	Highly available. NAT gateways in each Availability Zone are implemented with redundancy. Create a NAT gateway in each Availability Zone to ensure zone-independent architecture.	Use a script to manage failover between instances.
Bandwidth	Supports bursts of up to 10Gbps.	Depends on the bandwidth of the instance type.
Maintenance	Managed by AWS.You do not need to perform any maintenance.	Managed by you, for example, by installing software updates or operating system patches on the instance.
Performance	Software is optimized for handling NAT traffic.	A generic Amazon Linux AMI that's configured to perform NAT.
Cost	Charged depending on the number of NAT gateways you use, duration of usage, and amount of data that you send through the NAT gateways.	Charged depending on the number of NAT instances that you use, duration of usage, and instance type and size.
Type and size	Uniform offering; you don’t need to decide on the type or size.	Choose a suitable instance type and size, according to your predicted workload.
Public IP addresses	Choose the Elastic IP address to associate with a NAT gateway at creation.	Use an Elastic IP address or a public IP address with a NAT instance. You can change the public IP address at any time by associating a new Elastic IP address with the instance.
Private IP addresses	Automatically selected from the subnet's IP address range when you create the gateway.	Assign a specific private IP address from the subnet's IP address range when you launch the instance.
Security groups	Cannot be associated with a NAT gateway. You can associate security groups with your resources behind the NAT gateway to control inbound and outbound traffic.	Associate with your NAT instance and the resources behind your NAT instance to control inbound and outbound traffic.
Network ACLs	Use a network ACL to control the traffic to and from the subnet in which your NAT gateway resides.	Use a network ACL to control the traffic to and from the subnet in which your NAT instance resides.
Flow logs	Use flow logs to capture the traffic.	Use flow logs to capture the traffic.
Port forwarding	Not supported.	Manually customize the configuration to support port forwarding.
Bastion servers	Not supported.	Use as a bastion server.
Traffic metrics	Not supported.	View CloudWatch metrics.
Timeout behavior	When a connection times out, a NAT gateway returns an RST packet to any resources behind the NAT gateway that attempt to continue the connection (it does not send a FIN packet).	When a connection times out, a NAT instance sends a FIN packet to resources behind the NAT instance to close the connection.
IP fragmentation	Supports forwarding of IP fragmented packets for the UDP protocol. Does not support fragmentation for the TCP and ICMP protocols. Fragmented packets for these protocols will get dropped.	Supports reassembly of IP fragmented packets for the UDP, TCP, and ICMP protocols.

Scenario Based Questions

1. How to migrate an EC2 from one region to anohter

a) take snapshot of the ec2 by right click and create image option

b) copy the AMI to different region

c) then from the new AMI create an EC2

2) How to change the users secret access key

Secret access key cannot be changed. New key need to be generate and assign the user with policy IAMUserSSHKeys. So that next user can manage his own ssh keys.

3) Allow IAM users to change their own passwords

In IAM account Settings, enable Allow users to change their own password, and then click Apply Password Policy.

4) I accidentally deleted my default VPC in AWS. What to do now?

You can open a support ticket and they will re create it for you.

5) In S3

If there is no Edit in the file in S3 and if you upload another file of same name, it is overwritten if versioning is not enabled. A new version is created if version is maintained.

b) Within the Region data is replicated into multiple availablity zone's.

c) Ownership of the object is not transferable.

d) Version ID is the version id for the object and in combination with the key helps to unique

e) Subresources helps provide additional information for an object

Elastic Load Balancing

ELB - Elastic Load Balancer

It is a managed service by AWS (Scalability and Availablity is managed by AWS). Mainly it used for distributing traffic between instances. Once if the instance is registered with ELB, it performs continuous Health check. Once if it finds the unhealthy instance ELB will stop sending the traffic.

ELB can either be public or private it is based on the subnet where we are launching it. It is highly recommended to use 2 AZ's for high availablity. 

ELB creates instances in the subnet's in backend which we cannot see. Private IP's for ELB will be assigned from the subnet which we have selected for the ELB. ELB will expand based on the traffic. Traffic will be distributed to Availablity zone's in round robin manner.

Note: ELB can be accessed only by DNS CNAME and not by IP.

Cross zone load balancing 

It will be useful if we are using multiple zones and if the number of instances in the zones are not equal. For example, If we have 6 Instances in 2 AZ's and in Zone A 4 instance and Zone B 2 instance. ELB will blindly forward the requests in Round robin manner to Zone A & B which causes the instances in Zone B to handle more load compared to the instances in Zone A. To Over com this we can enable cross zone load balancing option.

If cross Load Balancing is enabled then ELB will take the count of all the instances in all the availablity zone's  and will forward the traffic to instance which has least number of connections.

Pre - Warming

If you expect a sudden increase in your hits. For eg: from 1 to 10,000 hits in particular time then it will take some time for ELB to auto scale in backend. To avoid this get your ELB pre warmed by contacting AWS support. They will configure the load balancer to have the appropriate level of capacity based on the traffic that you expect.  

They also need some more details like Start and End dates, Expected flash traffic, Expected request rate per second and the total size of the typical request/response.

Connection Draining 

If enabled, time you give to the instances to complete the already establieshed connection for the unhealthy instances. But it wont send any new traffic to those unhelathy instance.

Internal load balancer - Load balancer launched in the private network.

If an instance is using ELB that doesnt mean the EC2 can download files from internet.