S3 Bucket Level Operations

Bucket Level Operations

Here are the list of some bucket level operations with their explanation.

a) Versioning

Means keeping multiple variants of an object in the same bucket. Once enabled for a bucket. We can't disable it. Also Even after enabling it will take effect only for new objects. This option can be used to protect from unintended overwrites and deletions and allows us the ability to retrieve and restore deleted objects or rollback to previous versions

b) Multipart Delete

To be used to delete large number of objects from S3. The largest object that can be uploaded in a single PUT is 5 gigabytes. For objects larger than 100 megabytes, customers should consider using the Multipart Upload capability.

Note : Both empty and Non-empty buckets can be deleted

c) Encryption

(i)  Data In transit – By using SSL

(ii) Data at rest     – Client side encryption - Encrypt and Upload

                            – Server side encryption ( Req AWS to encrypt before saving)

o   with Amazon s3 managed keys (SSE –S3)

o   with KMS managed keys (SSE –KMS)

o   with customer provided keys (SSE – C)

d) Cross Region replication

This is a Bucket level feature which enables automatic asynchornous copying of objects of one bucket to a another bucket in different AWS region.

To activate Add replication configuration to your source bucket and provide info of destination bucket.

We can request Amazon S3 to replicate all (or) a subset of object with specific key name prefix. Replica will have same key name, metadata, creation time, owner, version, ACL and storage class.

Requirements for cross region replication -

           Version must be enabled

           Different region

           Between Different account is also possible (i.e) Cross account

When using VPC with S3, use VPC S3 endpoints as are horizontally scaled, redundant, and highly available VPC components

e) Server access logging

In Order to track requests fir access to your bucket, you can enable this option. Each request has "requestor, bucket name, time, status and error code:

There is no extra charge for enabling server access logging on an Amazon S3 bucket; however, any log files the system delivers to you will accrue the usual charges for storage.

(You can delete the log files at any time.) No data transfer charges will be assessed for log file delivery, but access to the delivered log files is charged the same as any other data transfer.

What is VPC

1) What is VPC

VPC, Virtual private cloud. if you choose tenancy as dedicated then you will be able to launch only Dedicated instances or dedicated hosts

2) Creating VPC

While creating VPC we may see below options and need to select the values.A short information about the options

(i) Tenancy - While creating VPC if you choose tenancy as dedicated then you will be able to launch only Dedicated instances or dedicated hosts

(ii) CIDR block - It represents range of IP Address Eg; 10.0.0.0/26 Start will be 10.0.0.0 and rang will be [ 2 ^ (32 - 26)  = 2^6 = 64 ] and so the end will be 10.0.0.63

(iii) Subnet - It represents range of IP Address. Amazon reserves first 4 and last 1 IP of every subnet. Default subnet within default VPC are assigned /20 netblocks.

3) VPC Resources & Settings

(i) NACL - Network Access controlled List. - Subnet level Traffic filtering .

(ii)  Internet Gateway: VPC side of a connection to the public Internet. 1 Per VPC

(iii) NAT Gateway: A highly available, managed Network Address Translation (NAT) service for your resources in a private subnet to access the Internet.

(a) Modify auto assign IP Setting - If true, enables auto assign public for Instances. Enable auto-assign public IPv4 or IPv6 addresses to automatically request an IP address for instances launched into this subnet.

3) Resoure Location

VPC - Specific to Region. We can't mention AZ.

Subnet - Specific to AZ

4) VPC Settings

Edit DNS resolution -

Edit DNS host name - If checked, instances will get private hostname.

CIDR block -

VPC (Virtual Private Cloud) - Logically isolated virtual network in the AWS cloud. You can define VPC's IP address space from a range you select

VPC wit single public subnet -

Using an Elastic IP address (EIP) enables an instance in a VPC, which is otherwise private, to be reached from the Internet through an Internet gateway (for example, it could act as a web server).

4) VPC options

(i)   VPC wit single public subnet

(ii)  VPC with public/private subnet

(iii) VPC With public/private subnet and Hardware VPN access

(iv) VPC with private subnet only and Hardware VPN access

5) Components of Amazon VPC?

Amazon VPC comprises a variety of objects that will be familiar to customers with existing networks:

Hardware VPN Connection: A hardware-based VPN connection between your Amazon VPC and your datacenter, home network, or co-location facility.

Virtual Private Gateway: The Amazon VPC side of a VPN connection.

Customer Gateway: Your side of a VPN connection.

Router: Routers interconnect subnets and direct traffic between Internet gateways, virtual private gateways, NAT gateways, and subnets.

Peering Connection: A peering connection enables you to route traffic via private IP addresses between two peered VPCs.

VPC Endpoint: Enables Amazon S3 and Amazon DynamoDB access from within your VPC without using an Internet gateway or NAT, and allows you to control the access using VPC endpoint policies.

Egress-only Internet Gateway: A stateful gateway to provide egress only access for IPv6 traffic from the VPC to the Internet

6) Charge – No Charge for VPC & VPC peering

7) VPC Peering – Is between availability zone’s only, Peering must non-overlapping IP range.

8) Limits

VPC           – 5 per region

Subnet       – 200 per VPC

IG               – 1 per VPC

Virtual private gateway – 5 per region

Customer Gateway       – 50 per region

9) Security group

Default security group config – Allow All outbound and No inbound

10) General Information's

1) Default VPC are assigned range of 172.31.0.0/16 in size

2) Currently AWS supports VPC between /28(In CIDR) & /16 in size.

3) We can't change the size of VPC. You must terminate and create new one

4) You can't assign IP address for multiple instances simultaneously

5) We cant dtach eth0

11) VPC Peering 

Connecting of two non CIDR overlapping VPC's. Peering is available between same region. Peering between different AWS account is possible.

Conditions/Restrictions

(i) No Tranisitive Peering (Eg:Let us there are 3 VPC's A,B,C. A is peered between B & C that does'nt mean that VPC C can connect B.

(ii) No Edge routing

(iii) No NAT routing

About Peering

(i)  We need to update route tables of the VPC after Peering. Like for the CIDR block of VPC 2 and entry in VPC 1 route table need to be created with the destination as Peer name.

(ii)  We cannot use the Security group of Other VPC even it is peered

(iii) Windows instances cannot boot correctly if launched into a VPC with ranges from 224.0.0.0 to 255.255.255.255 (Class D and Class E IP address rang

Difference between Public IP and Elastic IP Address

Public IP and Elastic IP Address

a) Public IP address 

Instance with Public IP address will get an external DNS name (example, ec2-506-0-198-88.compute-1.amazonaws.com)  name. External DNS name will be able to resolve outside the network and to the private IPv4 address of the instance from within the network of the instance. The public IP address is mapped to the primary private IP address through network address translation (NAT).

Public IP address can't be manually associated or disassociated for your instance. AWS releases or assigns the public IP address  in following cases

1) When instance is  stopped or terminated. Stopped instance receives a new public IP

2) When you associate an Elastic IP address with your instance Public IP address for your instance will be released.

3) When you associate an Elastic IP address with the primary network interface (eth0) of your instance in a VPC.

4) When you disassociate the Elastic IP address from your instance, it receives a new public IP address.

Reboot of EC2 WON'T change the public IP address of your instance changes

If you require a persistent public IP address that can be associated to and from instances as use an Elastic IP address.

b) Elastic IP address 

An Elastic IP address is a static IPv4 address designed for dynamic cloud computing. Elastic IP address will be associated with your AWS account.

Elastic IP address doesn’t incur charges as long as the below conditions are met

1)  The instance associated with the Elastic IP address is running.

If above conditions are not met then you will be charged for the Elastic IP Address.

If you’ve stopped or terminated an EC2 instance with an associated Elastic IP address and you don’t need that Elastic IP address any more, consider disassociating or releasing the Elastic IP address.

Note: Once if the Elastic IP address is released, you can’t get the same Elastic IP address again, you have to get a different Elastic IP address.

EC2 Tenancy Models

Here are different types of tenancy model available which we need to select while creating the EC2. Below are the list of models with explanation

a) Shared Tenancy 

The physical Hardware where your EC2 is hosted will be shared with other customers also. Pricing is less as the Hardware is shared with other customer's. When the instance is Stopped and Started the underline physical server will change but  Reboot will not change the underline hardware.

b) Dedicated Instances 

The physical hardware is completely dedicated for you and won't be shared with other customer and only your EC2 instances will be hosted there. In Simple terms, Hardware dedicated for your account. But during start and stop of the server the underline Hardware will change and we may get new dedicated hardware. And your shared instances (same account) wont be launched in the underlying hardware only dedicated instances of particular type will be launched in the same hardware.

For example: Requesting a m4.large dedicated instance could end with your instance on a server exactly the size of an m4.large.

In addition to instance charge of per hour rate an additional of 2$ will be charged per hour per region. This 2$ will be same if it is 10 instance or 1 instance. As it will be charged per hour per region

c) Dedicated Hosts 

Only difference between Dedicated instance and Dedicated host is even after stop & start underlying hardware will not change. This method is useful when we are going for a hardware based license model. Billing will be high as we are using full host.

As we get the complete hosts, irrespective of utilization we will pay the complete charge as the host is dedicated for You. This is bit costlier than others.  

Number of instances per dedicated hosts calculation is available in AWS help page. For example if we select m4 as our dedicated hosts then we can have 22 large m4 or 11 xlarge