Saturday, January 30, 2016

Amazon AWS fundamentals

I will be sharing the basic concepts around Amazon web service. I believe this will be helpful for the new users who are usually overwhelmed by the array of service choices offered in AWS. We are discussing basic understanding in AWS. Detailed information can be found on their help content.


EBS ( Elastic Block Storage):

Disks used for Virtual machines in the AWS cloud. Typical servers has computing power, storage and RAM.
However, in AWS compute and storage are separate.
We need to connect EBS(Storage) to EC2(Compute). We may have many EBS storage in your AWS account. Only, when you connect your EBS to the running instance makes a proper virtual machine.

Disk Performance:

Another key concept in AWS storage is performance of a disk. It is measured in Input Output per second (IOPS)
There are three types of EBS:
  • General purpose SSD ( 3 IOPS;)
  • Provisioned IOPS ( 40 to 200 IOPS)
  • Magnetic (4000 IOPS)
A database or high performance required application has to be hosted in Magnetic EBS volume which delivers better read write speed.


A snapshot can be used as a backup of the EBS volume of an instance. This can be an incremental backup of the data stored in EBS. This can be used to provision a cloned instance.

S3 ( Simple storage service):

S3 is also related to storage in AWS world. However, It is not equivalent to EBS in terms of storage. It allows developer to store objects in the external storage. Static files such as media or a static web site. We have used to store user’s profile picture in S3 and display those pictures in a direct link.
S3 uses bucket concept to store object. Each bucket can be accessible using unique URL. These stored objects can be versioned. Access to these objects is private by default. It can manage complex access control.


Security Groups:

Security group is equivalent to configuring the firewall in the server. Security group can be applied to as many as instance as possible. An instance can have many security groups. System admins keep security group per need and they apply according to the needs required. For example, Web server has to serve the content on port 80. So port 80 is opened in the Webserver-security group.
  1. Inbound : E.g for allowing SSH to the server
  2. Outbound : E.g. for exposing the database port to the outside world
When working with changes in security group, it is instant.

Elastic IPs

In AWS, by default dynamic ip address is assigned. For web server hosting and running website, static ip address is required. This can be solved by Elastic IP. Elastic IP can be attached to the running instance and we can map this static IP in MX record in DNS.

How to add volume in the EC2 instance?

  1. Create add volume under EC2 console.
  2. Attach the volume to the running instances.
  3. SSH to the running instance
  4. type dmsg command to see the unknown partition being added.
  5. [584679.841428] blkfront: xvdf: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled;
    [584679.845533]  xvdf: unknown partition table
    This means that partition is available for EC2 instance.
  6. sudo fdisk /dev/xvdf
  7. Type ‘p’ to see that there is no partition. ( just to confirm)
    Empty table will be shown if it is empty.
    Command (m for help): n
    Partition type:
       p   primary (0 primary, 0 extended, 4 free)
       e   extended
    Select (default p): p
    Partition number (1-4, default 1): 2
    First sector (2048-4194303, default 2048):
    Using default value 2048
    Last sector, +sectors or +size{K,M,G} (2048-4194303, default 4194303):
    Using default value 4194303
    Command (m for help): p
    Disk /dev/xvdf: 2147 MB, 2147483648 bytes
    255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x213ea459
    Device Boot      Start         End      Blocks   Id  System
    /dev/xvdf2            2048     4194303     2096128   83  Linux
    Command (m for help): w
    The partition table has been altered!

  8. Next step is to create the file system on it.
  9.  sudo mkfs -t ext4 /dev/xvdf2 
    Here I am passing the -t option for filesystem and my partition
    mke2fs 1.42.9 (4-Feb-2014)
    Filesystem label=
    OS type: Linux
    Block size=4096 (log=2)
    Fragment size=4096 (log=2)
    Stride=0 blocks, Stripe width=0 blocks
    131072 inodes, 524032 blocks
    26201 blocks (5.00%) reserved for the super user
    First data block=0
    Maximum filesystem blocks=536870912
    16 block groups
    32768 blocks per group, 32768 fragments per group
    8192 inodes per group
    Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912
    Allocating group tables: done
    Writing inode tables: done
    Creating journal (8192 blocks): done
    Writing superblocks and filesystem accounting information: done
  10. Mounting the file system. <code> sudo mkdir /ebsvolume
  11. sudo mount /dev/xvdf2 /ebs_volume
  12. Are we done? No. We need to mount this for every boot.
  13. sudo /etc/fstab 
    Add the following line in the above file to make the mounting at every boot.
    /dev/xvdf2 /ebs_volume  ext4 defaults,discard 0 0
  14. sudo reboot
  15. Login in to machine again and enjoy cd’ing in to your volume.

How to add swap space in AWS instances?

Follow 1-5 steps in how to add disk in AWS instances.
  1.  sudo swap /dev/xvdg1
  2. Run free command to see the available memory on the server. free
  3. total       used       free     shared    buffers     cached
    Mem:       1016324     171828     844496        372       9712      88128
    -/+ buffers/cache:      73988     942336
    Swap:            0          0          0
    This shows swap is 0 and yet to add.
  4. Activate the swap with following command. <code> sudo swap on /dev/xvdg1</code>
  5. Edit the /etc/fstab file to make this swap available on every boot. Add the following line.
  6. /dev/xvdg1 swap  swap defaults 0 0
  7. Reboot and Run the free command again to see the swap space activated. Happy swapping!

How to change the timezone in the AWS instances?

  1. All the timezone related information are available in the following directory.
    cd /usr/share/zoneinfo/Greenwich 
  2. Edit the system config to change the timezone.
    sudo vi /etc/sysconfig/clock
  3. ZONE=“Greenwich”
  4. ls -sf /usr/share/zoneinfo/Greenwich /etc/localtime

No comments:

Post a Comment