EC2

EC2 -101

  1. EC2: Virtual Machines in AWS
  2. EC2 Instance Options:
    1. On demand: Fixed rate by hour, No Commitment is required. Use cases: Applications with short term or unpredictable workloads. Dev/Test instances
    2. Reserved: Fixed rate (Significant discount), Commitment required (1-3 years). Use cases: Applications with predictable workload, Users are able to pay upfront cost,
    3. Spot: Spot price is market rate decided by AWS (AWS) and Bid Price (Customer) you get instance when bid price >= spot price. You lose when bid price<spot price. If AWS terminates you are charged for full hours, but if you terminate you are charged for both full hours and partial hours. Use case: Applications that have flexible start and end times, Urgent need for large compute capacity.
    4. Dedicated: dedicated physical server. Pay by hour. You can reserve the instance and get discount.Use cases : Regulation/License requirements that does not support multi-tenant
  3. EC2 Instance Types
    1. Compute optimized
      1. C4 – CPU Intensive Apps/DBs
    2. Memory Optimized
      1. R4 – Memory Intensive Apps/DBs
      2. X1 – Extreme Memory – SAP Hana, Apache Spark
    3. Storage Optimized
      1. D2 – Dense Storage; File Servers/Data warehousing / Hadoop
      2. I2 – High Speed Storage; No SQL DBs, Data warehousing etc.
    4. Graphics
      1. G2 – Graphics Intensive – Video Encoding/3D Application streaming
      2. P2 – Graphics/General Purpose GPU – Machine Learning, Bit coin mining
    5. General Purpose
      1. M4 – Application Servers
      2. T2 –Low Cost – Webservers, Small DBs
    6. Field programmable Gate Array – Hardware acceleration for your code
      1. F1
  4. EBS Volumes: Disks attached to VMs/EC2 instances
  5. EBS Volume types
    1. SSD
      1. General Purpose (GP2)
        1. Balances both price and performance
        2. 3 IOPS/GB (Max -10000 IOPs) (example 10GB gets 30 IOPs, 20GB gets 60 IOPs)
        3. Ability to burst up to 3000 IOPs
      2. Provisioned IOPS (IO1)
        1. Designed for I/O intensive applications (ex: NoSQL)
        2. Use if your application required more than 10000 IOPS
        3. Upto 20000 IOPS per volume
    2. Magnetic Storage HDD
      1. Throughput Optimized (ST1)
        1. Data warehouse, Big data, Log Processing
        2. Can’t be boot volumes
      2. Cold HDD (SC1)
        1. Low cost storage for Infrequently access workloads
        2. File Server
        3. Can’t be boot volumes
      3. Magnetic Standard
        1. Workloads where data is accessed infrequently
        2. Can be boot volumes
  6. We can mount multiple EBS volumes to a EC2 instance, but a EBS volume cannot be mounted to multiple EC2 instances.
  7. Exam Tips
    1. EC2
      1. Know the differences between EC2 Instance Types
      2. Spot instances: If you terminate you pay for one hour for the partial hour. If AWS terminates – You will not be charged for partial hours
    2. EBS
      1. You cannot mount 1 EBS Volume to multiple EC2 instances, Use EFS instead
      2. SSD GP2 -Upto 10,000 IOPs
      3. SSD IO1 – More than 10,000 IOPs
      4. HDD ST1 – Frequently accessed workloads
      5. HDD SC1 – Less frequently accessed data
      6. HDD Magnetic Standard – Bootable and Cheap infrequently accessed storage

Launch EC2 Instance

  1. Choose Region – All EC2 Instance Types are not available in all regions
  2. HVM & PV – Supported Virtualization types
  3. Using Key pair, you can securely connect to EC2 Instances
  4. Chmod 400 privatekey.pem
  5. ssh to ec2 instance: ssh ec2-user@<InstancePublicIP> -i <PrivateKey.pem>
  6. Termination Protection: If terminate protection is set to true then the Instance cannot be terminated
  7. Status Checks: System Status checks underlying hypervisor, Instance status checks EC2 Instance
  8. Encrypt EBS Volumes: non-root volumes can be encrypted by checking Encrypted check box. Root volumes of standard AMIs cannot be encrypted. If you want root volumes to be encrypted create your own AMI. Steps -> deploy ec2 instance from standard AMI, create an AMI from the instance and while creating encrypt the root device volume.
  9. Exam Tips:
    1. Termination protection is turned off by default.
    2. On EBS backed instance, the default action for the root EBS volume to be deleted when the instance is terminated
    3. To encrypt root EBS volumes you can use either third party tool like bitlocker or create custom AMI
    4. Additional volumes can be encrypted.

 

Security Groups

  1. Security Group is a virtual firewall. It is first level of defense.
  2. Multiple security groups can be assigned to an EC2 Instance. Multiple EC2 Instances can be part of a Security group.
  3. You can specify ‘Allow’ rules not “Deny’ rules. All rules are ‘Allow’ rules.
  4. Specific IP addresses cannot be blocked using Security groups, use ACLs instead
  5. Security groups are stateful. Traffic which is allowed in will be allowed out.
  6. Changes to security group are affected immediately.
  7. By default, all inbound traffic is blocked.

Volumes and Snapshots

  1. Volumes are virtual disks.
  2. Volumes exist on EBS and Snapshots are stored in S3.
  3. To attach EBS volume to EC2 instance both should be in same availability zone.
  4. We can detach volume from EC2 instance.
  5. Snapshot is a point in time copies of a Volume (virtual disk). Snapshots are stored in S3. We can take multiple snapshots of a volume and snapshots are incremental.
  6. Volumes can be created from Snapshot. While creating volume from snapshot you can modify
    1. Volume Type
    2. Size
    3. Availability Zone
  7. Raid. When you are not getting desired disk I/O create multiple EBS volumes and create Raid. Typically Raid 0 or Raid 10
    1. Raid. When you are not getting desired disk I/O create multiple EBS volumes and create Raid. Typically Raid 0 or Raid 10
    2. Raid0 = Striped, No Redundancy, Good Performance
    3. Raid1 = Mirrored, Redundancy
    4. Raid5 = Distributed Parity. Good for reads.AWS does not recommend RAID5 for EBS Volumes.
    5. Raid10 = Good Performance. Striping & Mirroring
  8. Snapshot of Raid Array: Take an application consistent snapshot. When EBS Snapshot is taken, it excludes the data cached by OS/Application which will be a problem for Raid array
    1. Stop the application
    2. Flush all caches to disk
  9. Methods Available for application consistent snapshot
    1. Freeze the file system
    2. Unmount the Raid Array
    3. Shut down EC2 Instance and take snapshot

AMI

  1. AMI is a template used to launch EC2 Instance
  2. AMI Contains
    1. Template for root Volume
    2. Block device mapping to EBS Volumes
    3. Launch permissions (who can launch instances from the AMI)
  3. AMIs are stored in S3
  4. Create AMI for single volume EC2 Instance
    1. Create EBS snapshot of EC2 Instances root volume
    2. Create AMI from EBS Snapshot
    3. Modify Image permissions (optional)
  5. AMI Security Hardening is critical if you want to share the AMI
  6. Instance Store: Volume which is from hypervisor and is ephemeral.
    Instance stores can be added to EC2 only while launching instance.
  7. AMI types
    1. Instance store backed AMIs:
      1. Root device is backed from Instance store.
      2. EBS Volumes can be mounted
      3. More than one instance store can be added but only while launching an instance
      4. Ephemeral. Power actions available are Reboot & Terminate
      5. Reboot: Data will not be lost
      6. Terminate: Root volume is deleted
    2. EBS backed AMIs:
      1. Root device is backed from EBS Volumes
      2. Only EBS Volumes can be mounted to Instances
      3. Power actions available tare Start, Stop, Reboot& Terminate
      4. Stop or Reboot: Data will not be lost
      5. Terminate: By default, Root volume is deleted. However, an option available to keep the root device on Termination
  8. Exam Tips
    1. AMIs are regional. EC2 Instances can be launched only in AMI’s region. However, AMIs can be copied to other regions
    2. Instance store volumes are sometimes called Ephemeral Storage
    3. Instance store backed instances cannot be stopped, if underlying host fails, you lose your data
    4. EBS volume backs instances can be stopped. You will not lose the data if instance is stopped
    5. Both types of instances can be rebooted and data will not be lost when rebooted

Elastic Load Balancer and Autoscaling groups

  1. Types of ELB
    1. Application load balancer
      1. Supports VPC
      2. Preferred for http/https
      3. Layer 7 load balancer
    2. Classic load balancer
      1. Supports EC2 Classic and VPC
      2. Routing decisions on http/https
      3. Layer 4 load balancer
  2. When ELB s created, it will have both IP and DNS. However, AWS manages the public IP and is not given to Customer. Customer should use ELB DNS. ELB DNS will be resolved to member EC2 Instances’ public IP
  3. Instances monitored by ELB are reported as ‘Inservice’ or ‘OutofService’
  4. Health checks check the instance health by talking to it.
  5. Launch Template: Before creating an auto scale group we must create launch configuration. Launch configuration is a template that define how to launch an instance.

    1. AMI
    2. Instance Type
    3. Configure details: IAM Role, Monitoring, Boot strap script, IP Address type, etc.
    4. Storage
    5. Security group
  6. Auto Scaling Group:
    An Auto Scaling group contains a collection of EC2 instances that share similar characteristics and are treated as a logical grouping for the purposes of instance scaling and management. For example, if a single application operates across multiple instances, you might want to increase the number of instances in that group to improve the performance of the application, or decrease the number of instances to reduce costs when demand is low.

    http://docs.aws.amazon.com/autoscaling/latest/userguide/AutoScalingGroup.html

  7. Create Auto scale group:
    1. Name
    2. Launch configuration
    3. Group Size –# of instances to start with
    4. Network(VPC)
    5. Subnets
    6. Receive traffic from Load balancer
    7. Health check,
    8. Scaling Policies: Keep this group at its initial size or Dynamically scale
    9. Notifications
    10. Tags
  8. Dynamically scaling policy – a) Min and Max # of Instances b) Policy to Increase group size in response to alarm c) Policy to decrease group size in response to alarm
  9. If instance is failed / scaling policy situation arises Autoscaling group will automatically replace instances by launching new instances

EC2 Placement Groups

  1. A placement group is a logical grouping of instances within a single availability zone. Placement groups enable applications to participate in a low-level latency 10Gbps network. Applications like Hadoop cluster.
  2. Placement groups can’t span multiple availability zones
  3. Placement group names must be unique within AWS account
  4. Only certain type of EC2 instances can be launched in a placement group (Compute Optimized, GPU, Memory and Storage Optimized)
  5. (AWS recommendation) Launch homogeneous instances (same sizes and same families) in a placement group
  6. Placement groups cannot be merged
  7. Existing instances cannot be moved to a placement group. If you want to move an existing instance, create an AMI from it and launch new instance from the AMI in placement group

Elastic File System

  1. EFS is a file storage service for EC2 instances
  2. Support NFS v4 protocol
  3. EFS storage capacity is elastic.
  4. Can scale up to petabytes
  5. You pay only for storage you used.
  6. Supports thousands of concurrent NFS connections
  7. Data is stored across multiple availability zones within a region
  8. Read after write consistency

Lambda

  1. Upload you code and create lambda function -> AWS lambda takes care of provisioning and managing the servers that you use to run the code. Need not worry about OS, patching,scaling etc.
  2. AWS Lambda encapsulates Datacenters, Hardware, Assembly code/protocols, High level languages, Application layer/AWS APIs
  3. Available Languages -> Java, node.js, Python, C#
  4. Pricing
    1. Number of requests
    2. Duration
  5. Lambda is server less

CloudWatch

  1. EC2 Instance monitoring tab: Basic Monitoring and Advanced Monitoring
  2. Dashboard has four types of widgets
    1. Text
    2. Number
    3. Line
    4. Stacked Area
  3. Alarms
    1. Metric (e.g. CPU Utilization)
    2. Threshold (e.g. CPU utilization >80% for 1 minute)
    3. Action (Notification, Autoscaling group action, EC2 action)
  4. Events: response to change in the state of resources. (e.g. When EC2 status change to running then invoke lambda function to update DNS)
  5. Logs: monitor, aggregate and store logs
  6. Log groups. Install the CloudWatch agent in EC2 instance, create metric filters and access logs
  7. Exam Tips:
    1. Standard monitoring – 5 mins, Detailed monitoring – 1 min
    2. CloudWatch: Dashboards, Alarms, Events & Logs
    3. Difference between CloudWatch and CloudTrail: CloudWatch for monitoring (e.g. CPU utilization ) and CloudTrail is for Auditing (e.g. create new user, create new role etc.)

AWS CLI

  1. AWS Credentials: SSH to EC2 instance and to use AWS CLI configure credentials by ‘aws configure’ (access key , secret access key , default region). But the downside is both credentials are stored locally in the instance which is a security risk.
  2. IAM User Role: Create an IAM role with necessary privileges and attach to EC2 instance. (Now: we can attach IAM role to existing EC2 instance using CLI. Earlier: IAM role can be attached to instance only while launching instance). Attaching roles are more secure than storing credentials in instance. Roles are easier to attach to manage.
  3. Roles are universal

Bash Scripting & Instance Metadata

  1. Bash script is list of AWS commands run when EC2 instance first starts up. (Use case: Bootstrap chef/puppet agent, run scripts to install software packages etc).
  2. Bash script can be provided as text/file in user data while launching EC2 instance.
  3. Access the metadata in ec2 instance itself @ http://169.254.169.254/latest/meta-data (public IP of EC2 instance, hostname, etc). 169.254.169.254 is a link local address.
S3
VPC
No tags for this post.

Leave a Comment