Storage Options and Use cases

  1. S3
    • Ideal Usage
      • Storage and distribution of static web content and media ( each object in S3 has unique http url address)
      • Host entire static website
      • Datastore for computing and large-scale analytics
      • Backup & Archival of critical data (frequent access) and to provide DR solution for BC solutions . Note: for infrequent access data use glacier
    • Anti-Usage
      • File System ( Install OS , DB etc)
      • Structured data with query ( Database)
      • Rapidly changing data ( EBS)
      • Backup and archival of infrequent access data ( use Glacier)
      • Dynamic website hosting (EC2)
    • Types
      • S3 : 99.999999999(11 9s) Durability, 99.99 Availability
      • S3-IA : 99.999999999(11 9s) Durability, 99.99 Availability. Lower fee. charged for retrieving data
      • S3-RRS : 99.99 Durability, 99.99 Availability. Reproducible data
  2. Glacier
    • Ideal
      • Archive offsite enterprise information (infrequent access data)
      • Magnetic tape replacement
    • Anti
      • Rapidly changing data
      • Real time access
    • 99.999999999(11 9s) Durability
  3. EBS Volumes
    • Ideal
      • Data that changes relatively frequently and long term persistence
      • Install database, OS etc. that require access to raw block-level storage
    • Anti
      • Temporary Storage (local instance store volumes,SQS, Elastic cache)
      • Highly durable storage.
      • Static data or web content(S3)
    • Types of volumes
      • SSD
        1. General Purpose SSD (gp2) – General purpose workloads. Boot volumes, virtual desktops, low latency interactive apps, Dev and Test environments
        2. Provisioned IOPS SSD (io1) – mission critical applications, large database workloads.
      • HDD ( can’t be boot volumes)
        1. Throughput Optimized HDD (st1)  – Streaming workloads, Big data , Data warehousing , Log processing
        2. Cold HDD (sc1) – Large volumes with infrequent access data , lowest storage cost is important
  4. EC2 Instance store volumes
    • Ideal
      • Temporary storage of information that changes continually –buffers, scratch data , temporary data that is replicated across a group of instances
    • Anti
      • Persistent storage (EBS,S3)
      • Relational database storage (EBS)
      • Shared storage (S3,EBS)
      • Snapshots (EBS)
  5. AWS Import/Export
    • Ideal
      • Initial data upload to AWS
      • Transfer data to S3 or Glacier
    • Anti
      • Data that can be transferred over internet within a week
  6. AWS Storage Gateway
    • Ideal
      • Corporate file sharing
      • On-prem backup applications to store primary backups on S3
      • DR
      • Data mirroring to AWS resources
    • Anti
      • Database storage
    • Options
      • File gateway: Provides onprem network file share. File data is cached in gateway. Gateway converts file data to S3 objects
      • Volume gateway – cached volumes : Store data in S3 and retain a copy of frequently accessed data locally
      • Volume gateway – stored volumes: Store data locally and asynchronously backup snapshots of data to S3
      • Tape gateway: Archive backup data in Glacier. On Prem backup applications backup to virtual tapes in gateway. The tapes are migrated to S3 and eventually to glacier
  7. Amazon Cloudfront
    • Ideal
      • Distribution of frequently accessed static contents
      • Deliver dynamic web applications over HTTP. Content can be static or dynamic
    • Anti
      • Infrequently requested data
      • Programmatic cache invalidation
  8. Amazon SQS
    • Ideal
      • scenarios where multiple application components must communicate and coordinate their work in a loosely coupled manner
    • Anti
      • Binary or large messages
      • Long-term storage
      • High speed message queuing or very short tasks
  9. RDS
    • Ideal
      • Applications that rely on traditional relational db engines like sql,mysql, oracle etc
      • Applications with structured data that requires sophisticated querying and joining capabilities
    • Anti
      • Index and query focused data ( DynamoDB)
      • Numerous Blobs (S3). Blobs stands for Binary large objects
      • Automated scalability (DynamoDB)
      • Complete control
      • Database platforms which are not provided by AWS
  10. Dynamo DB
    • Ideal
      • Applications that need a flexible NoSQL database
    • Anti
      • Applications that are tied to relational database (RDS/EC2 database AMI)
      • Joins and complex transactions (RDS/EC2 database AMI)
      • Blobs (S3)
      • Large data with low I/O rate (S3)
  11. Amazon Elastic cache
    • Ideal
      • Improve application performance by storing critical pieces of data in memory
      • Improve performance by caching results of I/O intensive queries
      • Applications that need complex data structures like lists,sets,hashes and sorted sets
    • Anti
      • Persistent data
  12. Amazon Redshift
    • Ideal
      • Analyze large datasets using existing business intelligence tools
    • Anti
      • OLTP workloads (RDS)
      • BLOB data (S3)
  13. Databases on EC2
    • Ideal
      • Running relational database ,ones not supported by RDS, on EC2 and EBS
      • Running relational database that are supported by RDS that require maximum level of control
    • Anti
      • Index and query focused data (Dynamo DB)
      • Numerous Blobs (S3)
      • Automatic Scaling
      • MySql,Oracle,SQL Server


Cloud Best Practices
10000 feet overview

Leave a Comment