S3 for object storage

Amazon offers storage options to accommodate different types of data to be stored in the cloud.

Storage Options

EFS (File storage)

Data is stored as files in folder hierarchies and accessed through a path making searching for data relatively easy.

EBS & EC2 instance (Block storage)

S3 & Glacier (Object storage)

S3 is an object store where objects are stores in a flat structures, called ‘buckets’ and are accessed by a unique ID called a ‘key’ that identifies the address of the data. A key can be any string. Object storage is best used for unstructured and non-relational data such as a blob of data or files, videos, images, text, documents, etc. Objects are stored in a flat address space which enables efficient scalability and object retrieval via REST APIs.

S3 is considered to be durable by redundantly storing objects in multiple regions across availability zones, S3 object storage works in an eventual consistency model where data consistency is only eventually achieved. Any CRUD operations on the data would take some time to propagate across replicas.

S3 provides unlimited storage capacity with no limit to the amount of data to be stored and will scale according to the storage requirements and demand. There is however, a 5 TB limit on the object size although no limit on the number of objects that can be stored.

Reduced Redundancy Storage (RRS)

Non-critical data is stored using RRS which reduces storage costs and can be used for objects that could be easily reproduced in the event of failure. The original data object is stored in S3 where RRS is used for a view.

Glacier

Glacier storage is an extension of S3 and is used for archiving data that is infrequently used where latency for data retrieval is not critical. While Glacier is just as durable as S3, the storage costs are much less. Objects in Glacier is managed through S3 where existing data objects are transitioned into Glacier when the data is ready to be archived.