00:00

Object Storage in System Design

Last updated:

Profile photos, invoices, product videos, backups and log archives share one trait: they are files that are written once, read many times and rarely edited. Putting them in a relational database bloats it, and putting them on a server's local disk ties them to one machine. Object storage is the purpose-built home for this kind of data.

Services such as Amazon S3, Google Cloud Storage and Azure Blob Storage store trillions of objects and scale without any capacity planning. This tutorial explains how object storage works, how it differs from block and file storage, and how presigned URLs let clients upload files safely. It builds on Database Sharding and Data Partitioning, which handle structured data.

1. What Is Object Storage?

Object storage is a storage architecture that manages data as objects. Each object contains the data itself, a unique key and metadata, and objects live in flat containers called buckets. Applications read and write whole objects through an HTTP API rather than through a mounted file system.

Because there is no folder hierarchy to keep consistent and no disk to attach, an object store can spread data across thousands of machines and data centres. Durability comes from keeping several copies or erasure-coded fragments of every object; Amazon S3, for example, is designed for 99.999999999% (eleven nines) durability.

2. Object Storage Architecture

Object storage architecture with presigned uploads and a CDN A client asks a Spring Boot application for an upload URL, and the application returns a short-lived presigned URL. The client then uploads the file directly to the shop-media bucket, which stores objects such as uploads/9f3c.jpg and reports/2026-11.pdf together with their metadata. Downloads reach the client through a CDN that caches objects from the bucket. Clientbrowser or app Spring Boot appsigns URLs CDNcaches downloads BUCKET: shop-media uploads/9f3c.jpgdata + metadata reports/2026-11.pdfdata + metadata logs/app-17.gzdata + metadata 1–2 3 PUT file 4 GET
Steps 1–2: the app returns a short-lived presigned URL. Step 3: the client uploads straight to the bucket. Step 4: downloads come through a CDN, so files never pass through the app.

The application server only signs requests. The heavy traffic of uploading and downloading files goes straight between the client, the bucket and the CDN, so the application stays small and does not need extra bandwidth for files.

3. How Object Storage Works

Take a user uploading a profile photo in a shopping app:

  1. Request: the app asks the backend for permission to upload an image.
  2. Sign: the backend creates a unique key such as uploads/9f3c… and returns a presigned URL: a normal HTTPS URL with an expiry time and a signature made from the backend's credentials.
  3. Upload: the app sends PUT to that URL with the image bytes. The object store checks the signature and expiry, stores the object and replicates it.
  4. Record: the backend saves the key in its database next to the user record. The database stores a short string, not megabytes of image data.
  5. Serve: other users load the photo through a CDN that fetches the object once and caches it near them.

Objects are immutable: changing a photo means writing a new object or version, never editing bytes in place.

4. Block, File and Object Storage

Cloud platforms offer three families of storage. Seeing them side by side makes it clear when object storage is the right choice.

Block storage

Provides a raw disk split into fixed-size blocks, such as Amazon EBS or a SAN volume. The operating system formats it with a file system. It has the lowest latency and suits databases and boot disks, but a volume is normally attached to one machine.

Block storage A database server mounts a 100 GB block volume as a disk. The volume is made of numbered fixed-size blocks, block 0 to block 11 in the drawing, and the server's file system decides what each block contains. Database servermounts /dev/sdb BLOCK VOLUME · 100 GB blk 0 blk 1 blk 2 blk 3 blk 4 blk 5 blk 6 blk 7 blk 8 blk 9 blk 10 blk 11
The server sees a raw disk made of numbered blocks and puts its own file system on top. Fast and low-latency, but usually attached to one machine at a time.

File storage

Provides a shared folder tree over a network protocol, such as NFS, SMB or Amazon EFS. Many servers can read and write the same paths, which suits shared content and legacy applications, but very large file systems are harder to scale.

File storage Two application servers connect to one shared file system over NFS or SMB. The share is organised as a folder tree: /shared contains a reports folder with 2026-11.pdf and an images folder with logo.png. App server 1 App server 2 File shareNFS / SMB /shared ├─ reports/ │ └─ 2026-11.pdf └─ images/ └─ logo.png
Several servers share one folder tree through a network protocol such as NFS or SMB. Paths and folders are familiar, but a single file server limits scale.

Object storage

Provides buckets of objects behind an HTTP API, such as Amazon S3, Google Cloud Storage, Azure Blob Storage or the self-hosted MinIO. It scales almost without limit and is the cheapest per gigabyte, but objects are replaced as a whole and each request has higher latency than a local disk.

Object storage An application talks to a bucket over HTTPS with PUT and GET requests. The bucket has a flat namespace of objects. One object is expanded to show its three parts: the key uploads/9f3c.jpg, the data of 2.4 MB, and metadata such as content type and tags. Other objects include logs/a.gz, img/b.png and docs/c.pdf. Application PUT / GET BUCKET · FLAT NAMESPACE key: uploads/9f3c.jpg data: 2.4 MB of bytes metadata: type, tags logs/a.gz img/b.png docs/c.pdf
Each object is a key, its data and its metadata in one flat namespace; the slashes in keys only look like folders. Applications read and write whole objects over an HTTP API.

Key concepts

  • Bucket: a named container with its own access rules, region and settings.
  • Key: the object's unique name inside the bucket. Prefixes such as reports/2026/ act like folders for listing.
  • Metadata: system data such as size and content type, plus your own tags.
  • Versioning: keeps older versions of a key so that overwrites and deletes can be undone.
  • Storage classes and lifecycle rules: move rarely used objects to cheaper tiers automatically, or delete them after a set time.
  • Consistency: Amazon S3 offers strong read-after-write consistency for all requests, so a read after a successful write returns the new data.

5. Advantages and Disadvantages

Advantages

  • Practically unlimited scale with no capacity planning.
  • Very high durability through replication or erasure coding.
  • Low cost per gigabyte, with cheaper tiers for cold data.
  • Simple HTTP API usable from any language, and direct client uploads through presigned URLs.
  • Rich metadata, versioning and lifecycle automation.

Disadvantages

  • Higher latency per request than block storage, so it is unsuitable for database files.
  • No partial updates: changing one byte means rewriting the object.
  • No real folders, so renaming a "folder" means copying every object under the prefix.
  • Request and data-transfer charges can grow with very chatty access patterns.

6. Block vs File vs Object Storage

AspectBlock storageFile storageObject storage
Unit of dataFixed-size blocksFiles in foldersObjects with key and metadata
Access methodMounted diskNFS or SMB shareHTTP API (PUT, GET, DELETE)
Shared by many serversRarelyYesYes, by any client with access
LatencyLowestLowHigher per request
ScalePer volumeUp to large file systemsVirtually unlimited
Typical useDatabases, boot disksShared content, home foldersMedia, backups, logs, data lakes
ExamplesAmazon EBS, SANAmazon EFS, NFS serverAmazon S3, MinIO, Azure Blob

7. Real-World Use Cases

  • User uploads: profile pictures, documents and attachments stored by key, with only the key kept in the database.
  • Static websites and media: images, scripts and videos served through a CDN; see What Is a CDN.
  • Backups and archives: database dumps moved to cold storage tiers by lifecycle rules.
  • Logs and analytics: data lakes where query engines read files directly from buckets.
  • Microservices: services pass a key or a presigned URL in an event instead of sending large files through a message broker.

8. Best Practices

  • Keep buckets private: block public access and share files only through presigned URLs or a CDN with origin access control.
  • Use short expiry times on presigned URLs, and generate keys on the server instead of trusting client file names.
  • Never store access keys in code: use IAM roles or workload identity, as the default credentials chain does.
  • Validate uploads: check content type and size, and scan files before other users can download them.
  • Enable encryption and versioning, and add lifecycle rules for temporary and old data.
  • Upload large files in parts: multipart upload allows parallel parts and retrying only the failed part.

9. Summary

  • Object storage keeps data as objects (key, data, metadata) in flat buckets behind an HTTP API.
  • It scales almost without limit and is highly durable and cheap per gigabyte.
  • Block storage suits databases, file storage suits shared folders, and object storage suits files written once and read often.
  • Presigned URLs let clients upload and download directly while the bucket stays private.
  • Store keys, not file contents, in your database, and serve popular objects through a CDN.

Next, see how those objects reach users quickly in What Is a CDN.