software-engineer-blog logoSoftware Engineer Blog

Question 6 · Track 2 · Store it and serve it

Design a file storage and sync service

The bytes are the easy half.

Store it and serve it. 4 clarifying questions to ask first, the answer in 4 moves, the follow-up that catches a memorised answer and 2 supporting topics to watch or read.

What it is really testing

Whether you separate metadata from blobs, and whether you sync diffs or whole files.

Ask these first

Before a single box goes on the board. The answers change the design, which is the point of asking out loud.

The answer, in 4 moves

In this order. Each move earns the next one — say them out loud rather than drawing all four and narrating afterwards.

  1. 1Split every file into fixed-size chunks, addressed by their hash.
  2. 2Keep a metadata service that owns the file tree; the blobs live in object storage.
  3. 3Sync by diffing chunk lists, so an edit uploads one chunk, not one file.
  4. 4Replicate chunks; deduplicate them across users for free.

The trap

The follow-up that separates a rehearsed answer from a real one.

The metadata service is the bottleneck, not the bytes. Object storage scales on its own; the tree of who-owns-what, listed on every client poll, is the part you have to design.

Watch or read

The pieces of this answer, each covered on its own. Take them whichever way suits you, then give the whole answer without looking.

S3 vs database blobs

Should raw image and video bytes live inside your database or in object storage like S3? Stuffing files into a bytea/BLOB column keeps everything in one place and works for a weekend project — then backups drag terabytes through your most expensive tier and reads pull huge binaries through the connection pool. Here's why object storage plus a tiny DB reference plus short-lived pre-signed URLs is how the big apps actually store media — and how the same pattern serves model weights and generated media for LLM apps.

ReelRead

How safe is a 60-minute signed URL?

You added a CORS rule to a private bucket and then wondered whether you had just made every customer invoice public. You had not — CORS lives in the browser and never decides who may fetch a file. The signed URL does, and it is a bearer token: whoever holds the string gets in, with no login, no session, and no way to cancel it. The expiry is the only dial you get, so here is how to set it on purpose.

ReelRead

Part of System Design Interviews, Answered Out Loud. The fundamentals underneath it are the free CS course.