S3 vs Database Blobs: Why Your Files Don't Belong in Postgres (and What Pre-Signed URLs Fix)

Instagram stores billions of photos and videos. Somewhere, a decision was made about where the actual bytes of every one of those files live — inside the database, or in object storage like S3. It's one of the easiest calls to get wrong on day one, and one of the most brutal to undo once you're at scale.

The one-line mental model:

• Blob in the database is keeping the whole parcel inside the filing cabinet — everything is in one drawer, one lock, one place. Convenient until the cabinet is full of bricks.
• Object storage is keeping the parcel in a warehouse and filing only the shelf number — the database stays a thin index, the heavy stuff lives where heavy stuff belongs.

---

Blobs in the database: everything in one place

The tempting path is to add a bytea (Postgres) or BLOB (MySQL) column and shove the file's bytes straight into the row:

CREATE TABLE photos (
    id          BIGSERIAL PRIMARY KEY,
    user_id     BIGINT NOT NULL,
    created_at  TIMESTAMPTZ DEFAULT now(),
    data        BYTEA NOT NULL     -- the entire JPEG lives here
);

For a weekend project this genuinely works, and it has real appeal:

• One place. One database, one backup, one restore.
• Transactions. The row and its bytes commit or roll back together — no orphaned files, no "the DB says it exists but the file is missing."
• Access control for free. Whoever can read the row can read the file; you already have that logic.

So why doesn't Instagram do this?

Why it falls apart

A relational database is engineered for small, structured rows that it can index, cache, and join. Multi-megabyte media is the opposite of that, and every part of the system pays for it:

• Backups and restores balloon. Every dump and every restore now drags terabytes of image bytes through your most expensive, hardest-to-scale tier. A logical backup that used to take minutes takes hours.
• Reads pull huge binaries through the connection pool. A single request for a 4 MB photo occupies a database connection while megabytes stream out of it. Connections are a scarce resource; you're spending them on plumbing bytes.
• The cache thrashes on pixels. Your database's buffer cache is precious RAM meant for hot indexes and rows. Fill it with JPEG bytes that are read once and it evicts the very data that makes queries fast.
• No CDN, no easy streaming. The bytes sit behind your application and your database. You can't put a global CDN in front of a SELECT, and range requests for video seeking become your problem to reinvent.

You end up scaling your most costly component to do a job it was never built for.

---

Object storage (S3): the file is an object, the DB keeps a reference

Object storage flips the layout. The file becomes an object in a bucket, addressed by a key. The database keeps only a tiny reference — a key or URL, a few bytes per row:

CREATE TABLE photos (
    id          BIGSERIAL PRIMARY KEY,
    user_id     BIGINT NOT NULL,
    created_at  TIMESTAMPTZ DEFAULT now(),
    object_key  TEXT NOT NULL      -- e.g. "photos/2026/07/ab12cd.jpg"
);

What you get in return:

• Cheap storage priced for bulk bytes, not for a transactional engine.
• Eleven nines of durability (99.999999999%) — the platform replicates and repairs objects for you.
• It scales itself. No sharding media across DB nodes; the bucket just grows.
• CDN-ready. Object storage sits naturally behind a CDN, so bytes are served from an edge near the user instead of from your origin.

Your database goes back to doing what it's good at: indexing, joining, and answering queries fast — over rows that are now a few bytes each.

The piece that ties it together: pre-signed URLs

The obvious worry: if the file lives in a bucket, isn't it just... public? No. The clean pattern keeps the bucket private and uses pre-signed URLs for access.

When a user needs to upload or view a file, your server generates a temporary, cryptographically-signed link that grants access to exactly one object, for just a few minutes:

import boto3

s3 = boto3.client("s3")

# Hand the browser a link to GET one object, valid for 5 minutes.
url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "user-media", "Key": "photos/2026/07/ab12cd.jpg"},
    ExpiresIn=300,   # seconds
)

The browser then talks straight to S3 with that link. This is the whole point:

• The bytes never touch your server. You offload the bandwidth entirely — your app returns a short string, not a 4 MB payload.
• The bucket stays locked down. Nothing is public; access is granted one object at a time.
• The link expires on its own. No revocation bookkeeping — a leaked URL is dead in minutes.

Uploads use the same trick in reverse: hand the browser a pre-signed PUT (or a pre-signed POST policy), and the client uploads directly to the bucket without the bytes ever passing through your API.

---

The comparison at a glance

Dimension	Blob in the database	Object storage + reference
Row size	Megabytes per row	A few bytes (a key/URL)
Backups & restores	Drag terabytes through the DB tier	Stay small and fast; media backed up by the object store
Reads	Huge binaries through the connection pool	Browser fetches bytes straight from S3
Cache behavior	Buffer cache thrashes on pixels	Buffer cache stays hot on indexes/rows
Cost per byte	Most expensive tier you run	Cheap bulk storage
Durability	Your backup discipline	~11 nines, managed for you
CDN & streaming	You reinvent it	Native — CDN in front, range requests included
Access control	Free via row permissions	Private bucket + short-lived pre-signed URLs
Transactional consistency	Atomic with the row	Eventual — needs cleanup for orphaned objects

Object storage isn't free of trade-offs. You lose the neat atomicity of "the row and the file commit together," so you need a small amount of discipline: write the object first, then the reference; and run a background sweep to garbage-collect objects whose rows never materialized (or rows whose objects were deleted). That bookkeeping is trivial next to the cost of scaling a database full of media.

---

The same pattern shows up in AI/LLM serving

If you're building anything with models, this isn't a niche web concern — object storage is the backbone of how model artifacts and generated media move around:

• Model weights and checkpoints. A single set of LLM weights is tens to hundreds of gigabytes. They live as objects in a bucket; your serving nodes pull them by key at startup. You would never put a checkpoint in a bytea column — same reasoning, one more order of magnitude.
• Generated media. An image or video generation endpoint produces large binaries on the fly. The clean flow is: write the output to object storage, store the key in your database next to the request record, and return a pre-signed URL so the user's browser downloads straight from the bucket — the bytes never round-trip through your inference server, which you want doing GPU work, not byte-shuffling.
• RAG source documents. The raw PDFs, images, and transcripts behind a retrieval system belong in a bucket; your vector store and metadata DB keep only the chunk text, embeddings, and the object key pointing back to the source. Retrieval stays fast because the heavy originals are out of the hot path.
• Training and eval datasets. Multi-terabyte datasets stream from object storage into training jobs. The database (or a catalog) tracks manifests and references, not the samples themselves.

The rule scales all the way up: heavy bytes in object storage, a thin reference in the database, moved with short-lived signed URLs. Whether it's an Instagram photo or a 70B checkpoint, the shape is identical.

---

The verdict

For a prototype where nothing will ever grow, a bytea column is fine — don't over-engineer a weekend project. For anything that will hold real user media or serve real traffic, the answer is settled:

Bytes in object storage. Reference in the database. Moved with short-lived pre-signed URLs behind a CDN.

Your database stays lean and fast, your storage stays cheap and durable, and your files stay private without your server ever sitting in the byte path. That's not a clever trick — it's simply how the big apps actually do it.

---

▶ Want the 90-second version? Watch the Short · and follow @software-engineer-blog for daily software & systems breakdowns.