software-engineer-blog logoSoftware Engineer Blog

Module 8 · Distributed systems

Unit 32 of 49

Unit 32 · Module 8 · Distributed systems

Queues, backpressure and dead letters

A queue does not remove load. It moves it in time — until it does not.

Unit 32 of the free 49-unit computer-science course, in distributed systems. 4 topics to watch or read, 3 interview questions answered in full and a short self-check.

Watch or read

4 topics make up this unit. Take each one whichever way suits you, then answer the questions below.

Pub/sub from scratch

Pub/sub decouples producers from consumers by publishing one event to a broker that fans it out to all subscribers with queues and retries. Compare direct calls vs. event-driven architecture.

ReelRead

Pub/sub vs API calls

A user signs up, and email, push, and analytics all need to know. Do you call each service yourself, or publish one event and walk away? Here's Pub/Sub vs direct API calls — event-driven async messaging vs synchronous request/response — with a clear rule for which one to reach for.

ReelRead

Airflow vs cron

cron and Airflow both run jobs on a schedule, but they answer different questions. cron asks 'is it time yet?' — one line, zero dependencies, forty years of uptime. Airflow asks 'what's the state of my whole pipeline?' — a DAG with dependencies, retries, backfills and a UI, at the cost of a scheduler, a Postgres and a worker pool. Here's the real distinction, with code, and a rule for picking.

ReelRead

Background jobs without a broker, in Postgres

A job is a row, not a function call. And a connection is not a transaction. Those two sentences separate a background worker that survives a deploy from one that quietly stops Postgres cleaning up. Measured on PostgreSQL 16.14 — including why 'idle in transaction blocks VACUUM' is false as usually told.

ReelRead

Interview questions this unit unlocks

Asked out loud, answered out loud. Read the answer, then say it in your own words.

Why put a queue between two services instead of calling directly?

To decouple availability and rate. A direct call means the caller is down when the callee is down, and the caller's throughput is capped by the callee's. A queue lets the producer keep accepting work while the consumer is slow or restarting, absorbs spikes rather than shedding them, and lets you scale the consumer independently. What you buy it with is latency, ordering that is now your problem, and a whole second system to operate.

What is backpressure, and what happens without it?

It is the consumer telling the producer to slow down. Without it the queue grows without bound: memory or disk fills, latency through the queue climbs until the results are useless, and the failure arrives all at once instead of gradually. Real backpressure means bounded queues and an explicit decision about what to do when full — block the producer, shed the load, or drop the oldest — and that decision is a product question, not a technical one.

The failure to name: a queue that is always empty is a queue doing its job; a queue that is always growing is an outage that has not happened yet.

What is a dead letter queue for?

Messages that fail repeatedly. Without one, a single poison message is retried forever at the head of the queue and blocks everything behind it, or is dropped and silently lost. After N attempts the message is moved to a separate queue with its failure reason, the consumer moves on, and a human gets to look at the ones that actually broke.

An empty DLQ that nobody alerts on is the same as no DLQ. The alert is the feature.

Self-check — 3 questions

Answer alone, at 2am, with no interviewer in the room.

Part of Everything You Need to Know About Computer Science.