Pub/Sub Explained From Scratch — One Event, Five Systems, Zero Direct Calls

When you scale from one service calling another to five services that all need to react to the same event, you hit a wall: every new consumer means touching the producer's code, and one broken downstream service can fail the entire operation. Pub/sub flips that model.

• Mental model: Publish once to a broker; the broker delivers a copy of that event to every subscriber in parallel, with queues and retries, while the publisher never learns who is listening.

---

The Problem: Direct Calls Break at Scale

Start from the simplest case: two services. Checkout finishes an order and needs to notify email.

// Synchronous direct call
const order = { id: 12345, total: 99.99 };
const emailResult = await emailService.send(order);
const inventoryResult = await inventoryService.reserve(order);
const warehouseResult = await warehouseService.ship(order);

Checkout knows the email service exists at a specific address and waits for a response. One service, one dependency. Straightforward.

Now the store grows. Marketing wants loyalty points. Analytics wants a record. The warehouse needs a pick list. You add call after call:

• Latency stacks: 300ms (email) + 250ms (inventory) + 400ms (warehouse) + 200ms (loyalty) = 1,150ms. Your customer stares at a spinner.
• Coupling explodes: Edit checkout every time a new team subscribes. Redeploy the entire checkout service. Risk a regression.
• Fragility spreads: If the email service is down for 30 seconds, the entire order fails. One broken listener breaks the sale.

This is the synchronous, coupled model. It works when you have one or two listeners. It collapses when you have five.

---

The Flip: Publish, Don't Call

Instead of checkout calling five services, checkout publishes one event to a message broker:

// Asynchronous publish
const order = { id: 12345, total: 99.99 };
await broker.publish('orders', {
  type: 'order.placed',
  data: order,
  timestamp: Date.now()
});
// Done. Returned in ~20ms. No waiting.

Checkout has no idea who is listening. It does not know email exists, or inventory, or loyalty. It announces a fact to a named channel (a topic) and moves on.

Each consumer subscribes independently:

// Email service
broker.subscribe('orders', async (event) => {
  if (event.type === 'order.placed') {
    await sendReceipt(event.data);
  }
});

// Inventory service
broker.subscribe('orders', async (event) => {
  if (event.type === 'order.placed') {
    await reserveStock(event.data);
  }
});

// Loyalty service (added later, no checkout change)
broker.subscribe('orders', async (event) => {
  if (event.type === 'order.placed') {
    await awardPoints(event.data);
  }
});

Each subscriber listens to the same topic and processes the same message. The broker delivers a copy to each one, in parallel. None of them block checkout.

---

How the Broker Handles Delivery

Fan-Out

The broker keeps a named channel (#orders). When checkout publishes, the broker sends one copy to every subscriber queue simultaneously. If email takes 10 seconds to process and inventory takes 2 seconds, they run in parallel, not sequentially. No waiting.

Buffering and Retries

Email crashes for 30 seconds. The message does not vanish. It sits in email's queue until the service comes back online, then drains the backlog. Inventory and analytics never notice. Messages are not lost; they are eventually delivered. (Kafka guarantees at-least-once delivery, which means a message might arrive twice—make your consumers idempotent.)

Decoupling

Checkout does not know how many subscribers exist. The fraud team adds a subscription to #orders next month. Loyalty adds another. None of them change checkout's code. There is no dependency graph. The topic is the contract.

---

Comparison: Direct Calls vs. Pub/Sub

Dimension	Direct Call (Sync)	Pub/Sub (Async)
Latency	Stacks: 300 + 250 + 400ms = 950ms	Publisher returns in ~20ms; consumers process in parallel
Coupling	Producer knows every consumer; edit checkout for each new one	Producer publishes to a topic; subscribers are unknown and independent
Failure Mode	One broken consumer fails the whole transaction	One broken consumer queues messages; others process in parallel
Scalability	Add a consumer → edit and redeploy producer	Add a consumer → subscribe to the topic; no producer change
Consistency	Strong; all calls complete before response	Eventual; subscribers may lag by seconds or more
Idempotence Required	No; call happens once	Yes; at-least-once delivery means duplicates can arrive

---

The Radio Station Analogy (and Where It Breaks)

A radio station broadcasts a song. Anyone tuned to that frequency hears it. The station does not know who is listening or care. One broadcast reaches millions.

Pub/sub works the same way: one publish, many subscribers, no direct connection between them.

But the analogy breaks in three places:

1. No request-response: A radio broadcast is one-way. If you need an answer (like "did the payment go through?"), you still need a direct call. Pub/sub is for announcing facts, not asking questions.

2. Eventual consistency: Listeners hear the broadcast live. Pub/sub subscribers may process messages seconds later. If a customer refreshes the page immediately after placing an order, the loyalty points may not appear yet.

3. Duplicates: Radio broadcasts once. Pub/sub brokers may deliver the same message twice (e.g., after a crash and recovery). Subscribers must be idempotent: processing the same message twice should have the same effect as processing it once.

---

Real-World Brokers: Same Idea, Different Trade-Offs

Kafka is a distributed log. Publish to a topic; every subscriber gets the full history and can replay from any offset. Designed for high throughput and durability.

RabbitMQ uses exchanges and queues. Publish to an exchange; it routes messages to queues based on routing keys. More flexible routing, simpler for small teams.

Redis Streams is in-memory with optional persistence. Fast, simple, good for lower-volume event streams.

AWS SNS/SQS decouples the two concerns: SNS fans out to multiple SQS queues. SNS is the publisher; SQS provides the buffering and retry logic.

All of them solve the same problem: decouple publisher from consumers, buffer messages, and retry on failure.

---

Pub/Sub and LLM Inference

In AI-serving systems, pub/sub reshapes how you handle token generation and batch inference. Instead of a request-response model (user sends prompt, waits for full completion), you can publish a "generate tokens" event to a queue, have multiple inference workers subscribe and pull batches, and stream results back asynchronously. This decouples the request handler from GPU schedulers, allows batching across many concurrent requests (improving TPOT), and makes it easy to add a new inference node without touching the request layer. For latency-sensitive workloads, pub/sub with streaming acknowledgments lets you return TTFT early while workers finish generating tokens in the background.

---

When to Reach For Pub/Sub

Reach for pub/sub when you have multiple consumers that need to react independently to the same event, when one consumer's slowness should not block the producer, when you expect to add new consumers over time, or when you need to buffer spikes. Reach for direct calls when you need a synchronous response (a payment authorization, a validation), when low latency is critical and you have only one consumer, or when you need strong consistency guarantees.

Watch the 90-second reel to see the checkout flow in action.