software-engineer-blog logoSoftware Engineer Blog

Module 5 · Networking

Unit 19 of 49

Unit 19 · Module 5 · Networking

Sockets, latency, timeouts and retries

Every network call fails eventually. Design for the failure, not the happy path.

Unit 19 of the free 49-unit computer-science course, in networking. 3 topics to watch or read, 3 interview questions answered in full and a short self-check.

Watch or read

3 topics make up this unit. Take each one whichever way suits you, then answer the questions below.

Circuit breaker

Read

Latency vs throughput

Latency and throughput both measure performance, but they answer different questions: how long ONE request takes versus how MANY you can serve per second. Here's the difference that actually helps you debug — milliseconds and p99 tail latency, RPS/QPS capacity, Little's Law, why pushing utilization toward 100% makes latency explode, and how the same trade-off shows up in LLM serving.

Read

Debouncing vs throttling

Debounce waits for silence and fires once; throttle watches the clock and fires steadily. The search box makes debounce look universal — a drag handler at 60 events/sec proves it isn't. Both mechanisms in plain JavaScript, both honest failure modes (debounce can starve forever, a naive throttle drops the final event), and a rule for picking one.

ReelRead

Interview questions this unit unlocks

Asked out loud, answered out loud. Read the answer, then say it in your own words.

What is the difference between latency and throughput?

Latency is time per request; throughput is requests per second. They are not the same axis and improving one can hurt the other — batching raises throughput by making each individual request wait, and adding workers raises throughput while queueing raises the latency each one sees. The question that resolves it is always which one the user actually feels.

Little's law is the sentence that makes this concrete: concurrency equals throughput times latency. Fix any two and the third is decided for you.

How do you set a timeout, and what happens if you do not?

Shorter than the caller's patience and longer than the honest p99 of the dependency — and a connect timeout separate from a read timeout, because failing to reach a host and being slow to answer are different failures. With no timeout, a hung dependency does not just fail the request; it holds a connection and a worker until the pool is exhausted, and a slowdown in one downstream service becomes an outage in yours.

You are retrying a failed call. What do you have to get right?

Three things. Retry only what is idempotent, or attach an idempotency key, or you will double-charge someone. Use exponential backoff with jitter, because synchronised retries from a thousand clients are a self-inflicted denial of service. And cap the total attempts, because retrying a dependency that is already overloaded is how a brownout becomes an outage.

The pattern that stops the whole class: a circuit breaker. After enough failures, stop calling entirely for a while and fail fast instead of queueing.

Self-check — 3 questions

Answer alone, at 2am, with no interviewer in the room.

Part of Everything You Need to Know About Computer Science.