software-engineer-blog logoSoftware Engineer Blog

Module 7 · Concurrency

Unit 26 of 49

Unit 26 · Module 7 · Concurrency

Async and the event loop

One thread, thousands of connections — as long as nothing blocks it.

Unit 26 of the free 49-unit computer-science course, in concurrency. 4 topics to watch or read, 3 interview questions answered in full and a short self-check.

Watch or read

4 topics make up this unit. Take each one whichever way suits you, then answer the questions below.

Synchronous vs asynchronous

Synchronous vs asynchronous comes down to one choice: when you send a request, do you block and wait for the answer, or fire it off and get notified later? Here's what each actually does — blocking vs non-blocking, callbacks, promises, async/await, and webhooks — the throughput payoff and the complexity cost, a clear rule for which to reach for, and how the same trade-off shows up in async servers and LLM streaming.

ReelRead

Callback vs promise vs webhook

Callback, promise, and webhook get treated like three flavors of the same thing — they aren't. The real difference is WHERE the result lives. Callback and promise hand a result back to the same program in memory in milliseconds; a webhook is another machine calling you back over the network, seconds to minutes later. Here's the mental model, the trade-offs, code for each, and a clear rule for which to reach for — including what it means for async LLM jobs.

ReelRead

asyncio.gather + a semaphore

500 LLM calls in a for-loop takes 17 minutes — and your CPU does nothing for all of it. await is not concurrency. Here's how asyncio.gather puts every request in flight at once, why it's named for the results and not the speed, and why asyncio.Semaphore is the one line that stops you from DDoS-ing yourself into a wall of 429s.

Read

setTimeout and setInterval

Read

Interview questions this unit unlocks

Asked out loud, answered out loud. Read the answer, then say it in your own words.

How does an event loop give you concurrency with one thread?

Because most of what a server does is wait. The loop keeps a queue of ready callbacks and a set of pending I/O registered with the kernel; when a socket becomes readable, the kernel says so and the loop runs the continuation. One thread can hold ten thousand open connections because at any instant nearly all of them are idle. It buys you nothing at all on CPU work — a tight loop blocks everything, and there is no scheduler to preempt it.

The rule that follows: never do blocking or CPU-heavy work on the loop. Hand it to a thread pool or a separate process.

What is the difference between concurrency and parallelism?

Concurrency is structuring a program so multiple things are in progress at once; parallelism is actually executing them at the same instant on different cores. One chef juggling five orders is concurrent. Five chefs is parallel. An event loop is concurrency with no parallelism, and that is exactly the right tool when the bottleneck is the network rather than the CPU.

You have 500 API calls to make. How do you run them?

Concurrently, but bounded. Firing all 500 at once will exhaust the connection pool, trip the remote rate limit, or both — so a semaphore capping in-flight requests at something like ten or twenty, and gather the results. The cap is the whole design: it is what turns "as fast as possible" into "as fast as the dependency will tolerate", and it is the number you tune when the remote starts returning 429.

Self-check — 3 questions

Answer alone, at 2am, with no interviewer in the room.

Part of Everything You Need to Know About Computer Science.