--- title: "Concurrency vs Parallelism: One Chef Juggling Orders vs Many Chefs Cooking (and Which Your Bottleneck Needs)" description: "Concurrency and parallelism sound like synonyms but they're two independent ideas. Concurrency is structure — one worker interleaving many tasks, great for I/O-bound work that spends its time waiting. Parallelism is execution — many cores each running a task at the same instant, the only thing that actually speeds up CPU-bound work. Here's the difference, why they're orthogonal, and how it maps to serving LLMs (async request handling vs batched GPU compute)." keywords: "concurrency vs parallelism, concurrency, parallelism, async await, event loop, I/O-bound, CPU-bound, multithreading, multiprocessing, GIL, goroutines, async programming, LLM serving, continuous batching, throughput, backend system design" created_at: "2026-07-02T09:00:00" post_type: "anonym_post" content_type: "technical_article" --- ![Banner](./banner.webp)

Prefer to watch? ▶ Concurrency vs Parallelism in 90 seconds ✈ Telegram

Almost every engineer uses "concurrency" and "parallelism" as if they mean the same thing. They don't — and the confusion costs real performance, because the fix for a *slow* system depends entirely on which one you're actually missing. The one-line mental model: - **Concurrency** is **one chef juggling many orders** — starting the pasta, then chopping onions while it boils, then plating a starter. One worker, many tasks in flight, progress by *interleaving*. - **Parallelism** is **many chefs each cooking a dish at the same instant** — more hands, more dishes finished per minute, literally simultaneous. One is about how you **structure** work. The other is about how it **executes**. They're not the same axis. --- ## Concurrency: structure, one worker, interleaved Concurrency is a way of **structuring a program** so independent tasks can make progress without blocking each other. A single worker switches between tasks: start a download, and while it's waiting on the network, handle an incoming request, then come back to the download when its data lands. On a single core, nothing runs *literally* at the same time — it just **interleaves** fast enough that everything keeps moving. That's exactly what you want for **I/O-bound** work, which spends most of its life *waiting* — on the network, the disk, a database. ```python import asyncio async def fetch(name, delay): print(f"{name}: start") await asyncio.sleep(delay) # waiting on I/O — yields control print(f"{name}: done") return name async def main(): # one thread, one core — but all three overlap their WAIT time await asyncio.gather( fetch("A", 2), fetch("B", 2), fetch("C", 2), ) asyncio.run(main()) # finishes in ~2s, not ~6s ``` Three tasks that each "take" two seconds finish in about two seconds total — not six. No extra cores were used. The event loop simply refused to sit idle while one task waited. --- ## It's structure — no extra cores required This is the part people miss: **concurrency does not need multiple cores.** An event loop, `async`/`await`, a single thread cooperatively switching between tasks — all concurrent, all on one CPU. Goroutines, coroutines, callbacks, and green threads are all concurrency *models*: ways to express "these tasks are independent and can overlap," independent of how many cores actually exist underneath. Concurrency is about **dealing with** many things at once. Whether any two of them *run* at the same instant is a separate question. --- ## Parallelism: execution, many workers, simultaneous Parallelism is about **execution**. Multiple workers — multiple CPU cores — each running a task at the *very same instant*. This is the thing that actually speeds up **CPU-bound** work, where the bottleneck isn't waiting, it's raw computation: resizing a million images, crunching a matrix, hashing a huge file. The pattern is **split → run → combine**: break the job into independent pieces, run the pieces simultaneously on different cores, then merge the results. More cores means more real throughput — up to the point where the work stops dividing cleanly. ```python from multiprocessing import Pool def heavy(n): # pure CPU work return sum(i * i for i in range(n)) if __name__ == "__main__": with Pool(4) as pool: # 4 cores actually running at once print(pool.map(heavy, [10_000_000] * 4)) ``` For CPU-bound work, async won't save you — there's no wait time to hide. You need more hands doing the crunching, which means real parallel execution across cores (and, in Python, separate processes to sidestep the GIL). --- ## The catch: they're orthogonal Here's what makes the terms so slippery — they're **independent axes**. You can have either without the other: - **Concurrent, not parallel:** async on a single core. Many tasks in flight, one at a time on the CPU. - **Parallel, barely concurrent:** the *same* operation applied across a giant array on many cores (SIMD / data parallelism) — simultaneous execution, almost no task-juggling. - **Both:** a multi-core service running an async runtime on each core — juggling *and* simultaneous. - **Neither:** a plain single-threaded, blocking script. Concurrency **enables** parallelism (you have to express independent tasks before you can run them at once) — but it **isn't** parallelism. Structure it concurrent; run it parallel.

	Concurrency	Parallelism
What it is	Structure — dealing with many tasks	Execution — doing many tasks
Workers	One can suffice (interleaving)	Many (one per core)
At the same instant?	Not necessarily	Yes, literally
Needs multiple cores?	No	Yes
Best for	I/O-bound (waiting)	CPU-bound (crunching)
Typical tools	async/await, event loop, goroutines	multiprocessing, threads on many cores, SIMD/GPU

--- ## Where this bites in AI/LLM serving If you run an LLM inference service, you live on *both* axes at once — and mixing them up wrecks either latency or GPU utilization. - **Concurrency handles the requests.** An inference server juggles hundreds of in-flight requests with an async event loop: accepting connections, streaming tokens back over SSE, waiting on clients. That's almost pure I/O — waiting on sockets — so concurrency (not extra cores) is what lets one process serve many users without blocking. - **Parallelism does the math.** The actual forward pass is brutally CPU/GPU-bound. The GPU runs thousands of arithmetic units *in parallel*, and **continuous batching** packs many requests' tokens into one simultaneous matrix multiply so the hardware is never idle. Model/tensor parallelism then splits a single large model across multiple GPUs running at the same instant. So a modern serving stack (vLLM, TGI, and friends) is a textbook case: **async concurrency at the request layer** to stay responsive while waiting, **batched parallelism at the compute layer** to maximize throughput. Confuse the two and you either starve the GPU (too little batching) or block your event loop with CPU work (too little parallelism offload). The distinction isn't academic — it's the difference between a server that streams smoothly under load and one that stalls. --- ## Which should you reach for? - **I/O-bound and mostly waiting** — network calls, DB queries, file reads, fan-out to other services? Reach for **concurrency** (async/await, an event loop). Extra cores won't help you wait faster. - **CPU-bound and crunching** — heavy computation, data transforms, model math? Reach for **parallelism** (multiple cores/processes, GPU). Async won't help when there's no wait to hide. - **Most real systems need both** — concurrency to stay responsive while things are in flight, parallelism to actually finish the heavy work. The skill is knowing which layer of your system is which. So: concurrency is **dealing with** many things at once; parallelism is **doing** many things at once. The next time something's slow, don't reach for the tool by reflex — ask first: *is my bottleneck waiting, or computing?* The answer tells you which axis you're actually short on. > Want the 90-second visual version? [Watch the reel.](https://youtube.com/shorts/ClgSvCXEpCc)