Service Boundaries: Where Does the Line Between Two…

Prefer to watch? ▶ The full episode ⚡ The 2-minute version ✈ Telegram

You have decided to split. That part is not in question here — this article takes it as given and asks only the next question, the one that actually costs money: where does the line go?

The split almost everyone draws first is by noun — one service per thing the business talks about. Order, user, product, pricing. Or by the org chart — one service per team. Both are visible on day one, on a whiteboard, before anyone has written a line of code. That is exactly why they get drawn, and it is also why neither of them is a boundary.

Six months later you have four services, and one page load needs all four.

The tell: a chain of synchronous calls

Here is the shape you are looking for. order_svc owns the orders table and nothing else on the page. So it asks:

# order_svc - GET /orders/{oid}
# it owns the orders table. it owns nothing else on this page.

@app.get("/orders/{oid}")
def get_order(oid: int):
    row = db.execute("SELECT * FROM orders WHERE id=?", [oid]).fetchone()

    # the customer name is not ours. ask the user service, and wait.
    user = httpx.get(f"{USER_SVC}/users/{row[1]}").json()

    # the title is not ours either - and this one runs PER LINE ITEM.
    for it in line_items(row):
        p = httpx.get(f"{PRODUCT_SVC}/products/{it[0]}").json()
        # ...and product_svc, to answer that, asks pricing_svc itself.

    return {"order_id": oid, "customer": user["full_name"],
            "items": priced, "total_cents": total}

Read that loop again. The fan-out is not a fixed number — it grows with the data on the page. product_svc is asked once per line item, and each of those asks pricing_svc. On the rig used for the episode, a one-line-item order produced 3 outbound calls; the two-line-item demo order produced 5. The formula is 1 + 2 × line_items.

The first version of this argument said "three services, three calls" and drew a single product fetch. Both were wrong. Any article that says three services means three calls is counting boxes on a diagram, not counting requests.

And 5 and 6 are both correct numbers that mean different things. The X-Hops header reports 5 outbound calls. The rig's total inbound count is 6, because the page load into order_svc is itself a request that can fail. Which one you use matters a great deal in a moment.

That is not four services. It is one distributed monolith: the latency of four, the failure modes of four, and the independence of none.

What the chain actually costs

Every figure below was measured on a real four-service rig, 2,000 requests per build, and the naive, collapsed, and modular builds return byte-for-byte identical responses (diff exit 0). Same feature, three architectures.

What was measured	Result
Naive chain (4 processes, HTTP) — mean / p95	10.12 ms / 12.21 ms
Cost of each extra network hop	~1.80 ms
Collapsed (1 process, 1 database) — mean / p95	1.11 ms / 1.18 ms
Speed-up, identical response bytes	9.09× (9.01 ms/req saved)
Actual work — every DB read, all four services	140 µs = 1.38% of the request
`order_svc` blocked on a socket	87.96% of the request
Outbound calls — 1-item order / 2-item order	3 / 5
Boundary in-process vs no boundary	+30 µs (+2.7%)
Boundary over the network vs no boundary	+9.01 ms (+809%)

Two lines in that table are the whole argument. The real work — every database read in all four services — is 1.38% of the request. order_svc spends 88% of its life blocked on a socket waiting for a service to answer a question it could have answered itself.

And a boundary is not intrinsically expensive. The same wall, drawn inside one process, costs 30 microseconds. Drawn across a network, it costs 9 milliseconds — three hundred times more, for the same separation of concerns. You are not paying for the boundary. You are paying for the transport you chose to put it on.

The availability bill: right rule, wrong exponent

Everyone knows the multiplication rule. Four services at 99.9% each, in a chain where all must succeed:

0.999⁴ = 99.6006% → 172.5 minutes of downtime per month

That figure is arithmetically correct — a 1,000,000-trial Monte Carlo matched it. It is also the wrong number, because the read path does not involve four things that must succeed. It involves six: the request into order_svc, plus the five calls it sets off.

0.999⁶ = 99.4015% → 258.6 minutes per month

That is an 86-minute-a-month gap, and it is hidden by counting boxes on a diagram instead of counting what has to answer. Availability is raised to the number of things that must succeed, not the number of boxes you drew.

One honest note, because the temptation to overclaim here is strong: the measured rig matched theory within one standard error once a rig artefact was removed (a sqlite3 connection shared across the threadpool produced phantom 404s and 500s — 36% of all observed failures, all of them pushing the rate down). Measurement did not beat the theory. The interesting result is the exponent, not a discrepancy.

The deploy story is worse than you have been told

Every article on this subject says: deploy them in the right order. Measured on a one-field rename across the boundary — name → full_name in user_svc, which order_svc reads — both orders fail:

Deploy user_svc first → order_svc raises KeyError: 'name' → HTTP 500
Deploy order_svc first → KeyError: 'full_name' → HTTP 500

There is no ordering that works, because the two versions are never simultaneously correct. The only thing that works is a three-deploy expand/contract sequence: emit both fields, migrate the reader, then drop the old field. The seeded history on the rig shows exactly that shape — a commit reading rename name -> full_name (expand: emit both) and, later, (contract: drop name).

So the real cost of that boundary is not "coordination". It is three releases to rename one field.

So what actually decides a boundary?

Two things, and neither of them is a noun.

1. Who owns the data

Exactly one service may write a given fact. Everyone else asks. The interesting move is not making the service smaller — it is deciding, per fact, whose fact it is.

# the fix is not a smaller service. it is deciding who owns each fact.

# WRONG - the order page asks the product service what the price is.
#         that is the price TODAY. the customer paid LAST MONTH.
price = httpx.get(f"{PRODUCT_SVC}/products/{pid}").json()["price"]

# RIGHT - the price charged is not a product fact at all.
#         it is a fact about THIS order, and it never changes again.
db.execute(
    "INSERT INTO orders (user_id, product_id, price_paid, title_at_sale)"
    " VALUES (?, ?, ?, ?)",
    [user_id, pid, price_now, title_now],
)

# reading that order back now touches one table, in one process.
# the call to pricing did not get faster. it stopped existing.

That last comment is the point. The price the customer paid is a fact about the order, not about the product. Once you see that, the call to pricing_svc on the read path is not slow — it is wrong, and it does not need optimising, it needs deleting. A correct ownership decision does not make a call faster. It removes the call.

This is also, incidentally, a correctness fix and not only a performance one: asking the product service for "the price" on an order page returns the price today, for an order placed last month.

2. What changes together

Two things that are always released in the same commit are one thing wearing two names. You do not have to guess at this — your version control has been recording the answer for years.

$ git log --name-only --pretty=format:%H | python3 cochange.py

  pair                            commits touching BOTH      share
  order_svc   + user_svc                    6               75.0%
  product_svc + pricing_svc                 1               25.0%
  order_svc   + product_svc                 0                0.0%
  order_svc   + pricing_svc                 0                0.0%

# 75%, and every user_svc commit touched order_svc too: one unit of change.
# 0% does NOT automatically mean the wall is earning its keep -
#   check whether that pair is on the READ PATH before you trust it.
#   both zero pairs here are on it, and they were resolved differently.

order_svc and user_svc co-change 75% of the time, and every single user_svc commit also touched order_svc. That is not two services. That is one unit of change with a network cable running through the middle of it.

The two mechanisms disagree, and you should say so

Look at the table again. Co-change says nothing at all about order_svc + product_svc — 0%. So the co-change test does not argue for merging order and product; the read-path fan-out does. Two mechanisms, pointing different ways, on the same pair.

Most writing on this topic presents the rules as if they always agree. They do not. When yours disagree, name which one you are following and why. Here: order and product get merged on fan-out grounds, and order and pricing stay apart despite an identical 0% because pricing comes off the read path entirely once price_paid is owned by the order.

The line, moved

Merge order + user + product into one deployable — three modules, one process, one database. Keep pricing a real service: different team, different release day, and crucially it is not on the read path at all any more, because the order stores price_paid and title_at_sale at the time of sale. Its only remaining edge is a one-way, nightly, asynchronous one.

Losing the network does not mean losing the wall:

# ONE process. ONE deployable. the boundary is still real.

# shop/users/api.py - the only way in. no table name ever leaves this file.
def get_user(uid: int) -> User:
    return _row_to_user(_db.execute(
        "SELECT id, name FROM users WHERE id=?", [uid]).fetchone())

# shop/orders/service.py
from shop.users.api import get_user       # a function, not a URL
def order_page(oid: int):
    o = _orders.by_id(oid)                # our own table
    return {"customer": get_user(o.user_id).name,
            "price": o.price_paid}        # our own column

# and the wall is enforced, not merely agreed:
#   a sqlite authorizer refuses any statement from shop.orders that
#   names the users table. a violation raises. it does not lint.

The last three lines are what separates this from wishful thinking. A module boundary that lives in a code-review convention is not a boundary; it is a hope. A SQLite authorizer that raises when shop.orders names the users table is a boundary, and it costs 30 microseconds.

Three architectures, one feature

	Four services (by noun)	One deployable, no wall	One deployable, modules
Mean latency	10.12 ms	1.11 ms	1.14 ms
Cost of the boundary	+9.01 ms (+809%)	—	+30 µs (+2.7%)
Things that must succeed	6	1	1
Availability at 99.9% each	99.4015%	99.9%	99.9%
Rename one field across the line	3 deploys (expand/contract)	1 commit	1 commit
Wall enforced by	the network	nothing	an authorizer that raises
Moving the line later	data migration	refactor	refactor

The costs, stated plainly

This is not free, and the honest version says so. You end up with fewer and larger services than the diagram wanted, which will feel like a regression to anyone who counts services. Moving a line later is a data migration, not a refactor — merging two databases is real work. And a big in-process module still needs someone to defend the wall, because the compiler will happily let you reach across it.

Which is exactly why the reversible move is to draw the line inside one deployable first: one module, one function as the only way in, and the wall enforced mechanically. If the line turns out to be in the wrong place, you move it with a refactor instead of a migration. If it turns out to be right, promoting it to a process later is a small, well-understood step — and you will have real co-change data by then to prove it.

Keeping two databases in step is its own problem, with its own name and its own article. Do not sign up for it by accident on a whiteboard.

The same two tests, on an LLM-serving stack

None of this is specific to shopping carts, and the AI stack is where the noun-split is currently being repeated most enthusiastically. A typical serving diagram has a gateway, an orchestrator, a retriever, a reranker, a model server, and an eval service — six boxes, one per noun, drawn on day one.

Run the two tests on it.

What changes together? In practice the prompt template, the few-shot examples, and the output parser change in the same commit, essentially always. Change the prompt and the parser breaks; tighten the schema and the prompt has to say so. That is a co-change rate near 100% — one unit of change. Split it into a "prompt service" and a "parsing service" and you have bought yourself three deploys to add a field, for exactly the reason name → full_name needed three.

The chunker and the embedder are the same story from the other side: change the embedding model and every vector in the index is invalid. They ship together or the index is quietly wrong.

Who owns the data? The vector index has exactly one legitimate writer — whichever service owns the ingest path. Everyone else reads. And the price_paid lesson transfers directly: when you log a trace, store the prompt text, the model version, and the retrieved chunk ids as they were at call time, on the trace row. They are facts about that call, not about the retriever's current state. A trace that has to ask the retrieval service what it would return today cannot be replayed, and your eval numbers will drift underneath you the same way a re-fetched price does.

What about the fan-out? A request that goes gateway → orchestrator → retriever → reranker → model server synchronously has the same six-exponent availability bill as the shop did, and every hop lands directly on time-to-first-token — the number your users actually feel. The 88%-blocked-on-a-socket figure is, if anything, kinder than what you will measure here, because a rerank hop is not 1.8 ms.

And which box genuinely earns its own process? The model server. Not because it is a different noun, but because it fails all three of the reasons the others failed: it scales on a completely different axis (GPU memory, batch size, KV cache), it releases on a different cadence, and it does not co-change with the prompt at all. That is what an earned boundary looks like — different data owner, different unit of change, different scaling axis. Not a different word on a whiteboard.

The verdict

A boundary you have to call synchronously on every request is not a boundary. It is a network hop you added to a function call.

The point of the line was never tidiness or a tidy diagram. It was independent deployability — can these two things ship apart, on different days, by different people, with nobody coordinating? If the answer is no, you do not have two services. You have one system, and the line you drew is a cost you are paying for nothing.

So before the next split, ask the two questions that do not appear on the whiteboard: who owns this fact, and what changes with it. Then draw the line inside one deployable, enforce it with something that raises rather than something that comments, and let it earn its process later.

References and further reading

The premise — you have decided to split, and now the line has to go somewhere

Sam Newman, Building Microservices (O'Reilly, 2nd ed. 2021) — independent deployability as the defining property of a service, and the case that a boundary you cannot ship apart is not one.
Martin Fowler, MicroservicePremium (martinfowler.com, 2015) — names the fixed cost every distributed boundary charges before it delivers any benefit, which is the ledger this whole article is keeping.

The tell — a synchronous chain across four services

Martin Fowler, First Law of Distributed Object Design (martinfowler.com, 2003) — "don't distribute your objects": the original statement that a remote call is not a local call with extra latency, it is a different thing.
Michael T. Nygard, Release It! (Pragmatic Bookshelf, 2nd ed. 2018) — integration points as the leading source of instability, and how a synchronous dependency turns one service's bad day into everyone's.

The availability arithmetic

Betsy Beyer et al., eds., Site Reliability Engineering (O'Reilly, 2016), ch. "Embracing Risk" — availability targets, error budgets, and why a service's own target has to account for everything it depends on to answer.

Decider 1 — who owns the data

Chris Richardson, Pattern: Database per Service (microservices.io) — one writer per fact, stated as a pattern with its trade-offs, including the queries it makes hard.
Eric Evans, Domain-Driven Design (Addison-Wesley, 2003) — bounded context: the same word means different things to different parts of the business, which is why "price" belongs to the order and not to the product.

Decider 2 — what changes together

David L. Parnas, On the Criteria To Be Used in Decomposing Systems into Modules (Communications of the ACM, 1972) — decompose by what is likely to change, not by processing steps; the ancestor of the co-change test.
Adam Tornhill, Your Code as a Crime Scene (Pragmatic Bookshelf, 2015) — mining version-control history for change coupling, i.e. how to get the git log table above out of your own repository.
Melvin E. Conway, How Do Committees Invent? (Datamation, 1968) — the org-chart split, and why it keeps getting drawn whether or not it is the right line.

The three-deploy rename

Danilo Sato, ParallelChange (martinfowler.com, 2014) — expand / migrate / contract, the only sequence that survives a rename across a boundary when neither deploy order works.

If a reference you'd expect is missing, say so in the comments and I'll add it.

Watch the reel: the 2-minute version draws the four-service shop and names the two deciders; the full episode runs the ownership fix, the co-change analysis, and the in-process module wall on a rig that really executes.