On Labyrinths and Routing

The empty hall

The first vestibule of The House¹ is the size of three tennis courts and contains, at the time of its description, exactly one statue and a great deal of cold air. The statue is of a faun. The cold air comes from somewhere else. There is no map, and the narrator — patient, careful, still — has not yet thought to ask for one.

I think about this opening when I think about the systems I've been building. Agentic swarms — small fleets of language-model workers passing tasks between themselves — are buildings the same way. Halls, vestibules, corridors. Some of them have statues in them. Most of them are cold.

The model that ships your routing decision is a kind of architect. It commits to the shape of a hallway you haven't walked down yet, and trusts that the shape will hold up when the next agent steps into it. If the routing is wrong, the swarm gets lost. Not lost in the way a function is lost — lost in the way a person is.

Routing as cartography

The earliest routing decision in any agent system is not which model but which kind of question is this?² The categorization is the map. If the map is wrong, no amount of model quality will save you.

The classifier is the lobby. Everything passes through it on the way somewhere else. We give it a budget — cycles, tokens, latency — and ask it to make a single, cheap decision: which corridor. Then we close the door behind it.

You can write the lobby down in five lines. The trick — the only trick — is making sure those five lines really, truly never panic. Below, the smallest viable router I keep returning to:

# router.py — the smallest viable lobby
from typing import Literal

Corridor = Literal["retrieve", "summarize", "tool", "refuse"]

def route(query: str, *, budget_ms: int = 120) -> Corridor:
    if looks_like_question(query):  return "retrieve"
    if looks_like_command(query):   return "tool"
    if looks_like_corpus(query):    return "summarize"
    return "refuse"  # the closed door is also a door

Notice the last line. The closed door is also a door. A router that cannot refuse is not a router; it is a hallway with no end, which is to say, a labyrinth in the bad sense.³

A tiny router

What follows is the entire production routing system for one of my swarms — under a hundred lines including imports, with a small evaluation harness underneath. I include it here because most architectural posts about agents are gestures at architectures rather than the architectures themselves.

The fourth step is where most teams fail. They reach for a third model, then a fourth. The hallway gets longer. The faun in the lobby grows tired and starts answering questions itself, badly. The cure for "the model isn't sure" is not "ask another model," it is "decline the corridor."

Costs of getting lost

What I cannot lift, I will trace with care.⁴

There is a real economic cost to getting lost. Each unrouted query is, in expectation, a few cents of compute and a few seconds of latency. At a low-volume internal tool that doesn't matter. At any scale it does. But the cost I keep returning to isn't the dollar cost — it's the state cost. A swarm that gets lost accumulates context. The logs grow. The conversation history grows. The next agent, downstream, has to read all of it.

• Token cost grows linearly. We can bound it.
• Coordination cost grows quadratically. Every new agent must, in principle, reason about every other one.
• Trust cost grows in a way I can't measure. Each wrong corridor erodes the operator's belief that the system knows where it is.

Trust cost is the one that ends projects. Internal tools die when their operators stop trusting them. Public tools die when their users do. The model is good is not enough; the system must seem to know which corridor it is in.

Coda

I started this essay thinking about The House and ended it thinking about logs. That is, I think, fitting. The narrator of Piranesi keeps a journal — meticulous, dated, indexed. He doesn't know he is writing a map. We who build agent systems do know, and we still mostly fail to write the map, because the map is boring and the corridors are interesting.

Write the map.