One mega-prompt trying to handle billing, bug reports, refunds, and account questions at once is fragile. Splitting it behind a classifier that picks a single domain first took our wrong-tool rate from roughly one in five down to under one in twenty. The model stopped guessing which knowledge applied and started answering inside a narrow scope.

The router is cheap

The classifier does not need to be the expensive model. A small, fast model with a tight prompt returns one label. That label decides which system prompt, which tools, and which knowledge base the main model sees next.

const intent = await classify(message); // "billing" | "bug" | "account" | ...
const route = routes[intent] ?? routes.fallback;
const reply = await answer(message, route.systemPrompt, route.tools);

Keeping the router separate means I can swap the answering model without retraining anything, and the classifier can run on a model that costs a fraction as much.

Narrow scope, better answers

Each route carries only the context it needs. The billing route knows about plans and invoices and has a tool to look up a subscription. It does not carry refund policy or API docs. That smaller surface is exactly why the answers improve. The model is not choosing between six knowledge bases mid-sentence, it is working inside one.

A fallback that admits ignorance

There is always a fallback route for messages that do not fit. Instead of forcing a label, the fallback asks a clarifying question or offers a human. Forcing every message into the nearest bucket is how you get confident answers to questions nobody asked.

Routing gives you measurement

The quiet benefit is data. Because every conversation is tagged with an intent, I can see which domains the bot handles well and which ones land on a human nine times out of ten. That tells me where to write better docs, where the product is confusing, and where the bot should not be trying at all.

SELECT intent, count(*) AS total,
       avg(case when handed_off then 1 else 0 end) AS handoff_rate
FROM conversations
GROUP BY intent
ORDER BY handoff_rate DESC;

When the refund intent showed a 70 percent handoff rate, that was not a prompt problem. The refund flow itself was unclear, and fixing the product did more than any wording could. The router turned a vague sense that "the bot is bad at refunds" into a number I could act on.