Spring AI 路由工作流程:先分類,再回答
Routing Workflow with Spring AI: Classify First, Then Answer
Spring AI 的 routing-workflow 範例以兩次模型呼叫分開處理輸入分類與回答生成,先選擇路由,再交由相應的專用提示詞作答。路由鍵須與路由表中的名稱完全相符,否則範例會拋出 IllegalArgumentException;文章建議限制輸出、規劃後備路由,並用標註資料集測試分類器。
Most LLM applications start with one prompt that tries to do everything. It classifies the request, answers it, and follows a policy, all in the same instruction. That works for a demo. In production, it becomes hard to tune, hard to test, and hard to explain when one kind of request gets a poor answer because the prompt was written for another kind.
The Routing Workflow pattern takes a different approach. Instead of one general prompt, you first ask the model to decide what kind of input it is looking at. Then you send the input to a prompt written specifically for that kind of input. Anthropic's article "Building Effective Agents" describes routing as one of the core workflow patterns for LLM systems, and the Spring AI team has published a compact example of it in the spring-ai-examples repository, under agentic-patterns/routing-workflow.
In this article I walk through the pattern, the code in the example, and the practical issues you will hit when you use it: how the classifier is built, why the route key has to match exactly, what happens when the model picks something unexpected, how to test the router, and where the pattern stops being the right tool. I will keep the code close to the example so you can follow along with the repository open.
What routing means in practice
Routing separates two decisions that a single prompt would otherwise make at once.
The first decision is classification: what kind of request is this? Is it a billing question, a technical problem, an account security issue, or a product question? The second decision is handling: given that it is a billing question, what should the response look like?
Separating them has three practical benefits.
- Specialized prompts. A billing prompt can focus on charges, refunds, and payment options without carrying instructions about password resets or feature education. Each prompt stays short and specific.
- Testability. You can evaluate the classifier on a labeled set of tickets and measure how often it picks the right route, separately from measuring the quality of the answers.
- Explainability. When an answer is wrong, you can ask whether the route was wrong or whether the route's prompt was wrong. Those are different problems with different fixes.
The cost is an extra model call per request, and a new failure mode: the classifier can pick the wrong route. The rest of this article is about managing both.
The pattern in one picture
The flow has three steps:
- The input arrives, along with a map of route names to specialized prompts.
- The model reads the input and selects one route, returning a short reasoning and the chosen route name.
- The input is sent to the selected route's prompt, and that response is returned to the caller.
Notice that the router makes two model calls per input: one to classify, and one to answer. That is the main trade-off. If you route cheap, simple requests to a smaller model, the classification step can be a meaningful part of the cost savings. If every request needs the most capable model anyway, routing adds latency and money without a clear gain.
The Spring AI example
The example's main class is RoutingWorkflow, in the package com.example.agentic. It takes a ChatClient in its constructor and exposes one public method:
public String route(String input, Map<String, String> routes)
The method takes the raw input and a map from route names to prompts. It returns the response from the selected route. The core logic is short:
public String route(String input, Map<String, String> routes) {
Assert.notNull(input, "Input text cannot be null");
Assert.notEmpty(routes, "Routes map cannot be null or empty");
String routeKey = determineRoute(input, routes.keySet());
String selectedPrompt = routes.get(routeKey);
if (selectedPrompt == null) {
throw new IllegalArgumentException("Selected route '" + routeKey + "' not found in routes map");
}
return chatClient.prompt(selectedPrompt + "\nInput: " + input).call().content();
}
Three details are worth noticing.
First, the route key returned by the classifier is looked up directly in the map. If the model returns a name that is not a key, the method fails with an IllegalArgumentException. That is a deliberate guard, and it matters a lot, as I will show later.
Second, the selected prompt is concatenated with the input using the same "\nInput: " separator used in the parallelization example. The route prompt ends with Input:, so the input follows it cleanly.
Third, the method does not return the classifier's reasoning. It only returns the final answer. If you need the reasoning for logging or auditing, you have to change the design so that it is returned or recorded.
The classifier
The interesting part is determineRoute:
private String determineRoute(String input, Iterable<String> availableRoutes) {
String selectorPrompt = String.format("""
Analyze the input and select the most appropriate support team from these options: %s
First explain your reasoning, then provide your selection in this JSON format:
\\{
"reasoning": "Brief explanation of why this ticket should be routed to a specific team.
Consider key terms, user intent, and urgency level.",
"selection": "The chosen team name"
\\}
Input: %s""", availableRoutes, input);
RoutingResponse routingResponse = chatClient.prompt(selectorPrompt).call().entity(RoutingResponse.class);
return routingResponse.selection();
}
This method does three things.
It builds a prompt that lists the available route names, asks for reasoning first, and asks for a JSON object with two fields. The order matters: asking for reasoning before the selection tends to make the selection more consistent with the explanation, because the model has written its argument before committing to the answer.
It asks Spring AI to convert the response into a RoutingResponse record, using the entity method:
public record RoutingResponse(String reasoning, String selection) {
}
With entity(RoutingResponse.class), Spring AI's structured output support sends a format description to the model and parses the reply into the record. You do not write the JSON parsing yourself, and you get a typed object back.
It returns only the selection. The reasoning is printed to the console in the example, which is useful for development but not something you would ship as-is.
There is a small detail in the template. The JSON braces are escaped as \\{ because the prompt is built with String.format, and the text block would otherwise interpret the braces in a way that causes trouble. If you rewrite this prompt, keep that in mind; it is an easy place to introduce a formatting bug.
Defining the routes
The example's Application class defines four routes as a map of names to prompts:
Map<String, String> supportRoutes = Map.of("billing", """
You are a billing support specialist. Follow these guidelines:
1. Always start with "Billing Support Response:"
2. First acknowledge the specific billing issue
3. Explain any charges or discrepancies clearly
4. List concrete next steps with timeline
5. End with payment options if relevant
Keep responses professional but friendly.
Input: """,
"technical", """
You are a technical support engineer. Follow these guidelines:
1. Always start with "Technical Support Response:"
2. List exact steps to resolve the issue
3. Include system requirements if relevant
4. Provide workarounds for common problems
5. End with escalation path if needed
Use clear, numbered steps and technical details.
Input: """,
"account", """
You are an account security specialist. Follow these guidelines:
...
Input: """,
"product", """
You are a product specialist. Follow these guidelines:
...
Input: """);
Each route prompt does four things well. It names a specific persona, sets a required opening line, lists the content the response should include, and ends with Input: so the route can append the ticket.
The required opening line is a useful trick. It gives you a cheap way to check, in tests or in monitoring, whether the right route handled the input. If a billing ticket comes back starting with "Technical Support Response:", something went wrong before the answer was written.
The example runs three tickets through the router: a login problem routed to account security, a small unexpected charge routed to billing, and a question about exporting data routed to the product route. The point is not that the classifier is perfect, but that each ticket goes to a prompt that matches its intent.
Why the route key has to match exactly
The most common failure in this pattern is a mismatch between what the model returns and what the map contains.
The classifier is given a list of route names and asked to pick one. The model may return Billing instead of billing, or billing team instead of billing, or Technical Support instead of technical. Any of those will fail the map lookup. In the example, the lookup returns null, and the method throws an exception that names the route it could not find.
You have three ways to handle this, from least to most robust.
Fail fast. This is what the example does. It is correct for development, because a mismatch is a bug you want to see. In production, it means a failed request, and you need a plan for it.
Normalize. Before the lookup, trim the string and compare it case-insensitively against the keys. This handles casing and whitespace, but not paraphrases like "billing team."
Constrain the output. Ask for an enum, or give the model an explicit list of allowed values and validate the result against it. Spring AI's structured output can be combined with an enum-typed field, which narrows the set of possible answers. The model can still pick a wrong route, but it cannot invent a new one.
In practice, I would combine the second and third approaches: constrain the output where possible, normalize the rest, and keep the fail-fast check as a final guard. A route that does not exist should never reach the handling step silently.
Handling unexpected or ambiguous inputs
Routing assumes that every input belongs to one of the known categories. Real inputs do not always cooperate. A ticket may mention a billing issue and a technical problem in the same message. Another may be spam, a test, or a request that none of the routes covers.
The example does not handle these cases, so you need to decide what the system should do. A few options are common.
Add a fallback route. A route named general or other with a prompt that asks the model to clarify or escalate catches anything the specialized routes do not cover. This is the simplest safety net, and it is the one that the README's own example uses.
Ask for a confidence value. Extend the classifier's record with a confidence field. If the confidence is below a threshold, send the input to a human queue or to the fallback route. The model's self-reported confidence is not a calibrated probability, so treat the threshold as a tuning parameter, not a guarantee.
Allow multiple routes. If the input touches several categories, the router can return a list of routes and run each handler, then merge the results. That turns the pattern into a small fan-out, which starts to resemble the parallelization workflow. It is a reasonable extension, but it changes the cost profile.
Whichever you choose, write it down as part of the design. An unhandled ambiguous input is a product decision, not a technical accident.
Testing the router
Routing lends itself well to testing, because there are two separate things to test and they can be tested separately.
The first is the classifier. Build a small labeled set of tickets, each with the route you expect, and measure how often the classifier picks that route. Run it whenever you change the route names, the classifier prompt, or the model. A small table of expected routes is more informative than a handful of spot checks.
The second is the handler. For each route, check that the response starts with the required opening line and follows the instructions in that route's prompt. Because the handler is a plain prompt, you can test it with a stubbed ChatClient that returns a fixed string, which checks the plumbing, and separately with a real model on a small set of inputs, which checks the content.
Here is a unit test for the plumbing, using a fake client that always selects the route you tell it to:
@Test
void routesToTheSelectedPrompt() {
ChatClient client = mock(ChatClient.class, RETURNS_DEEP_STUBS);
// The classifier call returns a RoutingResponse
when(client.prompt(contains("select the most appropriate support team")).call()
.entity(RoutingResponse.class))
.thenReturn(new RoutingResponse("Mentions a charge", "billing"));
// The handler call returns the answer for the selected prompt
when(client.prompt(contains("You are a billing support specialist")).call().content())
.thenReturn("Billing Support Response: ...");
RoutingWorkflow workflow = new RoutingWorkflow(client);
String result = workflow.route("I was charged twice", Map.of(
"billing", "You are a billing support specialist. Input: ",
"technical", "You are a technical support engineer. Input: "));
assertTrue(result.startsWith("Billing Support Response:"));
}
This test checks that the router sends the input to the prompt the classifier selected. It does not check whether the classifier is right, which is a question for the labeled evaluation set.
A test for the failure path matters just as much:
@Test
void failsWhenTheClassifierReturnsAnUnknownRoute() {
ChatClient client = mock(ChatClient.class, RETURNS_DEEP_STUBS);
when(client.prompt(anyString()).call().entity(RoutingResponse.class))
.thenReturn(new RoutingResponse("Unclear", "Billing Team"));
RoutingWorkflow workflow = new RoutingWorkflow(client);
assertThrows(IllegalArgumentException.class,
() -> workflow.route("Hello", Map.of("billing", "Billing prompt. Input: ")));
}
If someone later adds normalization, this test should change deliberately. Until then, it documents the current contract.
Prompt and model choices
The classifier and the handlers do not have to use the same model. The classifier does a narrow task: read a short input, compare it against a short list, and return one label. That is often well within the capability of a smaller, faster model. The handlers may need more reasoning and writing quality, so they may justify a larger model.
Spring AI lets you choose options per call. Set a low temperature on the classifier, because you want the same label for the same input, and a moderate temperature on the handlers, because you want fluent, varied responses. Verify these settings against the provider you use, since defaults differ.
Remember that the classifier's output is only as stable as its prompt. Changing a route name changes the classifier's behavior, so route names are part of the interface. Choose them carefully, keep them short and unambiguous, and treat a rename like a breaking change.
Cost and latency
Every routed request costs two calls, and the second call cannot start until the first returns, because the route decides which prompt to use. The latency is the sum of the two calls, not the maximum.
That affects the design in a few ways.
- If latency matters, use a fast model for the classifier, and consider whether the handlers can stream their output to the user.
- If cost matters, measure the share of requests that go to each route. If most requests go to one route, a simpler design with a default handler and an escape hatch may be cheaper.
- If the classifier is very accurate on a narrow set of inputs, you may be able to replace it with rules, such as keyword matching or a lookup table, for the most common cases, and use the LLM only for the rest. That is a hybrid design, and it is often the right answer in production.
Pitfalls
Overlapping routes. If two route descriptions cover the same kind of input, the classifier will be inconsistent. Write route names and descriptions so that each input has one clear best match, and test the boundary cases explicitly.
Routes that depend on each other. Routing decides where a request goes once. If the second step depends on the first step's output, you need a chain, not a router. The chain-workflow example in the same repository is the better fit for that case.
Route prompts that drift. Each route prompt is a separate piece of instruction that someone must maintain. Keep them in version control, review them like code, and keep the required opening lines consistent so that monitoring remains simple.
Trusting the reasoning. The classifier writes a justification, and it is tempting to treat it as proof. It is not. A fluent explanation can accompany a wrong label. Use the reasoning for debugging and review, and measure accuracy against labels.
Logging customer data carelessly. Support tickets often contain personal information. If you log the classifier's reasoning or the input, apply the same data handling rules you would apply to any customer record.
Ignoring the fallback. The example has no fallback route. In production, every input needs a destination, including the ones that match nothing you planned for.
When not to use this pattern
Routing is useful when inputs fall into a small number of categories that need different handling. It is not useful when every input needs the same handling, because then the classifier only adds cost.
If the handling itself needs to be decided step by step, with the model deciding what to do next based on results so far, you are looking at an orchestrator-workers design instead of a router. If the output of one step needs to be checked and improved before it is returned, an evaluator-optimizer loop fits better. And if several specialized answers are needed for the same input, a parallel fan-out is the right shape.
Where to go next
The example is small on purpose, and there are several natural extensions.
- Typed route names. Replace the string keys with an enum that both the classifier and the router use, so the compiler enforces the set of routes.
-
A fallback route. Add a
generalroute with a prompt that asks for clarification or escalation. - Confidence thresholds. Include a confidence field and route low-confidence inputs to a human queue.
- Labeled evaluation. Build a small dataset of tickets with expected routes, and run it in CI when you change prompts or models.
- Observability. Record the chosen route, the classifier's latency, and the handler's latency for each request, and track the route distribution over time.
- Hybrid classification. Handle the most common, unambiguous cases with rules, and use the model for the rest.
Conclusion
The Routing Workflow pattern separates the question "what is this?" from the question "how should we answer it?" That separation makes each part simpler to write, test, and explain. The Spring AI example shows the essential mechanics: a structured classifier, a map of specialized prompts, and a lookup that fails when the classifier returns something unexpected.
The pattern is only as good as its classifier and its route definitions. Choose route names that do not overlap, match them exactly, decide what happens to inputs that fit nowhere, and measure the classifier against labeled examples. If you do those things, routing turns one overloaded prompt into a set of focused ones, and each of them is easier to improve.
Key takeaways
- Routing separates classification from handling: the model first picks a route, then a specialized prompt answers.
- The Spring AI example makes two model calls per input, one to classify and one to answer.
- The classifier returns structured output, and the router looks up the selected route in a map.
- An unknown route name throws an exception in the example. Normalize or constrain the output in production.
- Every input needs a destination, so plan a fallback route and a way to handle ambiguous inputs.
- Test the classifier on labeled examples and the handlers with stubbed clients.
- Use routing when inputs fall into distinct categories that need different handling. Skip it when every input is handled the same way.
References
來源:DEV Community · dev.to