Tool Overload: When More Functions Make an Agent Worse, Not Better

Tool Overload: When More Functions Make an Agent Worse, Not Better

Teams building agents tend to treat tool access as purely additive: give the model another function and it becomes more capable, full stop. That holds for the first handful of tools. Past a certain roster size, it stops holding — tool selection accuracy degrades, the model calls a plausible-sounding tool that doesn't exist, or it reaches for the wrong one because two descriptions overlap just enough to blur the decision.

Why the Tool List Itself Is a Prompt

Every tool name, description, and parameter schema ships inside the context window exactly like any other prompt text, and it competes for the same attention budget as the rest of the instructions. The model isn't consulting an API reference and reasoning about which endpoint fits — it's doing next-token prediction over a list of options, the same way it predicts the next word in a sentence. A 40-item list where several entries have near-identical names (search_docs vs. search_knowledge_base) or vague, overlapping descriptions is asking the model to make a fine-grained distinction it was never given the information to make reliably. The failure doesn't look like "the model got dumber" — it looks like an off-by-one wrong tool call that's easy to misdiagnose as a model regression when the real cause is a crowded, ambiguous menu.

This gets worse under multi-turn use, where the tool list is re-sent on every turn and competes with a growing conversation history for the same limited attention, and worse again when several tools accept similar-shaped parameters, making it easy for the model to fill in the right shape with the wrong tool.

Practical Fixes: Namespace, Route, and Load Just-in-Time

A few patterns consistently help more than fine-tuning tool descriptions further:

• Namespace and route instead of flattening. Expose one search tool that internally routes to the right backend rather than five flat search variants — let deterministic code make the routing decision the model shouldn't have to.
• Load tools per phase, not globally. An agent doing code review doesn't need deployment or billing tools in its context at all; scoping the roster to the current task shrinks the decision space and removes an entire class of wrong-tool errors.
• Make descriptions mutually exclusive. If two tools are ever genuinely ambiguous to a human skimming the list, they'll be ambiguous to the model too — merge them or add explicit "use X when Y, not when Z" disambiguation.
• Eval tool selection separately from task success. A task can succeed despite a wrong intermediate tool call, which hides the regression until the roster grows enough that it doesn't. Track selection accuracy as its own metric so tool sprawl gets caught early.

The instinct to add "just one more tool" for a new capability is usually right in isolation and wrong in aggregate. Before the next addition, it's worth asking whether the roster needs trimming as much as it needs expanding — treating that as a real design tradeoff, not a rounding error.