Tool Use in AI Agents: Function Calling, Structured Output, and the Reliability Problem

Two approaches

Structured function calling

The provider-API approach. The host application defines tools as JSON schemas; the model emits a structured response that contains the function name and the arguments as a JSON object; the host parses and executes. OpenAI introduced this in mid-2023; Anthropic and Google followed with their own implementations. The benefit is reliability: the API is responsible for producing valid JSON arguments matching the schema, and modern models do this with high accuracy.

Freeform ReAct-style tool use

The pre-API approach, still useful in cases where the structured API does not fit. The model is prompted to emit a specific text format (“Action: search[query=...]”); the host parses the format with regex or constrained decoding. Less reliable than structured function calling but more flexible: any text format the prompt describes is callable, including formats the API does not support.

In practice, most production agents in 2026 use structured function calling for the bulk of their tool inventory and freeform parsing for edge cases. The two are not mutually exclusive.

Tool definition anatomy

A tool exposed to a language model has four parts. Each contributes to the model’s ability to select and use the tool correctly.

• Name. A short identifier (snake_case by convention). The model uses the name to refer to the tool.
• Description. A one or two-sentence natural-language explanation of what the tool does and when to use it. The description has outsized influence on tool selection accuracy; vague descriptions produce vague selections.
• Parameter schema. A JSON schema describing each argument: name, type, whether required, allowed values, and a per-argument description. Tight schemas reduce hallucinated arguments.
• Optional examples. Some APIs (and most prompt designs) benefit from one or two example calls demonstrating canonical usage.

Anthropic’s tool-use documentation gives a particularly clear treatment of the description-quality argument: clear, specific tool descriptions improve tool-selection accuracy substantially compared to terse or generic descriptions.

The reliability problem

The defining engineering concern with tool use is hallucination: the model invents a tool that does not exist, calls a real tool with malformed arguments, or calls a tool when no tool was needed. Reliability has improved substantially across model generations (the published rate of well-formed function calls has risen on every major model family’s benchmark releases), but it has not been solved.

Three mitigations recur in production designs. Strict schema validation at the host: reject any call whose arguments do not match the schema, and feed the validation error back to the model with a retry instruction. Tool-use unit evals: a fixed set of representative tasks where the correct tool selection is known, run automatically on every prompt or model change. See the evaluation reference. Constrained decoding: at inference time, restrict the model’s output to valid JSON paths that match the schema, eliminating malformed-syntax errors at the source.

MCP (Model Context Protocol)

Anthropic introduced the Model Context Protocol in late 2024 as a standardised way to expose tools and data sources to language models. Rather than each provider defining its own function-calling shape and each tool author writing one integration per provider, MCP defines a common protocol: a tool implemented as an MCP server can be consumed by any MCP-aware client, across providers.

Adoption progressed through 2025 and 2026. The MCP specification is publicly maintained, and several major orchestration frameworks now support MCP servers as a first-class tool source. The conceptual contribution is decoupling: the model no longer needs to know how each tool is implemented, and the tool no longer needs to know which model is calling it.

Parallel tool calls

Modern frontier models routinely emit multiple tool calls in a single response. The host code executes them in parallel and returns all results in the next round. The architectural implication is meaningful: tasks that decompose into independent sub-queries (research a list of items, gather data from several APIs, run several validations concurrently) complete in roughly the time of the slowest call rather than the sum of all calls.

Parallel tool calling combines naturally with the multi-agent pattern: a supervisor agent can fan out work to several sub-agents in parallel, then aggregate. The parallelism is exposed at the tool-call level rather than requiring explicit threading in the host code.

Tool-use evaluation benchmarks

Evaluating tool-use reliability has become its own sub-discipline. The publicly tracked benchmarks include BFCL (Berkeley Function-Calling Leaderboard), tau-bench (multi-turn tool use, Sierra 2024), Nexus Function Calling Benchmark, and the tool-use subsets of AgentBench and SWE-bench. Each evaluates a different slice of the problem: simple tool selection, multi-turn tool use, parallel tool calling, error recovery. The companion site benchmarkingagents.com covers the benchmark landscape in depth.