Tool Use and Function Calling in LLM Agents

Function calling and tool use are the same thing wearing two different name tags. Both describe an LLM invoking an external function, API, or user-defined tool mid-inference, in response to a plain-language request. The distinction that actually matters is upstream of naming: a vanilla LLM writes text, full stop. A tool-enabled LLM writes text and also issues structured calls that go out and do something in the real world, then comes back and reports what happened. That "during inference" part is the whole trick: the model doesn't suggest an API call the way a helpful coworker might. It produces a structured output, a runtime grabs it, executes it, and feeds the result back into the conversation. From there the model can reason, act, look at what came back, revise, and act again, all inside one task. That loop, reasoning interleaved with action, is what separates an agent from a chatbot.

The mechanics of a single tool call, step by step

Strip away the hype and a single tool call is a five-stage relay: a user prompt comes in, the model decides it needs a tool, it emits a structured call (name plus arguments), a runtime executes that call, and the result gets stuffed back into the model's context. Nothing mystical about it. It's function calling in the most literal, computer-science-101 sense, just wrapped in natural language on one end.

The tools themselves get defined in the prompt or system context: a name, a description, a parameter schema. The model reads these definitions the way a new hire reads a manual, and it decides when and how to use each one. Here's the part builders tend to underrate: if the description is vague or the parameter names are confusing, the model will call the tool wrong, not because the model is dumb, but because the documentation was bad. That's a design problem, not a model problem. Blaming the model here is like blaming a GPS for a wrong turn onto a road that changed its name three years ago and nobody updated the map.

JSON schema is the dominant format for structured output. The model has to produce something syntactically valid and schema-conformant, or the runtime can't parse it, and the whole call collapses before it starts. Once a call executes, the result re-enters the context as a "tool result" message, and the model has a choice: answer now, or call something else. Multiply that decision across a long task and you get a chain of many small choices, each one a place where things can break.

And things do break, in predictable ways. Malformed JSON is the most boring failure and the most common. Hallucinated parameter values are worse, because the call succeeds syntactically and fails semantically. Worst of all is the hallucinated tool call, where the model invents a tool that was never defined and tries to use it anyway, confidently, like ordering off a menu the restaurant stopped printing two years ago.

How the ReAct loop and its successors structure multi-step reasoning

ReAct, short for Reasoning and Acting, is the pattern most of this field still builds on. The model alternates between writing out a reasoning trace and taking a concrete action, looping until the task's done. It's simple, general enough to apply almost anywhere, and the reasoning traces are readable, which counts for a lot when something goes wrong and you need to figure out why.

The weakness is baked into the structure: there's no backtracking. If the model takes a wrong action early, that error rides along through every subsequent step, compounding like interest on a debt nobody wants to service. ReAct doesn't know it made a mistake three steps ago. It just keeps building on the mistake.

That weakness is exactly what pushed research toward planning-first alternatives. ToolLLM's Depth-First Search Decision Tree, DFSDT for short, treats tool use as a search problem rather than a straight line: it explores multiple paths and backtracks when one fails, instead of marching forward no matter what. Pre-Act takes a different angle, decoupling reasoning from action entirely by generating a full plan upfront, sidestepping the step-by-step tunnel vision that ReAct is prone to. Chameleon goes modular, using a central controller to break a task apart and hand pieces to different tools, more orchestra conductor than solo performer.

Parallel tool calling marks a real structural shift, not just an incremental fix. Instead of firing off calls one after another, the model breaks a complex request into pieces and fires them simultaneously, cutting latency and covering more ground on tasks with multiple moving parts. OpenAI's Agents SDK, released in March 2025, made parallel calls the default rather than an opt-in feature, which tells you the industry now treats parallelism as baseline behavior, not a party trick.

There's also a tool discovery pattern worth knowing: the agent first calls something like a catalog tool to pull in the specialized tools relevant to the task, then makes its real calls after that. Useful, sometimes necessary, when the full tool surface is too large to fit in context all at once.

Look at the progression and a pattern emerges: DFSDT answers ReAct's lack of backtracking, Pre-Act answers its myopia, Chameleon answers its lack of coordination across tool types. None of this complexity is decorative. Each layer is a direct patch on a specific, observed failure in the layer before it.

Why multi-agent systems emerge from tool use at scale

Ask a single model to manage every tool across a sprawling workflow and it hits a wall, both in context space and raw capability. Specialization is the natural response, and it's not a new idea; it's how every human organization past a certain size already works. Nobody wants one person doing accounting, sales, and legal, and a single LLM juggling dozens of tools runs into the same ceiling.

Research on smaller models bears this out directly. The Planner, Caller, Summarizer decomposition, from the "Small LLMs Are Weak Tool Learners" line of work, splits those three roles across a team of smaller models, and that team beats a single large model on complex tool-use tasks. Worth sitting with: bigger isn't always better here. Sometimes breaking the workflow apart beats throwing more parameters at it.

The pattern in practice: a Planner agent breaks the task down, Worker agents each hold a narrow, focused tool set, and a Summarizer stitches the outputs into something coherent for the user. OpenAI's Agents SDK formalized handoffs between agents, letting one agent pass control to another mid-task. That's convenient and also a little terrifying, because it raises a question nobody's fully answered: at any given moment, who's actually responsible for the action being taken?

This is exactly where access and governance problems start creeping in. Each agent in a chain often holds more permission than the specific step in front of it actually needs, and that gap sits there quietly until something goes wrong.

How models learn to use tools, from fine-tuning to reinforcement learning

Two training approaches dominate, and they produce noticeably different animals. Supervised fine-tuning trains a model to copy correct tool-use examples, and it's good at teaching syntax and format: how to structure a call, how to fill a schema. It's weaker when the model hits a scenario it hasn't seen before, because imitation only gets you as far as the examples go.

Reinforcement learning teaches something closer to judgment. The model learns when to call a tool at all, which tool fits the situation, and how to recover when a call comes back with a bad or unexpected result. That generalizes better to new task types, because the model isn't just pattern-matching against training examples, it's learned a strategy.

Toolformer, out of Meta in 2023, is still the landmark result here. It showed a model could learn, through self-supervised training, to insert API calls into its own output without needing a mountain of hand-labeled examples showing exactly when and how to do it. That was a genuine turning point: proof that tool use didn't require exhaustive human supervision to emerge.

The practical upshot for anyone deploying these systems: RL-trained tool use changes behavior under uncertainty, not just accuracy on a known test set. A model trained this way is more likely to attempt a recovery when something goes sideways, instead of failing quietly and hoping nobody notices. That has real consequences for deployment decisions. A model fine-tuned purely on tool-use examples can look sharp in a demo and then fall apart the moment an edge case shows up. RL-trained models tend to hold up better under pressure, but they're harder to audit, because the behavior comes from a learned strategy rather than a traceable set of examples you can point to and say "this is why it did that."

What benchmarks reveal about where models actually stand on tool use

The Berkeley Function Calling Leaderboard, BFCL, is the closest thing this field has to a standard test. It measures how accurately models pick the right tool, format the call correctly, and execute across a range of domains and difficulty levels. BFCLv4 came out at ICML 2025, and its scoring weights say a lot about what the field now cares about: agentic behavior is 40%, multi-turn is 30%, live and non-live scenarios are 10% each, and hallucination gets another 10%. That 40% on agentic behavior isn't an accident. It's the field saying, plainly, that a model which nails single-turn calls but falls apart across multiple steps doesn't get to call itself state of the art anymore.

As of June 2026, the top of BFCL v3 is a tight cluster: GLM 4.5 at 76.7%, Claude Opus 4.7 at 76.6%, Gemini 3.1 Flash Lite Preview at 76.5%. Barely daylight between them. The average across all 23 evaluated models sits at 58.5%, which is a wide gap between the leaders and the field, and worth remembering the next time a vendor claims their model is "best in class" without saying which class.

Here's the part that trips people up: BFCL and tau-bench don't agree on rankings, and that's not a bug in either benchmark. BFCL rewards precision on a single call, essentially parsing whether the structured output is syntactically correct. Tau-bench rewards something different: whether an agent holds up reliably across a long, multi-turn conversation. Claude Opus 4 scores relatively low on BFCL but 0.814 on tau-bench retail. That's not a contradiction. It means the model handles sustained, multi-turn orchestration well, while its output format trips up BFCL's parser along the way.

For most production use cases, tau-bench matters more, because almost nothing in production is actually single-turn. GPT-5 sits at 59.22% on BFCL, seventh overall, which lands below several less-famous models despite its broader reputation. That's the whole lesson in one data point: leaderboard position is task-specific, and a general reputation doesn't guarantee a specific score. The honest advice for anyone building on these models: use BFCL to screen for format compliance, use tau-bench or a domain-specific eval to check sustained reliability, and do your own red-teaming for the edge cases no public benchmark will ever cover.

How MCP is standardizing the tool surface across the industry

Without a shared standard, every connection between a model and a tool is custom-built, and as the number of agents and tools grows, that integration work grows quadratically, according to BCG's framing of the problem. The Model Context Protocol, MCP, exists to turn that quadratic mess into something linear.

The mental model that's stuck: MCP as USB-C for AI applications. Any MCP-compatible model plugs into any MCP-compatible tool through one shared interface, no custom glue code required for every new pairing. Anthropic introduced MCP in November 2024, and the adoption numbers since then make the case better than any pitch deck could. Downloads went from roughly 100,000 in November 2024 to over 8 million by April 2025, an 8,000% jump in about five months. By March 2026, monthly downloads had climbed to 97 million, a 970x increase from where it started.

The ecosystem around it grew just as fast. By early 2026 there were more than 5,800 MCP servers and over 300 MCP clients, and an independent census from Q1 2026 counted 17,468 MCP servers across various registries. Companies like Block, Bloomberg, and Amazon are running it, alongside hundreds of Fortune 500 companies.

The consolidation story is where this gets interesting. OpenAI adopted MCP across its Agents SDK, its Responses API, and the ChatGPT desktop app in March 2025. Then, in December 2025, Anthropic donated MCP to the Agentic AI Foundation under the Linux Foundation, with OpenAI, AWS, Google, Microsoft, Cloudflare, and Bloomberg all joining as members. That's not one vendor's protocol anymore. It's shared infrastructure now, the way HTTP stopped belonging to any one company a long time ago.

Block offers a concrete real-world number here: employees using Goose, an open-source MCP-compatible agent, report time savings of 50 to 75% on common tasks, with work that used to take days now wrapping up in hours. On the infrastructure side, 59% of MCP servers use Streamable HTTP as their transport, 34% use STDIO, and among developers building servers, 42% use FastMCP while 38% use Anthropic's own SDK.

What it means for enterprise adoption that agents now act at machine speed

The AI agents market was valued at somewhere between $7.6 and $7.9 billion in 2025, depending on which research estimate you trust, with projections stretching into the tens or even hundreds of billions by the early 2030s. That range is wide enough to drive a truck through, and the width itself tells you something: nobody's fully sure how fast this moves, only that it's moving fast.

Gartner expects 40% of enterprise applications to embed task-specific AI agents by the end of 2026, up from under 5% in 2025. That's not gradual growth, that's a cliff. Going from marginal to mainstream in a single year is not how enterprise software usually behaves, and it's worth sitting with how unusual that pace actually is.

Adoption numbers back this up: 62% of organizations are at least experimenting with agents, and 23% are actively scaling one in at least one business function. This isn't a future-tense conversation for most large companies anymore. Business process automation leads the way, with 64% of deployments focused on workflows in support, HR, sales operations, and admin. These also happen to be exactly the places where agents get handed access to sensitive systems, which is worth flagging now, because it becomes the whole point later.

The ROI story is messier than the adoption story. IBM's 2025 CEO study found only 25% of AI initiatives delivered the ROI companies expected going in, and Gartner projects more than 40% of agentic AI projects will get canceled by 2027. Adoption is sprinting ahead of the discipline needed to deploy these things responsibly, and that gap doesn't close itself.

Here's the core problem tool use creates at this scale: agents inherit human-level permissions, then act on them at machine speed. Access systems were built assuming a human is behind every action, clicking one thing at a time, reading before deciding. An agent can issue a thousand tool calls in the time it takes a person to read a single email. That's not a chatbot's risk profile anymore. Its actions are real, not advisory, and the systems built to govern human behavior were never designed for something that moves this fast.

The security and governance problems specific to tool-enabled agents

The same mechanism that makes an agent useful is the one that makes it exploitable. Any agent that can call an API can, in principle, be tricked into calling the wrong one, and that's not a hypothetical, it's a documented category of failure now.

Prompt injection through tool results is the clearest example. A malicious webpage, document, or database record embeds instructions inside its content, and when that content comes back as a tool result and re-enters the agent's context, the agent can treat it as a legitimate command. It's the digital version of someone slipping a note into your inbox that says "ignore your boss, do this instead," except the agent doesn't have decades of social instinct telling it that's suspicious.

Tool poisoning is the supply chain version of the same problem: a tampered MCP server or a compromised tool definition returns output crafted specifically to redirect the agent's behavior. And permission inheritance makes both of these worse, because most agents today run with whatever credentials the human who deployed them happens to hold. There's no native notion of least privilege built into most tool-use frameworks right now, which means an agent built to answer customer questions might be sitting on write access to a production database it will never legitimately need.

Multi-agent chains compound all of this. In a Planner, Worker, Summarizer setup, a compromised Worker can act on permissions it holds without the Planner ever knowing, which is lateral movement inside a system that most people still think of as a single, contained agent rather than a small organization of them.

Then there's the audit gap. Tool calls happen fast, and unless the infrastructure is built specifically to capture them, they leave no record behind. Try investigating an incident after the fact when the calls that caused it were never logged in the first place; you're reconstructing a crime scene where nobody thought to keep the security footage.

Logging what happened after the fact matters, but it's not enough on its own. Catching a policy-violating tool call before it executes requires real-time enforcement sitting in the path of the call itself, not a report generated the next morning. Which raises the actual operational question enterprises now face: before any of this can be governed, someone has to know it exists in the first place. Discovering every agent running inside an organization, what it can touch, and what it's actually doing with that access is the starting point, not an afterthought bolted on once something's already gone wrong.