Gemini API tool retrieval: which tools to defer
Gemini API tool retrieval entered preview on November 19. Defer long-tail tools, keep core and named tools loaded, and measure tokens before rollout.
Sean McLellan
Lead Architect & Founder

Google put tool retrieval for the Gemini API into preview on November 19, 2026, according to the Google announcement. Add a tool_search tool and set defer_loading: true on the tools you want held back. Gemini then sees only each deferred tool's name and description and loads a full parameter schema when a task needs it. The decision for teams building Gemini agents is which tools to defer, which to keep loaded, and whether a small tool list should use retrieval at all.
The feature works only in the Interactions API on v1beta, not in generateContent, so an agent built on that method cannot simply set the flag. Google's tool retrieval docs also say the API surface "may change before general availability."
What is Gemini API tool retrieval?
Tool retrieval lets Gemini search your tool library during a request and load only the schemas it needs. Google's reason is cost and accuracy. The docs say "a few dozen function declarations can consume tens of thousands of tokens on every request, and tool selection accuracy drops as the candidate list grows."
With server-side retrieval, three steps happen inside one API call. The model emits a tool_search_call naming the function it wants. The API returns a tool_search_result with that function's full schema. The model then emits a normal function_call, which your code runs as before. All three steps appear in interaction.steps. Google points to that as the place to debug tool names and descriptions.
There is no index to build or host. Google says the only code changes are the new tool and the flag on each deferred tool.
Which Gemini models support tool retrieval?
The docs list six models: gemini-4-carbon, gemini-4-neon, gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash. Google's announcement describes this as "Gemini 3.6 and later models, including all Gemini 4 models." Any other model returns the error deferred loading is not supported for this model.
If your agent still runs on an older model, retrieval is not available to it. Plan the model upgrade first. Our post on settings Gemini 3.6 Flash ignores covers one thing to check during that move.
Server-side, client-side or static loading
Google's docs compare three ways to give Gemini its tools. The right one depends on where your tool list lives and how big it is.

- Static loading sends every schema on every request. Google lists it as best for fewer than 20 stable tools.
- Server-side retrieval (
{"type": "tool_search"}plusdefer_loading) is the default mode. Gemini runs the search, and Google recommends it for latency-sensitive apps because "searches run inside one API call." - Client-side retrieval (
"execution": "client") hands the search to your app. You write the lookup, ranking and filtering, and each search costs one extra round trip. In exchange, tools don't have to be declared up front, so you can add one mid-conversation when a user connects an integration.
You can't mix the two retrieval modes in one request, and each request can include one tool_search entry. defer_loading works only with server-side retrieval.
Remote MCP servers fit the server-side mode. Set defer_loading: true on an mcp_server entry and every tool that server exposes loads on demand. The API runs the tool on the remote server and returns the call and result steps in the same response.
Does tool retrieval lower Gemini API costs?
It can, but not always. Tool retrieval has no separate charge, and tokens bill at the model's standard rates. The docs break down where tokens go:
- A deferred tool adds only its name and
short_description(or the first sentence of itsdescription) to input tokens. Its parameter schema is not billed until it loads. - Each server-side search triggers "an extra model call inside the request with the full context plus loaded schemas." Those tokens are reported in
total_tool_use_tokensand billed at the input rate. - A loaded schema stays in the conversation and bills as input on later turns. Loaded tools can't be unloaded.
Google states the trade-off directly: "With small libraries or very long contexts that search on every turn, static loading can cost less." The announcement adds: "Measure it against your own traffic."
Caching interacts with this too. Loaded schemas are appended later in the conversation rather than changing the start of the prompt, so implicit caching keeps working once the prefix meets the model's minimum cacheable size. But deferring tools shrinks the prompt, and the docs warn that this "can drop a medium prompt below that minimum size." A team that relies on cached prefixes should check prompt size after the change. Our comparison of Gemini and OpenAI caching costs shows how much caching can move a bill.
Which tools to defer and which to keep loaded
This section is BaristaLabs guidance built on Google's documented best practices.
Keep loaded: the tools used on most turns, and any tool named in your system instructions or returned by name in tool output. Google warns that the model may call a named tool "without searching first." The docs list Server side tool call context must be circulated when using deferred loading functions. as the error when, "in a stateless request, a tool_search_call step was omitted, or the model called a deferred tool by name without searching first."
Defer: the long tail. That means tools used for one workflow, one customer type or one integration, and large MCP servers you connect for occasional tasks.
Stay static: agents with fewer than 20 tools that rarely change, and very long conversations that would search on every turn.
Go client-side: when the tool list differs per user or tenant, or when you already keep a permission-filtered registry. Your app then decides which schemas a user can see before the model gets them. In our reading, that is the main reason to accept the extra round trip.
Two naming habits help retrieval find the right tool. Google suggests prefixing similar names by service (billing_get_customer, crm_get_customer) and naming the service at the start of short_description.
How to turn on tool retrieval without breaking multi-turn agents
Most failures in Google's error table come from conversation handling, not from the first request. A suggested rollout order:
- Switch the client to
v1betaand confirm the agent runs on one of the six supported models. - Add
{"type": "tool_search"}and setdefer_loading: trueon a few long-tail tools first. Keep fullparameterson each one, because a deferred tool without them can't be called after retrieval. - Send the full
toolsarray on every request, including follow-up turns that return afunction_result. The API doesn't keep tool declarations between turns. Droppingtool_searchon a later turn while tools are still deferred returns an error. - If you manage history yourself (
store=false), append every model step unchanged. Keeptool_search_callandtool_search_resultin order and preserve everysignaturefield. Dropping the call or a signature returns a 400 error. Dropping the result returns no error but silently removes the loaded schemas. - Check
interaction.stepsforfunction_callsteps on every response. Google notes thatstatusisn't alwaysrequires_actionwhen a function call comes back. - Handle the no-match case. A search that matches no deferred tool returns a text response, not a structured error, so your agent loop has to cope with plain text.
- Compare token counts before and after using
total_input_tokensandtotal_tool_use_tokenson real traffic, then decide whether to defer more.
If the agent already runs on the Interactions API, steps 3 and 4 are the likeliest to need changes. Our post on how a Gemini call becomes a stateful job covers that state model. For a similar change on OpenAI's side, see how Programmatic Tool Calling sequences tool search.
Why it matters for teams building Gemini agents
This is BaristaLabs interpretation. Small teams tend to add tools one integration at a time, and the per-request cost of that list grows with each one. Tool retrieval gives a supported way to keep a large library without paying for every schema on every call, and without building a vector index for tools. The trade-off is a preview API with stricter rules for replaying conversation history, and savings you have to measure yourself, since static loading can still cost less.
Recommended first step: pick one agent with more than 20 tools, defer only its long-tail tools in a test branch, replay a day of real requests, and compare total_input_tokens plus total_tool_use_tokens against the static version before turning it on in production. If you need help building or tuning that agent, see our AI development services.
Sources and scope
- Google, Introducing Tool Retrieval in the Gemini API, November 19, 2026.
- Google AI Studio documentation, Tool retrieval, accessed November 20, 2026.
- Gemini API documentation, Function calling: remote MCP, accessed November 20, 2026.
These sources describe how the preview works and how it bills. They don't publish token savings or tool-selection accuracy figures, and BaristaLabs did not run its own measurement for this post.
Gemini agent tools
Decide which Gemini tools to defer before you turn on tool retrieval
Share your agent's tool list and a sample of real requests. BaristaLabs can help split tools into always loaded, deferred and per-user, update the multi-turn handling, and compare token use against your current setup.
A list of your agent's tools and the model it runs on are enough to start.
Turn this idea into a pilot
Which workflow should go first?
Use the readiness check to compare impact, effort, risk, owner, and next step before requesting a review.
- 3-5 minutes
- Deterministic score
- No sensitive data
Practical AI Workflow Notes
Want more practical AI operations ideas?
Get short notes on applying AI inside real small-business workflows — from document handling and customer follow-up to internal reporting, compliance, and automation guardrails.