Skip to content

fix(google): google provider codeExecution tool no longer treated as a client tool #11486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
psinha40898 wants to merge 7 commits into
base: main
Choose a base branch
from
Open

fix(google): google provider codeExecution tool no longer treated as a client tool #11486

psinha40898 wants to merge 7 commits into from
+61 −10

Conversation

@psinha40898
Copy link
Contributor

@psinha40898 psinha40898 commented Jan 1, 2026

Background

Provider level tools calls do not need another step. The AI SDK was waiting for another step because it was treating it like a client tool call. So for structured outputs, the SDK was waiting for another step to parse the JSON.

Summary

  1. hasToolCalls -> hasClientToolCalls because provider tool calls do not need another step but client tools do
  2. Only client tool calls (!part.providerExecuted,) are added. The Gemini API is not consistent when the provider level tool call adds stuff to parts, (other provider tools do not), so this seemed the best way to handle that.
  3. Remove explicit setting of hasToolCalls (now hasClientToolCalls) to true during provider level code execution

Manual Verification

import { generateText, Output } from "ai";
import { google } from "@ai-sdk/google";
import { z } from "zod";
import dotenv from "dotenv";
dotenv.config();

async function main() {
  const result = await generateText({
    model: google("gemini-3-flash-preview"),
    tools: {
      code_execution: google.tools.codeExecution({}),
    },
    output: Output.object({
      schema: z.object({
        answer: z.number(),
        explanation: z.string(),
      }),
    }),
    prompt:
      "Calculate the sum of the first 50 prime numbers. Make sure to use the python tool. Show your work.",
  });

  // Check if any step contained a tool call
  const usedCodeExecution = result.steps.some((step) =>
    step.toolCalls.some((tc) => tc.toolName === "code_execution")
  );

  console.log("Did Gemini use Python?", usedCodeExecution);
  console.log(result.output);
}

main();

Before (main):

image

After (PR branch):

image

Previously this only coincidentally worked if Gemini decided to complete the task without the code execution tool even though it was included. This is just non deterministic LLM stuff, the point is that it should also work when Gemini does use its provider side code execution tool.

Checklist

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

Related Issues

Fixes #11466

@psinha40898 psinha40898 changed the title fix(google): google provider codeExecution tool no longer treated as a client tool that needs another step fix(google): google provider codeExecution tool no longer treated as a client tool Jan 1, 2026
@AVtheking
Copy link
Contributor

AVtheking commented Jan 2, 2026

Have you tested this on other gemini models other than gemini-3 series ? If not than please do and see, as before 3 series it wasn't possible to have structured output with provider defined tools.

@psinha40898
Copy link
Contributor Author

psinha40898 commented Jan 2, 2026 •
edited
Loading

Have you tested this on other gemini models other than gemini-3 series ? If not than please do and see, as before 3 series it wasn't possible to have structured output with provider defined tools.

That hasn't changed. The user will receive the appropriate error from the API which is how it should work IMO

@ruchernchong
Copy link

ruchernchong commented Jan 6, 2026

Have you tested this on other gemini models other than gemini-3 series ? If not than please do and see, as before 3 series it wasn't possible to have structured output with provider defined tools.

Are you asking from the perspective of backward compatibility to models before Gemini 3 or getting them to work as well?

@lgrammel
Copy link
Collaborator

lgrammel commented Jan 6, 2026

Can you add your test code as an example under examples/ai-core/src/generate-text?

@psinha40898
Copy link
Contributor Author

psinha40898 commented Jan 6, 2026 •
edited
Loading

Can you add your test code as an example under examples/ai-core/src/generate-text?

I added examples for both text output and structured output. My reasoning stems from the Google API treating the combination as a new feature for new models.
Thank you for the review!

vercel[bot]
vercel bot reviewed Jan 6, 2026
View reviewed changes
Copy link
Contributor

@vercel vercel bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:

Comment on line 448 references old variable name 'hasToolCalls' instead of the renamed 'hasClientToolCalls'

View Details
📝 Patch Details 
diff --git a/packages/google/src/google-generative-ai-language-model.ts b/packages/google/src/google-generative-ai-language-model.ts
index 69a86a393..5ddb1d5e3 100644
--- a/packages/google/src/google-generative-ai-language-model.ts
+++ b/packages/google/src/google-generative-ai-language-model.ts
@@ -445,7 +445,7 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
               }
             }
 
-            // Process tool call's parts before determining finishReason to ensure hasToolCalls is properly set
+            // Process tool call's parts before determining finishReason to ensure hasClientToolCalls is properly set
             if (content != null) {
               // Process all parts in a single loop to preserve original order
               const parts = content.parts ?? [];

Analysis

The PR renamed the variable 'hasToolCalls' to 'hasClientToolCalls' to distinguish between client tool calls and provider-executed tool calls. The variable is declared on line 383, modified on line 612, and passed to the finish reason mapping on line 621. However, the comment on line 448 still referenced the old variable name 'hasToolCalls'. This is a documentation inconsistency that could confuse code readers. The fix updates the comment on line 448 to correctly reference 'hasClientToolCalls' to match the actual variable name used throughout the code.

@ruchernchong ruchernchong mentioned this pull request Jan 6, 2026
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vercel vercel[bot] vercel[bot] left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

ai/provider provider/community

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Gemini 3 models support structured output with code execution but SDK returns errors

4 participants

@psinha40898 @AVtheking @ruchernchong @lgrammel