fix(google): add fallback thought signature sentinel for foreign function calls#6420
Open
igui wants to merge 1 commit into
Open
fix(google): add fallback thought signature sentinel for foreign function calls#6420igui wants to merge 1 commit into
igui wants to merge 1 commit into
Conversation
…gn function calls Gemini 2.5+ rejects multi-turn requests (HTTP 400 INVALID_ARGUMENT) when a function_call part has no thought_signature, which happens whenever the ChatContext contains tool calls produced by another provider — e.g. when google.LLM is a fallback in FallbackAdapter behind a non-Google LLM. Use Google's documented validator-skip sentinel for call_ids without a stored signature, keep real signatures unchanged, and continue omitting the field entirely for models that don't require thought signatures. Also tolerate a missing/None signature cache on the LLM instance. Fixes livekit#6135
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6135
Reimplementation of #6138 — that PR is stuck because its author was unable to sign the CLA, so this rewrites the same fix from scratch (I filed the original issue and have signed the CLA). Credit to @nightcityblade for the original implementation and review iterations.
Summary
When
google.LLM(Gemini 2.5+/3) sits behind another provider in aFallbackAdapter, the ChatContext can contain function calls produced by a non-Google model. Those calls have no stored thought signature, and Gemini rejects the request with HTTP 400INVALID_ARGUMENT("Function call is missing a thought_signature in functionCall parts").skip_thought_signature_validatorsentinel for anyfunction_callpart whosecall_idhas no stored thought signature.None_thought_signaturescache on theLLMinstance, both when formatting the request and when storing a returned signature.Nonecache sends the sentinel; pre-2.5 omits the field even with a cached signature).Note: per Google's docs the sentinel is a last resort and can degrade model performance around function calls; it is only ever used for call_ids the Gemini API did not generate, where the alternative is a hard 400.
Tests
uv run ruff check/uv run ruff format --checkon the touched filesuv run pytest tests/test_google_thought_signatures.py tests/test_plugin_google_llm.py -q— 55 passeduv run pytest tests/test_chat_ctx.py tests/test_tokenizer_xml_markup.py tests/test_expr_markup.py -q— 106 passed, 1 skipped