fix(tools): convert image/svg+xml to text in LoadArtifactsTool

1wos · GWeale · copybara-github · commit f9be94c624bf · 2026-06-16T13:54:50.000-07:00
Merge #5694 **Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.** ### Link to Issue or Description of Change **1. Link to an existing issue (if applicable):** - Closes: #5693 **2. Or, if no issue exists, describe the change:** **Problem:** `_is_inline_mime_type_supported` in `LoadArtifactsTool` returns `True` for `image/svg+xml` via the `image/` prefix match at `src/google/adk/tools/load_artifacts_tool.py:32-36`, so SVG artifacts get forwarded to Gemini as inline image data. Gemini rejects every SVG MIME variant with `400 INVALID_ARGUMENT - Unsupported MIME type: image/svg+xml`, instead of being text-converted like CSV / JSON / XML are since #4028. Empirically verified against `gemini-2.5-flash` via `google-genai==1.75.0` that `image/svg+xml`, `image/svg`, `application/svg+xml` and `image/xml` are all rejected by Gemini, while `image/png/jpeg/webp/avif/gif`, `audio/*`, `video/*` and `application/pdf` are accepted. So SVG is the only sub-case under a "supported prefix" that needs special handling today. **Solution:** Add an explicit denylist for subtypes that match a supported prefix but that Gemini rejects, and route SVG through the existing text-fallback path. Same shape as the #4028 fix. `src/google/adk/tools/load_artifacts_tool.py`: - Add `_GEMINI_UNSUPPORTED_INLINE_SUBTYPES = frozenset({'image/svg+xml'})` and short-circuit `_is_inline_mime_type_supported` on a hit, so SVG falls through to `_as_safe_part_for_llm`'s text-decoding branch instead of being forwarded inline. - Add `'image/svg+xml'` to `_TEXT_LIKE_MIME_TYPES` so the fallback path utf-8 decodes the SVG markup into a text `Part`, instead of returning the binary placeholder (`[Binary artifact: ...]`). SVG is XML so this is a reasonable representation for the model. ```diff +_GEMINI_UNSUPPORTED_INLINE_SUBTYPES = frozenset({ + 'image/svg+xml', +}) _TEXT_LIKE_MIME_TYPES = frozenset({ 'application/csv', 'application/json', 'application/xml', + 'image/svg+xml', }) def _is_inline_mime_type_supported(mime_type): normalized = _normalize_mime_type(mime_type) if not normalized: return False + if normalized in _GEMINI_UNSUPPORTED_INLINE_SUBTYPES: + return False return normalized.startswith(_GEMINI_SUPPORTED_INLINE_MIME_PREFIXES) or ( normalized in _GEMINI_SUPPORTED_INLINE_MIME_TYPES ) ``` ### Testing Plan **Unit Tests:** - [x] I have added or updated unit tests for my change. - [x] All unit tests pass locally. Added `test_load_artifacts_converts_svg_to_text` in `tests/unittests/tools/test_load_artifacts_tool.py`, mirroring the existing `test_load_artifacts_converts_unsupported_mime_to_text` (CSV) shape. It asserts that after `load_artifacts_tool.process_llm_request` runs: - `artifact_part.inline_data is None` (SVG is no longer forwarded as inline image data) - `artifact_part.text == svg_bytes.decode('utf-8')` (the SVG markup is delivered as a text Part) ``` $ pytest tests/unittests/tools/test_load_artifacts_tool.py -v ============================= test session starts ============================== platform darwin -- Python 3.13.5, pytest-8.4.2 collected 8 items tests/unittests/tools/test_load_artifacts_tool.py::test_load_artifacts_converts_unsupported_mime_to_text PASSED tests/unittests/tools/test_load_artifacts_tool.py::test_load_artifacts_converts_base64_unsupported_mime_to_text PASSED tests/unittests/tools/test_load_artifacts_tool.py::test_load_artifacts_keeps_supported_mime_types PASSED tests/unittests/tools/test_load_artifacts_tool.py::test_load_artifacts_converts_svg_to_text PASSED tests/unittests/tools/test_load_artifacts_tool.py::test_maybe_base64_to_bytes_decodes_standard_base64 PASSED tests/unittests/tools/test_load_artifacts_tool.py::test_maybe_base64_to_bytes_decodes_urlsafe_base64 PASSED tests/unittests/tools/test_load_artifacts_tool.py::test_maybe_base64_to_bytes_returns_none_for_invalid PASSED tests/unittests/tools/test_load_artifacts_tool.py::test_get_declaration_with_json_schema_feature_enabled PASSED ========================= 8 passed, 1 warning in 3.61s ========================= ``` The 1 warning is from a pre-existing test that exercises a `[WIP]` feature flag (`JSON_SCHEMA_FOR_FUNC_DECL`) and is unrelated to this change. **Manual End-to-End (E2E) Tests:** Reproduced the underlying Gemini API behaviour outside ADK with the `google-genai` SDK that ADK depends on: ```python # google-genai==1.75.0, gemini-2.5-flash from google import genai from google.genai import types client = genai.Client(api_key="...") with open("logo.svg", "rb") as f: part = types.Part.from_bytes(data=f.read(), mime_type="image/svg+xml") client.models.generate_content( model="gemini-2.5-flash", contents=["Describe this.", part], ) ``` Before this PR, the same code path inside ADK (`LoadArtifactsTool` → `_as_safe_part_for_llm` → inline forward) ends in: ``` google.genai.errors.ClientError: 400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': 'Unsupported MIME type: image/svg+xml', 'status': 'INVALID_ARGUMENT'}} ``` After this PR, the SVG artifact is delivered to the model as a text `Part` containing the SVG markup, and the agent run completes successfully. Control formats (`image/png/jpeg/webp/avif/gif`, `audio/mpeg/mp3`, `video/mp4/webm`, `application/pdf`) are unchanged: still forwarded as inline data and accepted by Gemini. ### Checklist - [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document. - [x] I have performed a self-review of my own code. - [x] I have commented my code, particularly in hard-to-understand areas. - [x] I have added tests that prove my fix is effective or that my feature works. - [x] New and existing unit tests pass locally with my changes. - [x] I have manually tested my changes end-to-end. - [x] Any dependent changes have been merged and published in downstream modules. ### Additional context Local pre-commit (`isort`, `pyink`, `addlicense`, `end-of-file-fixer`, `trailing-whitespace`) all pass on the two edited files. Related issue history: - #4028 closed by `fdc98d5c`. Same shape of bug for `application/csv`. This PR is the missed sub-case under the `image/` prefix. Co-authored-by: George Weale <gweale@google.com> COPYBARA_INTEGRATE_REVIEW=#5694 from 1wos:fix/image-svg-xml-unsupported 7b0be51 PiperOrigin-RevId: 933282374
diff --git a/src/google/adk/tools/load_artifacts_tool.py b/src/google/adk/tools/load_artifacts_tool.py
@@ -35,10 +35,28 @@
     'video/',
 )
 _GEMINI_SUPPORTED_INLINE_MIME_TYPES = frozenset({'application/pdf'})
+# MIME subtypes that match a supported prefix above but that Gemini
+# rejects with 400 INVALID_ARGUMENT when sent as inline data. These
+# must fall through to the text-conversion path in
+# `_as_safe_part_for_llm` instead of being forwarded as inline image
+# data. Verified empirically against gemini-2.5-flash via
+# google-genai 1.69.0 on 2026-05-13.
+_GEMINI_UNSUPPORTED_INLINE_SUBTYPES = frozenset({
+    'image/svg',
+    'image/svg+xml',
+    'image/xml',
+})
 _TEXT_LIKE_MIME_TYPES = frozenset({
     'application/csv',
     'application/json',
+    'application/svg+xml',
     'application/xml',
+    # SVG/XML image variants are XML-based and Gemini rejects them as
+    # inline image data (see _GEMINI_UNSUPPORTED_INLINE_SUBTYPES above), so
+    # they fall through here and are delivered to the model as text.
+    'image/svg',
+    'image/svg+xml',
+    'image/xml',
 })
 
 if TYPE_CHECKING:
@@ -60,6 +78,8 @@ def _is_inline_mime_type_supported(mime_type: str | None) -> bool:
   normalized = _normalize_mime_type(mime_type)
   if not normalized:
     return False
+  if normalized in _GEMINI_UNSUPPORTED_INLINE_SUBTYPES:
+    return False
   return normalized.startswith(_GEMINI_SUPPORTED_INLINE_MIME_PREFIXES) or (
       normalized in _GEMINI_SUPPORTED_INLINE_MIME_TYPES
   )
diff --git a/tests/unittests/tools/test_load_artifacts_tool.py b/tests/unittests/tools/test_load_artifacts_tool.py
@@ -144,6 +144,53 @@ async def test_load_artifacts_keeps_supported_mime_types():
   assert artifact_part.inline_data.mime_type == 'application/pdf'
 
 
+@mark.asyncio
+@mark.parametrize(
+    'mime_type',
+    ['image/svg+xml', 'image/svg', 'application/svg+xml', 'image/xml'],
+)
+async def test_load_artifacts_converts_svg_to_text(mime_type):
+  """SVG/XML image variants are rejected by Gemini with 400 INVALID_ARGUMENT,
+  so they must fall through to the text-conversion path instead of being
+  forwarded as inline image data.
+  """
+  artifact_name = 'logo.svg'
+  svg_bytes = (
+      b'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 10 10">'
+      b'<circle cx="5" cy="5" r="4"/></svg>'
+  )
+  artifact = types.Part(
+      inline_data=types.Blob(data=svg_bytes, mime_type=mime_type)
+  )
+
+  tool_context = _StubToolContext({artifact_name: artifact})
+  llm_request = LlmRequest(
+      contents=[
+          types.Content(
+              role='user',
+              parts=[
+                  types.Part(
+                      function_response=types.FunctionResponse(
+                          name='load_artifacts',
+                          response={'artifact_names': [artifact_name]},
+                      )
+                  )
+              ],
+          )
+      ]
+  )
+
+  await load_artifacts_tool.process_llm_request(
+      tool_context=tool_context, llm_request=llm_request
+  )
+
+  artifact_part = llm_request.contents[-1].parts[1]
+  # The SVG must NOT be forwarded as inline image data &mdash; Gemini would 400.
+  assert artifact_part.inline_data is None
+  # And the original SVG markup is delivered as a text part instead.
+  assert artifact_part.text == svg_bytes.decode('utf-8')
+
+
 def test_maybe_base64_to_bytes_decodes_standard_base64():
   """Standard base64 encoded strings are decoded correctly."""
   original = b'hello world'
