[Bug]: Error occured when Grounding Tools use DataStore with enable chunking mode. #694
Description
File Name
gemini/grounding/intro-grounding-gemini.ipynb
What happened?
Hi, I have an issue when using more than Grounding Tool class in GenerativeModel class. I have read the documentation regarding parse and chunking in Datastore in Vertex AI Agent Builder this link. I interested in capability of Layout Parser that could understand table in PDF file format and I create a Datastore and Search type App in Agent Builder with that parse mode.
I noticed that when choose Layout Parser, the chunking mode become mandatory and I don't really mind about this. However, when I use the Datastore ID in Gemini's grounding Tools. There's error occured and it is said as the relevant log output part. I have also tried a several others parse mode and it comes with the same error in every parse mode in Datastore that enabled chunking mode.
Here's the detail of code that I used and also the Datastore and Search type App configuration.
- Snippet code of Tools for Grounding Tool
from vertexai.generative_models import FunctionDeclaration, GenerativeModel, Part, Tool, Content, grounding
from vertexai.preview import generative_models as preview_generative_models
vertexai.init(project=PROJECT_ID, location='us-central1')
vertex_search_tool = Tool.from_retrieval(
retrieval=preview_generative_models.grounding.Retrieval(
source=preview_generative_models.grounding.VertexAISearch(
datastore=f"projects/{PROJECT_ID}/locations/global/collections/default_collection/dataStores/{DATASTORE_ID}"
),
)
)
model = GenerativeModel(
"gemini-1.0-pro",
generation_config={"temperature": 0}
)
chat = model.start_chat()
response = chat.send_message(PROMPT, tools=[vertex_search_tool])
-
Vertex AI Datastore settings or configuration
a. Data store type is unstructure with data source from Google Cloud Storage
b. File type that I put in Datastore is PDF
c. Default Document Parser : Layout Parser
d. Chunking Mode enabled automatically when I choosed Layout Parser.
e. Chunk size 500
f. Disabled "Include ancestor headings in chunks"
g. No exception file type -
Vertex AI Search type App settings or configuration
a. App Type is search (as mentioned above)
b. Search app configuration Content is Generic type
c. Enterprise edition features is enabled
d. Advanced LLM features is enabled
e. The Datastore used only one, that is the Datastore mentioned in point 2.
Detail of environment and library:
- Python version : 3.10.13
- SDK version of google-cloud-aiplatform library : 1.51.0
- SDK version of vertexai library : 1.49.0
I have tried it too in google-cloud-aiplatform version 1.50.0 and 1.49.0 and got the same error.
Is there any solution regarding this case or is there any important step that I missed? I hope you could give me solutions or insights for this problem. Thank a lot beforehand! =D
Best Regards
Relevant log output
---------------------------------------------------------------------------
_InactiveRpcError Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/google/api_core/grpc_helpers.py:65, in _wrap_unary_errors.<locals>.error_remapped_callable(*args, **kwargs)
64 try:
---> 65 return callable_(*args, **kwargs)
66 except grpc.RpcError as exc:
File /opt/conda/lib/python3.10/site-packages/grpc/_channel.py:1176, in _UnaryUnaryMultiCallable.__call__(self, request, timeout, metadata, credentials, wait_for_ready, compression)
1170 (
1171 state,
1172 call,
1173 ) = self._blocking(
1174 request, timeout, metadata, credentials, wait_for_ready, compression
1175 )
-> 1176 return _end_unary_response_blocking(state, call, False, None)
File /opt/conda/lib/python3.10/site-packages/grpc/_channel.py:1005, in _end_unary_response_blocking(state, call, with_call, deadline)
1004 else:
-> 1005 raise _InactiveRpcError(state)
_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "`extractive_content_spec` must be not defined when the datastore is using 'chunking config'"
debug_error_string = "UNKNOWN:Error received from peer ipv4:142.251.171.95:443 {grpc_message:"`extractive_content_spec` must be not defined when the datastore is using \'chunking config\'", grpc_status:3, created_time:"2024-05-16T11:21:52.51250561+00:00"}"
>
The above exception was the direct cause of the following exception:
InvalidArgument Traceback (most recent call last)
Cell In[8], line 1
----> 1 response = chat.send_message("Get a list of datasets", tools=[vertex_search_tool])
File /opt/conda/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:809, in ChatSession.send_message(self, content, generation_config, safety_settings, tools, stream)
802 return self._send_message_streaming(
803 content=content,
804 generation_config=generation_config,
805 safety_settings=safety_settings,
806 tools=tools,
807 )
808 else:
--> 809 return self._send_message(
810 content=content,
811 generation_config=generation_config,
812 safety_settings=safety_settings,
813 tools=tools,
814 )
File /opt/conda/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:905, in ChatSession._send_message(self, content, generation_config, safety_settings, tools)
903 while True:
904 request_history = self._history + history_delta
--> 905 response = self._model._generate_content(
906 contents=request_history,
907 generation_config=generation_config,
908 safety_settings=safety_settings,
909 tools=tools,
910 )
911 # By default we're not adding incomplete interactions to history.
912 if self._response_validator is not None:
File /opt/conda/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:496, in _GenerativeModel._generate_content(self, contents, generation_config, safety_settings, tools, tool_config)
471 """Generates content.
472
473 Args:
(...)
487 A single GenerationResponse object
488 """
489 request = self._prepare_request(
490 contents=contents,
491 generation_config=generation_config,
(...)
494 tool_config=tool_config,
495 )
--> 496 gapic_response = self._prediction_client.generate_content(request=request)
497 return self._parse_response(gapic_response)
File /opt/conda/lib/python3.10/site-packages/google/cloud/aiplatform_v1beta1/services/prediction_service/client.py:2103, in PredictionServiceClient.generate_content(self, request, model, contents, retry, timeout, metadata)
2100 self._validate_universe_domain()
2102 # Send the request.
-> 2103 response = rpc(
2104 request,
2105 retry=retry,
2106 timeout=timeout,
2107 metadata=metadata,
2108 )
2110 # Done; return the response.
2111 return response
File /opt/conda/lib/python3.10/site-packages/google/api_core/gapic_v1/method.py:113, in _GapicCallable.__call__(self, timeout, retry, *args, **kwargs)
110 metadata.extend(self._metadata)
111 kwargs["metadata"] = metadata
--> 113 return wrapped_func(*args, **kwargs)
File /opt/conda/lib/python3.10/site-packages/google/api_core/grpc_helpers.py:67, in _wrap_unary_errors.<locals>.error_remapped_callable(*args, **kwargs)
65 return callable_(*args, **kwargs)
66 except grpc.RpcError as exc:
---> 67 raise exceptions.from_grpc_error(exc) from exc
InvalidArgument: 400 `extractive_content_spec` must be not defined when the datastore is using 'chunking config'Code of Conduct
- I agree to follow this project's Code of Conduct