Skip to content

Latest commit

 

History

History

vertexai-vector-search-firestore

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
src
src
 
 
.env.example
.env.example
 
 
README.md
README.md
 
 
package.json
package.json
 
 
tsconfig.json
tsconfig.json
 
 

README.md

Sample Vertex AI Plugin Retriever and Indexer with Firestore

This sample app demonstrates the use of the Vertex AI plugin retriever and indexer with Firestore for storing document content and metadata. This guide will walk you through setting up and running the sample.

Prerequisites

Before running this sample, ensure you have the following:

  1. Node.js installed.
  2. PNPM (Node Package Manager) installed.
  3. A deployed index to an index endpoint in Vertex AI Vector Search.
  4. A Firestore instance.

Getting Started

Step 1: Clone the Repository and Install Dependencies

Clone this repository to your local machine, and follow the instructions in the root README.md to build the core packages. This sample uses workspace:* dependencies, so they will need to be accessible.

Then

cd js/testapps/vertex-vector-search-bigquery && pnpm i

Step 3: Set Up Environment Variables

Ensure you have a deployed index in Vertex AI Vector Search.

Create a .env file in the root directory and set the following variables (see the .env.example as well if needed)

PROJECT_ID=your-google-cloud-project-id
LOCATION=your-vertex-ai-location
FIRESTORE_COLLECTION=your_firestore_collection_here
VECTOR_SEARCH_PUBLIC_DOMAIN_NAME=your-vector-search-public-domain-name
VECTOR_SEARCH_INDEX_ENDPOINT_ID=your-index-endpoint-id
VECTOR_SEARCH_INDEX_ID=your-index-id
VECTOR_SEARCH_DEPLOYED_INDEX_ID=your-deployed-index-id
GOOGLE_APPLICATION_CREDENTIALS=path-to-your-service-account-key.json

Step 4: Run the Sample

Start the Genkit server:

genkit start

Sample Explanation

Overview

This sample demonstrates how to define a custom document indexer and retriever using Cloud Firestore. It integrates with Vertex AI for indexing and retrieval of documents.

Key Components

  • Firestore Document Indexer: Stores documents in a Firestore collection
  • Firestore Document Retriever: Retrieves documents from the Firestore collection based on neighbor IDs.
  • Genkit Configuration: Configures Genkit with the Vertex AI plugin, setting up the project, location, and vector search index options.
  • Indexing Flow: Defines a flow for indexing documents.
  • Query Flow: Defines a flow for querying indexed documents.

Firestore Document Indexer

The firestoreDocumentIndexer function writes document content and metadata to a Firestore collection, and returns the generated Firestore document ids:

const firestoreDocumentIndexer: DocumentIndexer = getFirestoreDocumentIndexer(
  db,
  FIRESTORE_COLLECTION
);

Firestore Retriever

The firestoreDocumentRetriever function reads from the Firestore Collection and retrieves documents based on neighbor IDs:

const firestoreDocumentRetriever: DocumentRetriever =
  getFirestoreDocumentRetriever(db, FIRESTORE_COLLECTION);

Defining Flows

Two flows are defined: indexFlow for indexing documents and queryFlow for querying documents.

  • Index Flow: Converts text inputs to documents and indexes them.
  • Query Flow: Retrieves documents based on a query and returns the results sorted by distance.

Running the Server

The server is started using the startFlowsServer function, which sets up the Genkit server to handle flow requests.

startFlowsServer();

License

This project is licensed under the Apache License, Version 2.0. See the LICENSE file for details.

Conclusion

This sample provides a basic demonstration of using Vertex AI plugins with Genkit for document indexing and retrieval. It can be extended and adapted to suit more complex use cases and integrations with other data sources and services.

For more information, please refer to the official Firebase Genkit documentation.