Skip to content

feat: add Azure AI Vector Search#1485

Merged
logan-markewich merged 25 commits intorun-llama:mainfrom
manekinekko:feat/add/azure-ai-search
Dec 18, 2024
Merged

feat: add Azure AI Vector Search#1485
logan-markewich merged 25 commits intorun-llama:mainfrom
manekinekko:feat/add/azure-ai-search

Conversation

@manekinekko
Copy link
Contributor

@manekinekko manekinekko commented Nov 14, 2024 •
edited
Loading

Vector Search Capabilities

  • Multiple search modes support:
    • Default (vector-only search)
    • Sparse (text-only search)
    • Hybrid (combined vector and text search)
    • Semantic Hybrid (with reranking)
  • Configurable vector search algorithms (HNSW and Exhaustive KNN)
  • Vector compression support (binary and scalar)
  • Customizable top-k results

Index Management

  • Three index management modes:
    • No validation
    • Validate index
    • Create if not exists
  • Automatic index schema generation
  • Configurable field mappings
  • Support for hidden fields
  • Language analyzer configuration

Document Operations

  • Batch document upload with size limits handling
  • Document deletion by:
    • Document ID
    • Node IDs
    • Metadata filters
  • Node retrieval with filtering
  • Metadata field mapping and filtering
  • Support for multiple metadata field types

Query Features

  • OData filter generation
  • Metadata filters support
  • Hybrid search capabilities
  • Semantic search with reranking
  • Async operation support

Customization Options

  • Configurable embedding dimensionality
  • Custom field mappings
  • Flexible metadata handling
  • Compression type selection
  • Search algorithm selection

@changeset-bot
Copy link

changeset-bot bot commented Nov 14, 2024 •
edited
Loading

⚠️ No Changeset found

Latest commit: 2626fb6

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel
Copy link

vercel bot commented Nov 14, 2024 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
legacy-llama-index-ts-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 18, 2024 5:22pm
llamaindex-ts-doc ❌ Failed (Inspect) Dec 18, 2024 5:22pm

@vercel
Copy link

vercel bot commented Nov 14, 2024

@manekinekko is attempting to deploy a commit to the LlamaIndex Team on Vercel.

A member of the Team first needs to authorize it.

@manekinekko manekinekko marked this pull request as ready for review November 16, 2024 13:29
@pkg-pr-new
Copy link

pkg-pr-new bot commented Nov 18, 2024 •
edited
Loading

Open in Stackblitz

@llamaindex/autotool

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/autotool@1485

@llamaindex/cloud

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/cloud@1485

@llamaindex/community

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/community@1485

@llamaindex/core

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/core@1485

@llamaindex/experimental

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/experimental@1485

@llamaindex/env

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/env@1485

llamaindex

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/llamaindex@1485

@llamaindex/node-parser

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/node-parser@1485

@llamaindex/readers

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/readers@1485

@llamaindex/wasm-tools

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/wasm-tools@1485

@llamaindex/workflow

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/workflow@1485

@llamaindex/anthropic

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/anthropic@1485

@llamaindex/clip

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/clip@1485

@llamaindex/groq

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/groq@1485

@llamaindex/deepinfra

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/deepinfra@1485

@llamaindex/huggingface

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/huggingface@1485

@llamaindex/ollama

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/ollama@1485

@llamaindex/openai

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/openai@1485

@llamaindex/portkey-ai

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/portkey-ai@1485

@llamaindex/replicate

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/replicate@1485

@llamaindex/vercel

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/vercel@1485

@llamaindex/vllm

npm i https://pkg.pr.new/run-llama/LlamaIndexTS/@llamaindex/vllm@1485

commit: 2626fb6

@manekinekko manekinekko force-pushed the feat/add/azure-ai-search branch from d2177bd to 391c1ad Compare November 18, 2024 16:12
manekinekko
manekinekko commented Nov 19, 2024
View reviewed changes
@manekinekko manekinekko force-pushed the feat/add/azure-ai-search branch from 0c70e9c to 3dc3583 Compare November 19, 2024 08:35
@manekinekko manekinekko force-pushed the feat/add/azure-ai-search branch from 3dc3583 to a811dd3 Compare November 19, 2024 13:19
@manekinekko manekinekko force-pushed the feat/add/azure-ai-search branch from e6f6c90 to 6470e10 Compare December 6, 2024 12:13
@logan-markewich
Copy link
Contributor

logan-markewich commented Dec 18, 2024

Docs can be fixed in another PR, I don't think this is related

@logan-markewich logan-markewich merged commit 09b933f into run-llama:main Dec 18, 2024
@manekinekko
Copy link
Contributor Author

manekinekko commented Dec 19, 2024

Thank you @logan-markewich @himself65 for helping reviewing and merging this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@himself65 himself65 himself65 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@manekinekko @logan-markewich @himself65