Skip to content

Commit 1fc7e91

Browse files
westey-mwestey-m
authored
.Net: Switch to using interfaces for search instead of query objects. (#8690)
### Motivation and Context As part of the vector search adr, we have agreed to switch to a different design for exposing different search experiences. ### Description - Switch to individual interfaces for search. - Making vector search part of the collection interface ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
1 parent 6b92a18 commit 1fc7e91

31 files changed

Lines changed: 503 additions & 520 deletions

File tree

‎docs/decisions/00NN-vector-search-design.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -371,3 +371,7 @@ class AzureAISearchVectorStoreRecordCollection<TRecord>: IVectorStoreRecordColle
371371
```
372372

373373
## Decision Outcome
374+
375+
Chosen option: 4
376+
377+
The consensus is that option 4 is easier to understand for users, where only functionality that works for all vector stores are exposed by default.

‎dotnet/samples/Concepts/Memory/VectorSearch_Simple.cs‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
// Copyright (c) Microsoft. All rights reserved.
2+
3+
using Microsoft.SemanticKernel.Connectors.AzureOpenAI;
4+
using Microsoft.SemanticKernel.Data;
5+
using Microsoft.SemanticKernel.Embeddings;
6+
7+
namespace Memory;
8+
9+
/// <summary>
10+
/// A simple example showing how to ingest data into a vector store and then use vector search to find related records to a given string.
11+
///
12+
/// The example shows the following steps:
13+
/// 1. Create an embedding generator.
14+
/// 2. Create a Volatile Vector Store.
15+
/// 3. Ingest some data into the vector store.
16+
/// 4. Search the vector store with various text and filtering options.
17+
/// </summary>
18+
public class VectorSearch_Simple(ITestOutputHelper output) : BaseTest(output)
19+
{
20+
[Fact]
21+
public async Task ExampleAsync()
22+
{
23+
// Create an embedding generation service.
24+
var textEmbeddingGenerationService = new AzureOpenAITextEmbeddingGenerationService(
25+
TestConfiguration.AzureOpenAIEmbeddings.DeploymentName,
26+
TestConfiguration.AzureOpenAIEmbeddings.Endpoint,
27+
TestConfiguration.AzureOpenAIEmbeddings.ApiKey);
28+
29+
// Initiate the docker container and construct the vector store.
30+
var vectorStore = new VolatileVectorStore();
31+
32+
// Get and create collection if it doesn't exist.
33+
var collection = vectorStore.GetCollection<ulong, Glossary>("skglossary");
34+
await collection.CreateCollectionIfNotExistsAsync();
35+
36+
// Create glossary entries and generate embeddings for them.
37+
var glossaryEntries = CreateGlossaryEntries().ToList();
38+
var tasks = glossaryEntries.Select(entry => Task.Run(async () =>
39+
{
40+
entry.DefinitionEmbedding = await textEmbeddingGenerationService.GenerateEmbeddingAsync(entry.Definition);
41+
}));
42+
await Task.WhenAll(tasks);
43+
44+
// Upsert the glossary entries into the collection and return their keys.
45+
var upsertedKeysTasks = glossaryEntries.Select(x => collection.UpsertAsync(x));
46+
var upsertedKeys = await Task.WhenAll(upsertedKeysTasks);
47+
48+
var vectorSearch = collection as IVectorizedSearch<Glossary>;
49+
50+
// Search the collection using a vector search.
51+
var searchString = "What is an Application Programming Interface";
52+
var searchVector = await textEmbeddingGenerationService.GenerateEmbeddingAsync(searchString);
53+
var searchResult = await vectorSearch!.VectorizedSearchAsync(searchVector, new() { Limit = 1 }).ToListAsync();
54+
55+
Console.WriteLine("Search string: " + searchString);
56+
Console.WriteLine("Result: " + searchResult.First().Record.Definition);
57+
Console.WriteLine();
58+
59+
// Search the collection using a vector search.
60+
searchString = "What is Retrieval Augmented Generation";
61+
searchVector = await textEmbeddingGenerationService.GenerateEmbeddingAsync(searchString);
62+
searchResult = await vectorSearch!.VectorizedSearchAsync(searchVector, new() { Limit = 1 }).ToListAsync();
63+
64+
Console.WriteLine("Search string: " + searchString);
65+
Console.WriteLine("Result: " + searchResult.First().Record.Definition);
66+
Console.WriteLine();
67+
68+
// Search the collection using a vector search with pre-filtering.
69+
searchString = "What is Retrieval Augmented Generation";
70+
searchVector = await textEmbeddingGenerationService.GenerateEmbeddingAsync(searchString);
71+
var filter = new VectorSearchFilter().EqualTo(nameof(Glossary.Category), "External Definitions");
72+
searchResult = await vectorSearch!.VectorizedSearchAsync(searchVector, new() { Limit = 3, Filter = filter }).ToListAsync();
73+
74+
Console.WriteLine("Search string: " + searchString);
75+
Console.WriteLine("Number of results: " + searchResult.Count);
76+
Console.WriteLine("Result 1 Score: " + searchResult[0].Score);
77+
Console.WriteLine("Result 1: " + searchResult[0].Record.Definition);
78+
Console.WriteLine("Result 2 Score: " + searchResult[1].Score);
79+
Console.WriteLine("Result 2: " + searchResult[1].Record.Definition);
80+
}
81+
82+
/// <summary>
83+
/// Sample model class that represents a glossary entry.
84+
/// </summary>
85+
/// <remarks>
86+
/// Note that each property is decorated with an attribute that specifies how the property should be treated by the vector store.
87+
/// This allows us to create a collection in the vector store and upsert and retrieve instances of this class without any further configuration.
88+
/// </remarks>
89+
private sealed class Glossary
90+
{
91+
[VectorStoreRecordKey]
92+
public ulong Key { get; set; }
93+
94+
[VectorStoreRecordData(IsFilterable = true)]
95+
public string Category { get; set; }
96+
97+
[VectorStoreRecordData]
98+
public string Term { get; set; }
99+
100+
[VectorStoreRecordData]
101+
public string Definition { get; set; }
102+
103+
[VectorStoreRecordVector(1536)]
104+
public ReadOnlyMemory<float> DefinitionEmbedding { get; set; }
105+
}
106+
107+
/// <summary>
108+
/// Create some sample glossary entries.
109+
/// </summary>
110+
/// <returns>A list of sample glossary entries.</returns>
111+
private static IEnumerable<Glossary> CreateGlossaryEntries()
112+
{
113+
yield return new Glossary
114+
{
115+
Key = 1,
116+
Category = "External Definitions",
117+
Term = "API",
118+
Definition = "Application Programming Interface. A set of rules and specifications that allow software components to communicate and exchange data."
119+
};
120+
121+
yield return new Glossary
122+
{
123+
Key = 2,
124+
Category = "Core Definitions",
125+
Term = "Connectors",
126+
Definition = "Connectors allow you to integrate with various services provide AI capabilities, including LLM, AudioToText, TextToAudio, Embedding generation, etc."
127+
};
128+
129+
yield return new Glossary
130+
{
131+
Key = 3,
132+
Category = "External Definitions",
133+
Term = "RAG",
134+
Definition = "Retrieval Augmented Generation - a term that refers to the process of retrieving additional data to provide as context to an LLM to use when generating a response (completion) to a user’s question (prompt)."
135+
};
136+
}
137+
}

‎dotnet/src/Connectors/Connectors.AzureAISearch.UnitTests/AzureAISearchVectorStoreRecordCollectionTests.cs‎

Lines changed: 18 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -567,16 +567,15 @@ public async Task CanSearchWithVectorAndFilterAsync()
567567
var filter = new VectorSearchFilter().EqualTo(nameof(MultiPropsModel.Data1), "Data1FilterValue");
568568

569569
// Act.
570-
var searchResults = await sut.SearchAsync(
571-
VectorSearchQuery.CreateQuery(
572-
new ReadOnlyMemory<float>(new float[4]),
573-
new()
574-
{
575-
Limit = 5,
576-
Offset = 3,
577-
Filter = filter,
578-
VectorFieldName = nameof(MultiPropsModel.Vector1)
579-
}),
570+
var searchResults = await sut.VectorizedSearchAsync(
571+
new ReadOnlyMemory<float>(new float[4]),
572+
new()
573+
{
574+
Limit = 5,
575+
Offset = 3,
576+
Filter = filter,
577+
VectorFieldName = nameof(MultiPropsModel.Vector1)
578+
},
580579
this._testCancellationToken).ToListAsync();
581580

582581
// Assert.
@@ -608,16 +607,15 @@ public async Task CanSearchWithTextAndFilterAsync()
608607
var filter = new VectorSearchFilter().EqualTo(nameof(MultiPropsModel.Data1), "Data1FilterValue");
609608

610609
// Act.
611-
var searchResults = await sut.SearchAsync(
612-
VectorSearchQuery.CreateQuery(
613-
"search string",
614-
new()
615-
{
616-
Limit = 5,
617-
Offset = 3,
618-
Filter = filter,
619-
VectorFieldName = nameof(MultiPropsModel.Vector1)
620-
}),
610+
var searchResults = await sut.VectorizableTextSearchAsync(
611+
"search string",
612+
new()
613+
{
614+
Limit = 5,
615+
Offset = 3,
616+
Filter = filter,
617+
VectorFieldName = nameof(MultiPropsModel.Vector1)
618+
},
621619
this._testCancellationToken).ToListAsync();
622620

623621
// Assert.

‎dotnet/src/Connectors/Connectors.Memory.AzureAISearch/AzureAISearchVectorStoreRecordCollection.cs‎

Lines changed: 51 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ namespace Microsoft.SemanticKernel.Connectors.AzureAISearch;
2222
/// </summary>
2323
/// <typeparam name="TRecord">The data model to use for adding, updating and retrieving data from storage.</typeparam>
2424
#pragma warning disable CA1711 // Identifiers should not have incorrect suffix
25-
public sealed class AzureAISearchVectorStoreRecordCollection<TRecord> : IVectorStoreRecordCollection<string, TRecord>, IVectorSearch<TRecord>
25+
public sealed class AzureAISearchVectorStoreRecordCollection<TRecord> : IVectorStoreRecordCollection<string, TRecord>, IVectorizableTextSearch<TRecord>
2626
#pragma warning restore CA1711 // Identifiers should not have incorrect suffix
2727
where TRecord : class
2828
{
@@ -342,72 +342,84 @@ public async IAsyncEnumerable<string> UpsertBatchAsync(IEnumerable<TRecord> reco
342342
}
343343

344344
/// <inheritdoc />
345-
public IAsyncEnumerable<VectorSearchResult<TRecord>> SearchAsync(VectorSearchQuery vectorQuery, CancellationToken cancellationToken = default)
345+
public IAsyncEnumerable<VectorSearchResult<TRecord>> VectorizedSearchAsync<TVector>(TVector vector, Data.VectorSearchOptions? options = null, CancellationToken cancellationToken = default)
346346
{
347-
Verify.NotNull(vectorQuery);
347+
Verify.NotNull(vector);
348348

349349
if (this._firstVectorPropertyName is null)
350350
{
351351
throw new InvalidOperationException("The collection does not have any vector fields, so vector search is not possible.");
352352
}
353353

354-
string? queryText = null;
355-
string? filterString = null;
356-
var searchFields = new List<string>();
354+
if (vector is not ReadOnlyMemory<float> floatVector)
355+
{
356+
throw new NotSupportedException($"The provided vector type {vector.GetType().Name} is not supported by the Azure AI Search connector.");
357+
}
358+
359+
// Resolve options.
360+
var internalOptions = options ?? Data.VectorSearchOptions.Default;
361+
string? vectorFieldName = this.ResolveVectorFieldName(internalOptions.VectorFieldName);
362+
363+
// Configure search settings.
357364
var vectorQueries = new List<VectorQuery>();
358-
int limit = 3;
359-
int offset = 0;
360-
bool includeVectors = false;
365+
vectorQueries.Add(new VectorizedQuery(floatVector) { KNearestNeighborsCount = internalOptions.Limit, Fields = { vectorFieldName } });
366+
var filterString = AzureAISearchVectorStoreCollectionSearchMapping.BuildFilterString(internalOptions.Filter, this._storagePropertyNames);
361367

362-
if (vectorQuery is VectorizedSearchQuery<ReadOnlyMemory<float>> floatVectorQuery)
368+
// Build search options.
369+
var searchOptions = new SearchOptions
363370
{
364-
// Resolve options.
365-
var internalOptions = floatVectorQuery.SearchOptions ?? Data.VectorSearchOptions.Default;
366-
string? vectorFieldName = this.ResolveVectorFieldName(internalOptions.VectorFieldName);
367-
368-
// Configure search settings.
369-
filterString = AzureAISearchVectorStoreCollectionSearchMapping.BuildFilterString(internalOptions.Filter, this._storagePropertyNames);
370-
vectorQueries.Add(new VectorizedQuery(floatVectorQuery.Vector) { KNearestNeighborsCount = internalOptions.Limit, Fields = { vectorFieldName } });
371-
limit = internalOptions.Limit;
372-
offset = internalOptions.Offset;
373-
includeVectors = internalOptions.IncludeVectors;
374-
}
375-
else if (vectorQuery is VectorizableTextSearchQuery vectorizableTextQuery)
371+
VectorSearch = new(),
372+
Size = internalOptions.Limit,
373+
Skip = internalOptions.Offset,
374+
Filter = filterString,
375+
};
376+
searchOptions.VectorSearch.Queries.AddRange(vectorQueries);
377+
378+
// Filter out vector fields if requested.
379+
if (!internalOptions.IncludeVectors)
376380
{
377-
// Resolve options.
378-
var internalOptions = vectorizableTextQuery.SearchOptions ?? Data.VectorSearchOptions.Default;
379-
string? vectorFieldName = this.ResolveVectorFieldName(internalOptions.VectorFieldName);
380-
381-
// Configure search settings.
382-
filterString = AzureAISearchVectorStoreCollectionSearchMapping.BuildFilterString(internalOptions.Filter, this._storagePropertyNames);
383-
vectorQueries.Add(new VectorizableTextQuery(vectorizableTextQuery.QueryText) { KNearestNeighborsCount = internalOptions.Limit, Fields = { vectorFieldName } });
384-
limit = internalOptions.Limit;
385-
offset = internalOptions.Offset;
386-
includeVectors = internalOptions.IncludeVectors;
381+
searchOptions.Select.AddRange(this._nonVectorStoragePropertyNames);
387382
}
388-
else
383+
384+
return this.SearchAndMapToDataModelAsync(null, searchOptions, internalOptions.IncludeVectors, cancellationToken);
385+
}
386+
387+
/// <inheritdoc />
388+
public IAsyncEnumerable<VectorSearchResult<TRecord>> VectorizableTextSearchAsync(string searchText, Data.VectorSearchOptions? options = null, CancellationToken cancellationToken = default)
389+
{
390+
Verify.NotNull(searchText);
391+
392+
if (this._firstVectorPropertyName is null)
389393
{
390-
throw new NotSupportedException($"A {nameof(VectorSearchQuery)} of type {vectorQuery.QueryType} is not supported by the Azure AI Search connector.");
394+
throw new InvalidOperationException("The collection does not have any vector fields, so vector search is not possible.");
391395
}
392396

397+
// Resolve options.
398+
var internalOptions = options ?? Data.VectorSearchOptions.Default;
399+
string? vectorFieldName = this.ResolveVectorFieldName(internalOptions.VectorFieldName);
400+
401+
// Configure search settings.
402+
var vectorQueries = new List<VectorQuery>();
403+
vectorQueries.Add(new VectorizableTextQuery(searchText) { KNearestNeighborsCount = internalOptions.Limit, Fields = { vectorFieldName } });
404+
var filterString = AzureAISearchVectorStoreCollectionSearchMapping.BuildFilterString(internalOptions.Filter, this._storagePropertyNames);
405+
393406
// Build search options.
394407
var searchOptions = new SearchOptions
395408
{
396409
VectorSearch = new(),
397-
Size = limit,
398-
Skip = offset,
410+
Size = internalOptions.Limit,
411+
Skip = internalOptions.Offset,
399412
Filter = filterString,
400413
};
401-
searchOptions.SearchFields.AddRange(searchFields);
402414
searchOptions.VectorSearch.Queries.AddRange(vectorQueries);
403415

404416
// Filter out vector fields if requested.
405-
if (!includeVectors)
417+
if (!internalOptions.IncludeVectors)
406418
{
407419
searchOptions.Select.AddRange(this._nonVectorStoragePropertyNames);
408420
}
409421

410-
return this.SearchAndMapToDataModelAsync(queryText, searchOptions, includeVectors, cancellationToken);
422+
return this.SearchAndMapToDataModelAsync(null, searchOptions, internalOptions.IncludeVectors, cancellationToken);
411423
}
412424

413425
/// <summary>

‎dotnet/src/Connectors/Connectors.Memory.AzureCosmosDBMongoDB/AzureCosmosDBMongoDBVectorStoreRecordCollection.cs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -237,6 +237,12 @@ public async IAsyncEnumerable<string> UpsertBatchAsync(
237237
}
238238
}
239239

240+
/// <inheritdoc />
241+
public IAsyncEnumerable<VectorSearchResult<TRecord>> VectorizedSearchAsync<TVector>(TVector vector, VectorSearchOptions? options = null, CancellationToken cancellationToken = default)
242+
{
243+
throw new NotImplementedException();
244+
}
245+
240246
#region private
241247

242248
private async Task CreateIndexAsync(string collectionName, CancellationToken cancellationToken)

‎dotnet/src/Connectors/Connectors.Memory.AzureCosmosDBNoSQL/AzureCosmosDBNoSQLVectorStoreRecordCollection.cs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -360,6 +360,12 @@ async IAsyncEnumerable<AzureCosmosDBNoSQLCompositeKey> IVectorStoreRecordCollect
360360
}
361361
}
362362

363+
/// <inheritdoc />
364+
public IAsyncEnumerable<VectorSearchResult<TRecord>> VectorizedSearchAsync<TVector>(TVector vector, VectorSearchOptions? options = null, CancellationToken cancellationToken = default)
365+
{
366+
throw new NotImplementedException();
367+
}
368+
363369
#endregion
364370

365371
#region private

‎dotnet/src/Connectors/Connectors.Memory.Pinecone/PineconeVectorStoreRecordCollection.cs‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -227,6 +227,12 @@ await this.RunOperationAsync(
227227
}
228228
}
229229

230+
/// <inheritdoc />
231+
public IAsyncEnumerable<VectorSearchResult<TRecord>> VectorizedSearchAsync<TVector>(TVector vector, VectorSearchOptions? options = null, CancellationToken cancellationToken = default)
232+
{
233+
throw new NotImplementedException();
234+
}
235+
230236
private async Task<T> RunOperationAsync<T>(string operationName, Func<Task<T>> operation)
231237
{
232238
try

0 commit comments

Comments
 (0)