InMemory Vector Embeddings in Semantic Kernel
My last two blogs on Semantic Kernel were focused on laying foundations of basics of Semantic Kernel. They primarily highlighted Dependency Injection and ChatCompletion services of Semantic Kernel.
In this article, we will go a step further and see how to implement InMemory vector embeddings for similarity search in Semantic Kernel.
In the year 2024, I published a two part article on vector based similarity search for Cosmos DB. You can find those articles here and here.
Fundamentally, the vectorization concept in Semantic Kernel is similar with what I highlighted in my Cosmos DB blogs on vectorization. The core idea is unchanged: data is transformed into embeddings to perform similarity search and reasoning. What changes is not the concept itself but how Semantic Kernel operationalizes these embeddings through in memory operations.
SetUp
Create a new console application and add the following packages
dotnet add package Microsoft.Extensions.DependencyInjection --version 10.0.2
dotnet add package Microsoft.SemanticKernel --version 1.72.0
dotnet add package Microsoft.SemanticKernel.Connectors.OpenAI --version 1.7.0
dotnet add package Microsoft.SemanticKernel.Connectors.InMemory --version 1.72.0
dotnet add package Microsoft.Extensions.Configuration.Json --version 10.0.3
dotnet add package Microsoft.Extensions.AI --version 10.3.0
Of the above the following two extensions are the most important
Microsoft.SemanticKernel.Connectors.InMemory
Microsoft.Extensions.AI
>> SemanticKernel.Connectors.InMemory is required for creating an InMemoryVectorStore to store vector embeddings directly in RAM
>> Microsoft.Extensions.AI exposes an interface IEmbeddingGenerator to create the necessary embeddings .
Code
Let's create some sample data. But before that, we need to define the structure.
using Microsoft.Extensions.VectorData;
public class Hotel
{
[VectorStoreKey]
public string HotelId { get; set; }
[VectorStoreData(IsIndexed = true)]
public string HotelName { get; set; }
[VectorStoreData]
public string Description { get; set; }
[VectorStoreVector(1536)]
public ReadOnlyMemory<float>? DescriptionEmbedding { get; set; }
[VectorStoreData]
public string[] Tags { get; set; }
[VectorStoreVector(1536)]
public ReadOnlyMemory<float>? TagListEmbedding { get; set; }
}
What we have above are
VectorStoreKey → This acts as a unique record identifier (primary key)
VectorStoreData → This is used to store metadata field and can be optionally indexed
VectorStoreVector(1536) → This is an embedding vector used for similarity search with specified dimension size. In our case we have the dimension size of 1536.
Data
Lets take example for Hotels and create some data with details.
private static List<Hotel> CreateHotelRecords()
{
var hotel = new List<Hotel>
{
new Hotel {
HotelId = "1",
HotelName = "Sea Breeze Resort",
Description = "Beachfront resort with ocean view rooms and seafood restaurant.",
Tags = new[] { "beach", "resort", "seafood", "luxury" }
},
new Hotel {
HotelId = "2",
HotelName = "City Central Hotel",
Description = "Modern hotel in downtown area close to shopping malls and nightlife.",
Tags = new[] { "city", "business", "shopping", "nightlife" }
},
new Hotel {
HotelId = "3",
HotelName = "Lakeview Retreat",
Description = "Peaceful retreat near the lake with spa and yoga facilities.",
Tags = new[] { "lake", "spa", "relaxation", "yoga" }
},
new Hotel {
HotelId = "4",
HotelName = "Desert Mirage Inn",
Description = "Boutique desert hotel with camel tours and sunset dining experience.",
Tags = new[] { "desert", "boutique", "sunset", "adventure" }
}
};
return hotel;
}
Configuration
We read the confguration file appsettings.json, that stores the values for the Endpoint, Deployment name and the ApiKey.
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false)
.Build();
Kernel
We start by building the kernel and adding the vectorstore to the kernel service container.
var builder = Kernel.CreateBuilder();
builder.Services.AddInMemoryVectorStore();
Next, we will add the EmbeddingGenerator to the kernel service container.
builder.Services.AddAzureOpenAIEmbeddingGenerator(
deploymentName: configuration["AppSettings:Embed_DeploymentName"],
endpoint: configuration["AppSettings:EndPoint"],
apiKey: configuration["AppSettings:ApiKey"]);
Note : AddAzureOpenAIEmbeddingGenerator is an experimental method. You will get the following warning and the code will not compile. You will have to explicitly disable the warning
To turn off the warning use : #pragma warning disable SKEXP0010
#pragma warning disable SKEXP0010
builder.Services.AddAzureOpenAIEmbeddingGenerator(
deploymentName: configuration["AppSettings:Embed_DeploymentName"],
endpoint: configuration["AppSettings:EndPoint"],
apiKey: configuration["AppSettings:ApiKey"]);
In the next step, build the kernel
var kernel = builder.Build();
Now comes the most important aspect i.e. creating embeddings
var embeddingGenerator = kernel.Services.GetRequiredService<IEmbeddingGenerator<string, Embedding<float>>>();
Earlier we had registered an embedding generator inside the Dependency Injection (DI) container. In the above code we are requesting the same service to be returned back through the IEmbeddingGenerator interface so that the embeddings could be generated and stored in a InMemoryVectorStore which is what we will do in the next step.
var vectorStore = new InMemoryVectorStore(new()
{
EmbeddingGenerator = embeddingGenerator
});
Now, we need to send the record collection to the InMemoryVectorStore vectoreStore that we created above. In our case the collection is the hotels object.
Ensure that the collection exists in the the memory vector store which is what the code below does
var collection = vectorStore.GetCollection<string, Hotel>("hotels");
await collection.EnsureCollectionExistsAsync();
var hotelRecords = CreateHotelRecords().ToList();
Note : At this stage we haven't updated the collection with the vector embedding values.
To create the embeddings we will have to traverse the collection and create the embeddings for each record in the collection.
foreach (var hotel in hotelRecords)
{
var descriptionEmbeddingTask = embeddingGenerator.GenerateAsync(hotel.Description);
var featureListEmbeddingTask = embeddingGenerator.GenerateAsync(string.Join("\n", hotel.Tags));
hotel.DescriptionEmbedding = (await descriptionEmbeddingTask).Vector;
hotel.TagListEmbedding = (await featureListEmbeddingTask).Vector;
}
In the next step we update the collection with the embedding values through the inbuilt Upsert method.
await collection.UpsertAsync(hotelRecords);
And that's it , we have created the InMemoryVector store and its equivalent vector embeddings.
We can test it through SearchAsync method of the vector collection.
var searchString = "I am looking for a hotel close to the ocean";
var searchVector = (await embeddingGenerator.GenerateAsync(searchString)).Vector;
var resultRecords = await collection.SearchAsync(
searchVector, top: 1, new()
{
VectorProperty = r => r.DescriptionEmbedding
}).ToListAsync();
We return only the TOP 1 result where the vector property of the search string is closet to the DescriptionEmbedding of the hotel collection that we had created earlier and the result is displayed in the console window.
Console.WriteLine("Search string: " + searchString);
Console.WriteLine("Result: " + resultRecords.First().Record.Description);
Console.WriteLine();
Conclusion
The article was an introductory article on how vector embeddings can be created and stored in memory. This approach is ok where the data is not significant or you don't have much concerns regarding the execution time.
In an upcoming article I will touch base on how you can you Azure SQL to store and retrieve those embeddings. Also in near future I would pen an article on how to integrate Azure AI search with vector embeddings.
Thanks for reading !!!
