# Chat History with Redis, ChatReducer and SummarizingChatReducer in Microsoft Agent Framework My previous article on [Chat History in Microsoft Agent Framework](https://www.azureguru.net/chat-history-in-microsoft-agent-framework) focused on storing and retrieving chat history using a file-based approach.The intention of that article was to provide a simple introduction about implementing a basic chat history mechanism in Microsoft Agent Framework (MAF). While a file-based approach is useful for understanding the underlying concepts it is definitely not an ideal choice for most real-world production set ups. As the number of users and conversations grows, storing chat history in file systems can be very challenging with added complexities. In a production environment, chat history needs to be stored on a persistent data store so that it can be accessed across multiple instances of an application. This is where a distributed data stores such as **Redis** becomes useful. It provides a fast and scalable mechanism for storing messages and the maintain chat history. But just simply storing the entire conversation indefinitely introduces another important challenge: **the size of the conversation context**. As a conversation becomes longer, sending the complete chat history to the underlying agents on every request increases token consumption, latency and cost. It can also eventually exceed the model's context window. Though MAF provides **ChatReducer** to address this problem, one particularly useful implementation is **SummarizingChatReducer** that reduces growing conversation by summarizing older messages unlike ChatReducer that simply discards them. This helps to preserve important conversation context in place. This article will demonstrate storing conversation to Redis and how to leverage SummarizingChatReducer to summarize and reduce long conversations while at the same time maintain the conversation context. ### **Custom Chat History Provider** In MAF, the most crucial component is the **ProviderSessionState** object wrt custom chat history implementation. It acts as the abstraction layer through which MAF manages and retrieves chat history through session **StateBag.** In my previous [article](https://www.azureguru.net/chat-history-in-microsoft-agent-framework) , I used **IChatReducer** to reduce chat size stored in **ProviderSessionState .** But drawback with this approach is that you lose the conversation context as the reduction is based on a fixed number (`N`) of messages. Because of this, important aspects of the conversation is discarded. The agent therefore has access only to the most recent `N` messages. Given the drawback of the above approach a more feasible and practical approach is to leverage **SummarizingChatReducer** where values for **targetCount** and **threshold** properties define the number of messages that are summarized and stored so that correct conversational context is maintained. > Property targetCount is the used to specify the number of recent messages that should be retained post reduction while threshold is the number of messages allowed beyond targetCount before summarization. When the conversation grows beyond **targetCount + threshold** values , older messages are summarized into a single message that persists the entire conversational context. This message replaces the old messages which are discarded and the most recent messages defined by targetCount stays. For example : Lets assume that SummarizingChatReducer has targetCount of 1 and threshold of 5 and the no of prompts sent are 6. Once the total number of chat messages recorded in **ProviderSessionState** is equal or greater than 6 (1+5), summarization kicks in. 1 recent message is retained post summarization in its original form and all messages are summarized. There are two major approaches for storing and retrieving summarized conversational context. * In the first approach , you store the entire conversation along with summarized messages in a persistent storage like Redis and retrieve only the summarized messages in the **ProvideChatHistoryAsync** method. The major advantage with this approach is that you have the true conversation history at your disposal but disadvantage being that the size of the persistent storage would grow and managing it can be challenging. * In the second approach ,only the summarized messages are stored in a persistent storage like Redis along with all the messages above the limit set by the targetCount value. Major advantage with this approach is that you have less data to manage but disadvantage is that you lose all historical trails of the conversation as only the summarized conversational context is available and stored. ### Redis You can either use Redis on Azure or use a Redis image on Docker. I am using Docker. Spin up Redis on Docker through following command in the Docker terminal. ```yaml docker run -d --name local-redis -p 6379:6379 redis:latest ``` ![](https://cdn.hashnode.com/uploads/covers/6693c62c166ee9c594cffda0/3b7307e7-b44d-44be-a7a9-18fb3a666677.png align="center") Redis is running on port number : **6379** ### **Project SetUp** Create a new console application and add the following packages ```csharp dotnet add package Azure; dotnet add package Azure.AI.OpenAI; dotnet add package Microsoft.Agents.AI; dotnet add package Microsoft.Extensions.AI; dotnet add package StackExchange.Redis; dotnet add package System.Text.Json.Serialization; dotnet add package Microsoft.Extensions.Configuration; dotnet add package Microsoft.Extensions.DependencyInjection; dotnet add package Microsoft.Extensions.Logging; ``` Add **appsetting.json** to the project ```csharp "AppSettings": { "Chat_DeploymentName": "Deployment Name", "EndPoint": "Azure OpenAI endpoint", "ApiKey": "Azure OpenAI API key" } ``` ### **Code** Once all the artifacts in place, add the following code to **Program.cs** **Program.cs >>** ```csharp var configuration = new ConfigurationBuilder() .SetBasePath(Directory.GetCurrentDirectory()) .AddJsonFile("appsettings.json", optional: false) .Build(); ``` **Create a DI container** ```csharp ServiceCollection servicecollection = new ServiceCollection(); ``` **Read credentials and register** `Chatclient` ```csharp var credential = new AzureKeyCredential(configuration["AppSettings:ApiKey"]); servicecollection.AddKeyedChatClient( "ChatClient", ( sp => new AzureOpenAIClient(new Uri(configuration["AppSettings:EndPoint"]), credential) .GetChatClient(configuration["AppSettings:Chat_DeploymentName"]) .AsIChatClient() ) ); ``` **Register ChatClientAgent in the DI container and build the Service Provider** ```csharp servicecollection.AddSingleton(); ServiceProvider serviceprovider = servicecollection.BuildServiceProvider(); ``` **Fetch keyed IChatclient from the DI container** ```csharp var _chatclient = new ChatClientAgent(serviceprovider.GetKeyedService("ChatClient")).ChatClient; ``` **Set SummarizingChatReducer properties** ```csharp SummarizingChatReducer summaryReducer = new SummarizingChatReducer( chatClient: _chatclient, targetCount: 2, threshold: 3); ``` Here, the **targetCount** is set to 2 and the **threshold** is set to 3. This means that the conversation can grow to 5 (2+3) messages before summarization is triggered. Once threshold reaches 5 (2+3), keep 2 (targetCount) most recent messages in its raw form and summarize all the messages. **ChatClientAgentOptions and ChatHistoryProvider** ```csharp var options = new ChatClientAgentOptions { ChatOptions = new ChatOptions { Instructions = "You are a helpful chat assistant." }, ChatHistoryProvider = new RedisChatHistoryProvider(summarizingChatReducer: summaryReducer, targetCount: 2, threshold: 3) }; ``` > Why am I passing targetCount and threshold values to the constructor of CustomChatReducer(RedisChatHistoryProvider) when these values have been already configured for SummarizingChatReducer ? > > This is because, SummarizingChatReducer does not expose targetCount and threshold as publicly accessible properties. Also I am passing summarizingChatReducer instance as constructor argument. **Agent and Session** ```csharp var agent = _chatclient.AsAIAgent(options); var agentsession = await agent!.CreateSessionAsync(); ``` Define CustomChatHistoryProvider (**RedisChatHistoryProvider)** **RedisChatHistoryProvider.cs >>** Define properties of session state in a class called **SessionState.** We have defined \_targetCount , \_threshold and \_summarizingChatReducer because we want these properties to be part of **InvokingContext** through **ProviderSessionState** (explained later).These properties can be accessed through the **ProvideChatHistoryAsync** and **StoreChatHistoryAsync** methods. ```csharp internal class SessionState { public int _targetCount { get; set; } = 0; public int _threshold { get; set; } = 0; public SummarizingChatReducer _summarizingChatReducer { get; set; } = null; [JsonPropertyName("Messages")] public List lstChatMessages { get; set; } = []; [JsonPropertyName("UserName")] public string UserName { get; set; } = ""; } ``` **Redis Connection >>** ```csharp ConnectionMultiplexer redis = ConnectionMultiplexer.Connect("localhost:6379"); private IDatabase db; ``` **RedisChatHistoryProvider Constructor >>** We create an instance of ProviderSessionState for storing and handling the SessionState. ```csharp private readonly Microsoft.Agents.AI.ProviderSessionState _sessionstate; public RedisChatHistoryProvider(Func initializer = null, string statekey = "", SummarizingChatReducer summarizingChatReducer = null, int targetCount = 0, int threshold = 0) { _sessionstate = new(initializer => { db = redis.GetDatabase(); string messages = db.StringGet(statekey); return new SessionState { lstChatMessages = String.IsNullOrEmpty(messages) ? Enumerable.Empty().ToList() : JsonSerializer.Deserialize>(messages), _targetCount = targetCount, _threshold = threshold, _summarizingChatReducer = summarizingChatReducer, UserName = statekey }; } , statekey = "Sachin" ); } ``` > I have hardcoded statekey value to "Sachin".You can use any other uniquely identifiable value for statekey. In the above code, the constructor parameters targetCount, threshold ,summarizingChatReducer are respectively assigned to the Session state properties of \_targetCount, \_threshold and \_summarizingChatReducer. This way these properties are now part of the **InvokingContext** of the overridden methods ProvideChatHistoryAsync and StoreChatHistoryAsync. **ProviderSessionState** is used for initializing and persisting provider state in the session's StateBag. State(in our case it is SessionState object) is a inbuild session state management that maintains conversation history and is stored in the StateBag using the StateKey property as the key. As mentioned earlier, we can use either of two approaches: store the entire conversation in Redis and retrieve only the relevant summarized/reduced messages when required or store only the summarized messages in Redis. We will use the second approach, where we store only the summarized messages along with the number of messages specified by targetCount in Redis. **StoreChatHistoryAsync >>** ```csharp protected async override ValueTask StoreChatHistoryAsync(InvokedContext context, CancellationToken cancellationToken = default) { var cntx = _sessionstate.GetOrInitializeState(context.Session); IEnumerable reducedMessages = new List(); var allNewMessages = context.RequestMessages.Concat(context.ResponseMessages ?? []); cntx.lstChatMessages.AddRange(allNewMessages); var jsonSerializerOptions = new JsonSerializerOptions { WriteIndented = true, Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping }; if (cntx.lstChatMessages.Count >= cntx._targetCount + cntx._threshold) { reducedMessages = await cntx._summarizingChatReducer.ReduceAsync(cntx.lstChatMessages, cancellationToken); await db.StringSetAsync(cntx.UserName, System.Text.Json.JsonSerializer.Serialize(reducedMessages, jsonSerializerOptions)); } else { await db.StringSetAsync(cntx.UserName, System.Text.Json.JsonSerializer.Serialize(cntx.lstChatMessages, jsonSerializerOptions)); } } ``` In above code there are two lists : **allNewMessages** and **reducedMessages** allNewMessages holds response and request messages which is then added to lstChatMessages of **InvokedContext** context declared as **cntx**. ```csharp var cntx = _sessionstate.GetOrInitializeState(context.Session); IEnumerable reducedMessages = new List(); var allNewMessages = context.RequestMessages.Concat(context.ResponseMessages ?? []); cntx.lstChatMessages.AddRange(allNewMessages); ``` > Recall that lstChatMessages is a property in the SessionState class **SerializationOption** ```csharp var jsonSerializerOptions = new JsonSerializerOptions { WriteIndented = true, Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping }; ``` Check if the conversation count is greatest than targetCount and threshold combined together, trigger the **ReduceAsync** method and save the reduced messages in Redis if not then save all the conversation messages to Redis ```csharp if (cntx.lstChatMessages.Count >= cntx._targetCount + cntx._threshold) { reducedMessages = await cntx._summarizingChatReducer.ReduceAsync(cntx.lstChatMessages, cancellationToken); await db.StringSetAsync(cntx.UserName, System.Text.Json.JsonSerializer.Serialize(reducedMessages, jsonSerializerOptions)); } else { await db.StringSetAsync(cntx.UserName, System.Text.Json.JsonSerializer.Serialize(cntx.lstChatMessages, jsonSerializerOptions)); } ``` **Prompts** I will first send the following five prompts together in a single session ```csharp Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "My name is Sachin") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");//Prompt 1 Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "I live in India") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");//Prompt 2 Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "I work in the IT industry") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n"); //Prompt 3 Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "I am a software developer") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");//Prompt 4 Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "I work mainly with C# and .NET") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n"); //Prompt 5 ``` > Recall that we have set the values of targetCount and threshold to 2 and 3 respectively. In the above case **Prompt 5** and the agent response will be stored to Redis in its raw form while the rest of prompts are summarized. ![](https://cdn.hashnode.com/uploads/covers/6693c62c166ee9c594cffda0/c95bce73-fa08-4275-9e67-6bba6e074aff.png align="center") In the image above **1** in red is the summarized text and **2** and **3** in red are the stored messages that are based on the targetCount value. ### Execution ![](https://cdn.hashnode.com/uploads/covers/6693c62c166ee9c594cffda0/a5a781a4-2849-4c99-b76e-92e5e3f488da.gif align="center") ### Conclusion ChatSummarizer provides an effective way to manage conversation history by reducing a large number of messages into a concise summary while preserving the important and required context. This helps prevent conversation history from growing indefinitely and reduces the amount of data that needs to be stored and processed.This gives us a balance between retaining recent conversation details and maintaining a conversational context of older messages. This helps improve memory efficiency, reduce token usage and there by maintain efficient and relevant context throughout longer conversations. Thanks for reading !!!