In MAF, if no external storage provider is configured, the chat history by default is stored in server memory. The InMemoryChatHistoryProvider component stores chat messages in a StateBag where the StateKey acts as the identifier used to access the stored conversation state.
StateBag is a container in memory that stores relevant conversation that can be identified through unique identifier called StateKey .
In practice, it is not efficient to store chat history in memory, as chat history cannot be persisted for further analysis. Storing chat history in memory works best for prototypes and local development with simple setups. So the only option is to implement a custom history chat provider.
If you want to configure external chat storages ,ChatHistoryProvider is the base class that your custom chat history provider should always inherit from.
A single ChatHistoryProvider instance can be reused across many sessions, so storing session data inside the provider itself will have issues for session specific chat histories. Instead session-specific data should be stored in the StateBag which is scoped per session/conversation.
In this blog I am going to walk you through the implementation of a custom chat history provider that stores and retrieves conversations to and from a file system.
You can watch the step by step process in the following video.
https://youtu.be/RzsWMPE9eJU?si=azh4_b7fli_I-YJH
SetUp
Create a new console application and add the following packages
dotnet add package Azure;
dotnet add package Azure.AI.OpenAI;
dotnet add package Microsoft.Agents.AI;
dotnet add package Microsoft.Extensions.AI;
dotnet add package Microsoft.Extensions.Configuration;
dotnet add package Microsoft.Extensions.DependencyInjection;
dotnet add package Microsoft.Extensions.Logging;
Add appsetting.json to the project
"AppSettings": {
"Chat_DeploymentName": "Deployment Name",
"EndPoint": "Azure OpenAI endpoint",
"ApiKey": "Azure OpenAI API key"
}
Code
Now that we have all the underlying artifacts in place, add the following code to read the settings from appsettings.json
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false)
.Build();
Create a DI container
ServiceCollection servicecollection = new ServiceCollection();
Read the credentials and register Chatclient
var credential = new AzureKeyCredential(configuration["AppSettings:ApiKey"]);
servicecollection.AddKeyedChatClient(
"ChatClient",
(
sp =>
new AzureOpenAIClient(new Uri(configuration["AppSettings:EndPoint"]), credential)
.GetChatClient(configuration["AppSettings:Chat_DeploymentName"])
.AsIChatClient()
)
);
Register an AI Agent in the DI container
servicecollection.AddScoped<AIAgent>(sp =>
{
Func<ChatClientAgentOptions> func = () =>
{
return new ChatClientAgentOptions
{
ChatOptions = new ChatOptions
{
Instructions = "You are a helpful chat assistant."
},
Name = "ChatHistoryProviderAgent"
,
ChatHistoryProvider = new ChatHistoryProvider.CustomChathistoryProvider()
};
};
return new ChatClientAgent(sp.GetKeyedService<IChatClient>("ChatClient"), options: func());
});
Note the option ChatHistoryProvider in the code above
ChatHistoryProvider = new ChatHistoryProvider.CustomChathistoryProvider()
CustomChathistoryProvider() is the custom chat history provider that I created with namespace ChatHistoryProvider.
You can instead use InMemoryChatHistoryProvider if you want to use memory chat storage .
ChatHistoryProvider = new InMemoryChatHistoryProvider(new InMemoryChatHistoryProviderOptions { ChatReducer = new MessageCountingChatReducer(10)})
In code above, we configured the ChatReducer to limit the conversation history to only 10 messages. That value is customizable.
You can also implement your own custom ChatReducer through IChatReducer interface.
public class Chat_reducer : IChatReducer
{
private readonly int _messages;
public Chat_reducer(int messages)
{
_messages = messages;
}
public Task<IEnumerable<ChatMessage>> ReduceAsync(IEnumerable<ChatMessage> messages,CancellationToken cancellationToken)
{
return Task.FromResult(messages.TakeLast(_messages));
}
}
and then set ChatHistoryProvider value to the instance of Chat_reducer.
ChatHistoryProvider = new InMemoryChatHistoryProvider(new InMemoryChatHistoryProviderOptions { ChatReducer = new Chat_reducer(10)})
Custom Chat History Provider
Custom chat history provider irrespective of the underlying data storage should always inherit from Microsoft.Agents.AI.ChatHistoryProvider namespace.
We start with defining properties of session state through class SessionState
internal class SessionState
{
[JsonPropertyName("Messages")]
public List<ChatMessage> lstChatMessages { get; set; } = [];
[JsonPropertyName("UserName")]
public string UserName { get; set; } = "";
[JsonPropertyName("SessionId")]
public string SessionId { get; set; } = Guid.NewGuid().ToString();
[JsonPropertyName("FileName")]
public string FileName { get; set; } = "";
}
We then create an instance of ProviderSessionState for storing and handling the SessionState.
ProviderSessionState is used for initializing and persisting provider state in the session's StateBag. State( in our case it is SessionState object) is a inbuild session state management that maintains conversation history and is stored in the StateBag using the StateKey property as the key.
CustomChathistoryProvider Constructor >>
private readonly Microsoft.Agents.AI.ProviderSessionState<SessionState> _sessionstate;
public CustomChathistoryProvider(Func<AgentSession, SessionState> initializer = null, string statekey = "")
{
_sessionstate = new(initializer =>
{
string filepath = $"I:\\Repos\\MAF\\{statekey}.json";
var json = !File.Exists(filepath) ? "" : File.ReadAllText(filepath);
return new SessionState
{
lstChatMessages = json == "" ? Enumerable.Empty<ChatMessage>().ToList() : JsonSerializer.Deserialize<List<ChatMessage>>(json)
};
}
, statekey = "Sachin"
);
}
What you see above is constructor of theCustomChathistoryProvider class that accepts a function (delegate) initializer and a string key statekey.
The constructor argument takes AgentSession as input and returns SessionState.
Remember that the ChatHistoryProvider is a reference to the AgentSession.
In this code the chat history JSON file is stored at location I:\Repos\MAF\ . You can customize the file path. The JSON file is named based on the StateKey value.
We are storing the chat conversation on a file system in JSON format. It first needs to read through a JSON file stored on a predefined location to load the chat messages and if not found the SessionState returns empty.
return new SessionState
{
lstChatMessages =
json == ""
? Enumerable.Empty<ChatMessage>().ToList()
: JsonSerializer.Deserialize<List<ChatMessage>>(json)
};
The ChatHistoryProvider base class has two methods that are required to be overridden by the custom chat history provider class.
They are ProvideChatHistoryAsync and StoreChatHistoryAsync.
ProvideChatHistoryAsync supplies chat history when required and returns them through the InvokingContext context.
protected override ValueTask<IEnumerable<ChatMessage>> ProvideChatHistoryAsync(InvokingContext context, CancellationToken cancellationToken = default)
{
var cntx = _sessionstate.GetOrInitializeState(context.Session);
if (cntx.lstChatMessages.Count > 0)
{
return new(
cntx
.lstChatMessages
);
}
return new(Enumerable.Empty<ChatMessage>());
}
StoreChatHistoryAsync serializes and stores the chat history to the JSON file by combining the request and response chat messages through the InvokedContext context.
protected override ValueTask StoreChatHistoryAsync(InvokedContext context, CancellationToken cancellationToken = default)
{
var cntx = _sessionstate.GetOrInitializeState(context.Session);
var allNewMessages = context.RequestMessages.Concat(context.ResponseMessages ?? []);
cntx.lstChatMessages.AddRange(allNewMessages);
cntx.UserName = _sessionstate.StateKey;
string filepath = $"I:\\Repos\\MAF\\{cntx.UserName}.json";
var json = JsonSerializer.Serialize(cntx.lstChatMessages);
File.WriteAllText(filepath, json);
return default;
}
CustomChatHistoryProvider.cs >>
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace ChatHistoryProvider
{
internal class CustomChatHistoryProvider : Microsoft.Agents.AI.ChatHistoryProvider
{
private readonly Microsoft.Agents.AI.ProviderSessionState<SessionState> _sessionstate;
public CustomChatHistoryProvider(Func<AgentSession, SessionState> initializer = null, string statekey = "")
{
_sessionstate = new(initializer =>
{
string filepath = $"I:\\Repos\\MAF\\{statekey}.json";
var json = !File.Exists(filepath) ? "" : File.ReadAllText(filepath);
return new SessionState
{
lstChatMessages = json == "" ? Enumerable.Empty<ChatMessage>().ToList() : JsonSerializer.Deserialize<List<ChatMessage>>(json)
};
}
, statekey = "Sachin"
);
}
protected override ValueTask<IEnumerable<ChatMessage>> ProvideChatHistoryAsync(InvokingContext context, CancellationToken cancellationToken = default)
{
var cntx = _sessionstate.GetOrInitializeState(context.Session);
if (cntx.lstChatMessages.Count > 0)
{
return new(
cntx
.lstChatMessages
);
}
return new(Enumerable.Empty<ChatMessage>());
}
protected override ValueTask StoreChatHistoryAsync(InvokedContext context, CancellationToken cancellationToken = default)
{
var cntx = _sessionstate.GetOrInitializeState(context.Session);
var allNewMessages = context.RequestMessages.Concat(context.ResponseMessages ?? []);
cntx.lstChatMessages.AddRange(allNewMessages);
cntx.UserName = _sessionstate.StateKey;
string filepath = $"I:\\Repos\\MAF\\{cntx.UserName}.json";
var json = JsonSerializer.Serialize(cntx.lstChatMessages);
File.WriteAllText(filepath, json);
return default;
}
}
internal class SessionState
{
[JsonPropertyName("Messages")]
public List<ChatMessage> lstChatMessages { get; set; } = [];
[JsonPropertyName("UserName")]
public string UserName { get; set; } = "";
[JsonPropertyName("SessionId")]
public string SessionId { get; set; } = Guid.NewGuid().ToString();
[JsonPropertyName("FileName")]
public string FileName { get; set; } = "";
}
}
Execution >>
Out first prompt is
Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "My name is Sachin") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");
The conversation is stored in file named Sachin.json .
If you recall, the StateKey value is Sachin and chat history JSON file is named with the StateKey value.
The agent response is highlighted in red above.
Next, we terminate the session and send the following prompt through a new session.
Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "Forget my name and never ever recall what my name is") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");
As you can see above , the agent confirms that it has been asked not to recall the name.
Now, terminate the previous session and in a new session send the following prompt.
Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "What is my name ?") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");
The reason to send the above prompt is to check whether if the agent can or cannot recall the name.
The response indicates that it cannot recall the name.
Next , we change the ProvideChatHistoryAsync to provide only the very first prompt that was sent at the start of the conversation.
protected override ValueTask<IEnumerable<ChatMessage>> ProvideChatHistoryAsync(InvokingContext context, CancellationToken cancellationToken = default)
{
var cntx = _sessionstate.GetOrInitializeState(context.Session);
if (cntx.lstChatMessages.Count > 0)
{
return new(
cntx
.lstChatMessages
.Where(x => x.AuthorName == "Sachin")
.OrderBy(x => x.CreatedAt).Take(1).ToList()
);
}
return new(Enumerable.Empty<ChatMessage>());
}
this is the code that returns the very first prompt in the chat history.
return new(
cntx
.lstChatMessages
.Where(x => x.AuthorName == "Sachin")
.OrderBy(x => x.CreatedAt).Take(1).ToList()
);
The first prompt in the conversation history was
My name is Sachin.
We then resend the following prompt to the agent to check if it remembers the name given that in the current context it only has the above prompt available from the chat history .
Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "What is my name ?") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");
As you can see above, the agent can now recall the name.
Program.cs >>
using Azure;
using Azure.AI.OpenAI;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
internal class Program
{
private async static Task Main(string[] args)
{
ServiceCollection servicecollection = new ServiceCollection();
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false)
.Build();
var credential = new AzureKeyCredential(configuration["AppSettings:ApiKey"]);
servicecollection.AddKeyedChatClient("ChatClient", (sp) => new AzureOpenAIClient(
new Uri(configuration["AppSettings:EndPoint"]), credential)
.GetChatClient(configuration["AppSettings:Chat_DeploymentName"])
.AsIChatClient());
servicecollection.AddScoped<AIAgent>(sp =>
{
Func<ChatClientAgentOptions> func = () =>
{
return new ChatClientAgentOptions
{
ChatOptions = new ChatOptions
{
Instructions = "You are a helpful chat assistant."
},
Name = "ChatHistoryProviderAgent"
,
ChatHistoryProvider = new ChatHistoryProvider.CustomChatHistoryProvider()
};
};
return new ChatClientAgent(sp.GetKeyedService<IChatClient>("ChatClient"), options: func());
});
ServiceProvider serviceprovider = servicecollection.BuildServiceProvider();
var agent = serviceprovider.GetService<AIAgent>();
var agentsession = await agent!.CreateSessionAsync();
Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "My name is Sachin") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");
// Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "Forget my name and never ever recall what my name is") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");
// Console.WriteLine(await agent.RunAsync(new ChatMessage(ChatRole.User, "What is my name ?") { CreatedAt = DateTimeOffset.UtcNow, AuthorName = "Sachin" }, session: agentsession, new AgentRunOptions { AllowBackgroundResponses = true }) + "\n");
}
}
Conclusion
As you can see that with careful design considerations, chat history can be stored in external storage systems instead of relying entirely on server memory which in turn improve scalability , reduces memory overhead and support more efficient and effective session management.
Thanks for reading !!!
