Human-In-The-Loop (HITL) in Multi Agent Setup in Microsoft Agent Framework

Human-In-The-Loop or commonly called as HITL is a common design pattern across multi agent setups that requires human approval or intervention at critical stage of a multi agent execution process. In essence, HITL ensures accuracy and compliance during an execution of a process in a multi agent flow. This ensures that agents don't act completely autonomously.

Lets imagine a very simple business use case where an Agent A creates/extracts invoices and in the same flow Agent B approves them.

But letting Agent B to act autonomously and approve every generated invoice without proper validations through human intervention is prone to huge business risk and process breakdown.

With HITL implementation in the above use case, Agent A creates/extracts invoices and post human intervention and confirmation Agent B can proceed with the approval.

HITL in Microsoft Agent Framework

In MAF, there are two common scenarios where Human-In-The-Loop (HITL) can be implemented:

The first is within multi-agent orchestration and the second is within workflow-based orchestration.

This article will focus on HITL for a multi agent orchestration. I will soon have another article on implementation of HITL across MAF workflows.

In MAF, HITL revolves around the ApprovalRequiredAIFunction inbuilt function .It's a very important function in MAF wrt HITL.

In HITL in MAF while using an AIFunction, it's possible to indicate if the function requires human approval before being executed. This is done by wrapping the AIFunction instance in an ApprovalRequiredAIFunction instance.

When an LLM requests a tool, the FunctionInvokingChatClient returns ToolApprovalRequestContent in the response. This response is then parsed by the caller.

In the next step the caller decides to approve/reject and in turn creates ToolApprovalResponseContent. The caller then sends the response back through CallId and the framework based on the approve/reject response executes or skips the tool execution.

The LLM flow can either move on or wait for the user to approve the request. This is done by checking List<ToolApprovalRequestContent> count.

SetUp

Create a new C# console application and add the following packages

dotnet add package Azure;
dotnet add package Azure.AI.OpenAI;
dotnet add package Microsoft.Agents.AI;
dotnet add package Microsoft.Extensions.AI;
dotnet add package Microsoft.Extensions.Configuration;
dotnet add package Microsoft.Extensions.DependencyInjection;

Add appsetting.json to the project

"AppSettings": 
{ 
    "Chat_DeploymentName": "Deployment Name",
    "EndPoint": "Azure OpenAI endpoint",
    "ApiKey": "Azure OpenAI API key"
}

Code

Now that all the underlying artifacts in place, add the following code to read the settings from appsettings.json and register ChatClient.

var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false)
.Build();

var credential = new AzureKeyCredential(configuration["AppSettings:ApiKey"]);

 servicecollection.AddKeyedChatClient("ChatClient", (sp => new AzureOpenAIClient(
          new Uri(configuration["AppSettings:EndPoint"]), credential)
              .GetChatClient(configuration["AppSettings:Chat_DeploymentName"]).AsIChatClient()));

Create a DI service registry.

 ServiceCollection servicecollection = new ServiceCollection();

Now that the basic set up is ready, create two agents.

The first agent generates invoices and the second agent approves/rejects generated invoices based on user approval. As mentioned earlier, our flow would continue waiting for user approval incase the approval is rejected.

InvoiceGeneratingAgent >>

servicecollection.AddSingleton<ChatClientAgent>(sp =>
{
    Func<ChatClientAgentOptions> func = () =>
   {
       return new ChatClientAgentOptions
       {
           ChatOptions = new ChatOptions
           {
               Instructions = "You are an helpful assistant.You generate invoices for clients",
           },
           Name = "InvoiceGeneratingAgent",
           Id = "1"
       };
   };
    return new ChatClientAgent(sp.GetKeyedService<IChatClient>("ChatClient"), options: func());

});

InvoiceApprovalAgent >>

 servicecollection.AddSingleton<ChatClientAgent>(sp =>
 {
     Func<ChatClientAgentOptions> func = () =>
    {
        return new ChatClientAgentOptions
        {
            ChatOptions = new ChatOptions
            {
                Instructions = "You are an helpful assistant.You approve generated invoices for clients after getting user confirmation",
            },
            Name = "InvoiceApprovalAgent",
            Id = "2"
        };
    };
     return new ChatClientAgent(sp.GetKeyedService<IChatClient>("ChatClient"), options: func());
 });

Next, create two AIFunctions

GenerateInvoice >>

public static string GenerateInvoice(string InvoiceNumber) => $"Invoice {InvoiceNumber} generated";

ApproveInvoice >>

private static string ApproveInvoice(string InvoiceNumber, string ClientName, string DueDate) => $"Invoice number {InvoiceNumber} approved for  client {ClientName}, Duedate: {DueDate}";

Register the functions in the DI service registry

GenerateInvoice >>

servicecollection.AddSingleton<AIFunction>(
    sp =>
    {
        Func<AIFunctionArguments, string> func = args =>
        {
            var ctx = FunctionInvokingChatClient.CurrentContext;
            return GenerateInvoice(ctx.Options.AdditionalProperties["InvoiceNumber"].ToString()!);
        };
        return AIFunctionFactory.Create(
            func,
            new AIFunctionFactoryOptions { Name = "GenerateInovice" }
        );
    }
);

ApproveInvoice >>

servicecollection.AddSingleton<AIFunction>(
    sp =>
    {
        Func<AIFunctionArguments, string> func = args =>
        {
            var ctx = FunctionInvokingChatClient.CurrentContext;
            return ApproveInvoice(
                ctx.Options.AdditionalProperties["InvoiceNumber"].ToString(),
                ctx.Options.AdditionalProperties["ClientName"].ToString()!,
                ctx.Options.AdditionalProperties["DueDate"].ToString()!
            );
        };
        return new ApprovalRequiredAIFunction(
            AIFunctionFactory.Create(func, new AIFunctionFactoryOptions { Name = "ApproveInvoice" })
        );
    }
);

The ApproveInvoice function is wrapped with ApprovalRequiredAIFunction .

The FunctionInvokingChatClient reads the arguments required for the underlying functions through AdditionalPropertiesDictionary.

You can read about FunctionInvokingChatClient in my article here.

ServiceProvider serviceprovider = servicecollection.BuildServiceProvider();

var AIAgents = serviceprovider.GetServices<ChatClientAgent>();
List<ChatClientAgent> lst = new(AIAgents);

var Aitools = serviceprovider.GetServices<AIFunction>();
List<AITool> lstAitools = new(Aitools);

IChatClient invoicegeneratingagent = lst[0].ChatClient;
IChatClient invoiceapprovalagent = lst[1].ChatClient;

ChatOptions chtoptions = new ChatOptions();
List<ChatMessage> chatMessage = [new(ChatRole.User, "Please generate invoice")];

AdditionalPropertiesDictionary AdditionalProperties_dict = new()
{
    ["ClientName"] = " XYZ Corporation",
    ["InvoiceNumber"] = "1234-5678-999",
    ["DueDate"] = DateTime.Now.ToShortDateString()
};

chtoptions.Tools = lstAitools;
chtoptions.AdditionalProperties = AdditionalProperties_dict;
chtoptions.Instructions = "We need to generate invoice and get it approved. Please coordinate the approval.";

AgentSession session = await invoiceapprovalagent.AsAIAgent().CreateSessionAsync();
chatMessage = [new(ChatRole.User, "Please approve invoice")];

var int_response = await invoiceapprovalagent.GetResponseAsync(chatMessage, chtoptions);

List<ToolApprovalRequestContent> functionApprovalRequests = int_response.Messages
.SelectMany(x => x.Contents)
.OfType<ToolApprovalRequestContent>()
.ToList();

while (functionApprovalRequests.Count > 0)
{    
    List<ChatMessage> userInputResponses = functionApprovalRequests
        .ConvertAll(functionApprovalRequest =>
        {
             Console.WriteLine($"The agent would like to invoke the following function {((FunctionCallContent)functionApprovalRequest.ToolCall).Name}, please reply Y to approve Invoice No : {AdditionalProperties_dict["InvoiceNumber"]} for client {AdditionalProperties_dict["ClientName"]} ");

            return new ChatMessage(ChatRole.User, [functionApprovalRequest.CreateResponse(Console.ReadLine()?.Equals("Y", StringComparison.OrdinalIgnoreCase) ?? false)]);
        });

    var response = await invoiceapprovalagent.GetResponseAsync(userInputResponses, chtoptions);

    functionApprovalRequests = response.Messages.SelectMany(m => m.Contents).OfType<ToolApprovalRequestContent>().ToList();

Major aspects of the above piece of code

• Fetch a list of all the registered AI agents into variable lst of type List from the DI registry.

• Similarly, fetch a list of all AIFunction registered in DI registry into the variable lstAitools of type List.

• Access and store ChatClient associated with the two registered agents as invoicegeneratingagent and invoiceapprovalagent.

• Access ChatClient to send AdditionalPropertiesDictionary through ChatOptions so that the FunctionInvokingChatClient can access and pass them to the underlying AIFunctionduring invoking them.

• Set ClientName,InvoiceNumber,DueDate in AdditionalPropertiesDictionary

• Get the count of ToolApprovalRequestContent and store it in variable functionApprovalRequests

• Iterate through functionApprovalRequests and keep running as long as there are pending approval requests.

• Ask human for approval and create a response message based on "Y" or "N" input and wrap the response in the ChatMessage and send the response back to the approval agent.

• Continue through the iteration until the approval is not received in form of "Y"

 while (functionApprovalRequests.Count > 0)
 {     
     List<ChatMessage> userInputResponses = functionApprovalRequests
         .ConvertAll(functionApprovalRequest =>
         {
             Console.WriteLine($"The agent would like to invoke the following function {((FunctionCallContent)functionApprovalRequest.ToolCall).Name}, please reply Y to approve Invoice No : {AdditionalProperties_dict["InvoiceNumber"]} for client {AdditionalProperties_dict["ClientName"]} ");

             return new ChatMessage(ChatRole.User, [functionApprovalRequest.CreateResponse(Console.ReadLine()?.Equals("Y", StringComparison.OrdinalIgnoreCase) ?? false)]);
         });

      var response = await invoiceapprovalagent.GetResponseAsync(userInputResponses, chtoptions);

     functionApprovalRequests = response.Messages.SelectMany(m => m.Contents).OfType<ToolApprovalRequestContent>().ToList();
}

Execution >>

Conclusion

Human-In-The-Loop (HITL) plays a crucial role in multi-agent systems by combining AI automation with required human intervention. While agents can independently perform tasks and invoke functions, important decisions can still be routed through a human approval . This helps improve reliability and compliance in agent driven business setups.

Hope this article helped unravel implementation of Human-In-The-Loop (HITL) in MAF.

Thanks for reading !!!