Unit Test for MCP!

Imagine you have an MCP server; it's likely you'd want to create unit tests to ensure it functions as intended. In this post, I'll walk through how to write unit tests for MCP servers. We'll be testing an existing MCP Server available in Smithery: Time Server MCP. As shown, it includes the following tools: Write Unit Test First, we create a new xunit project and add skUnit from NuGet. Next, according to the skUnit documentation, our only task is to configure it in the test constructor. public class TimeServerMcpTests { ScenarioAssert ScenarioAssert { get; set; } IChatClient ChatClient { get; set; } public TimeServerMcpTests(ITestOutputHelper output) { var configuration = new ConfigurationBuilder() .AddUserSecrets() .Build(); var apiKey = configuration["AzureOpenAI_ApiKey"]; var endpoint = configuration["AzureOpenAI_Endpoint"]; var deploymentName = configuration["AzureOpenAI_Deployment"]; ChatClient = new AzureOpenAIClient( new Uri(endpoint), new System.ClientModel.ApiKeyCredential(apiKey)) .GetChatClient(deploymentName) .AsIChatClient(); ScenarioAssert = new ScenarioAssert(ChatClient, output.WriteLine); } [Fact] public async Task Tools_MustWork() { // Our tests will be here... } } Let's start by writing the body for our test method. The first step is to establish a connection to the MCP Server. var clientTransport = new StdioClientTransport(new StdioClientTransportOptions { Name = "Time MCP Server", Command = "cmd", Arguments = [ "/c", "npx", "-y", "@smithery/cli@latest", "run", "@yokingma/time-mcp" ], }); // Here's our MCP ready for testing. await using var mcp = await McpClientFactory.CreateAsync(clientTransport); Now, we built a ChatClient on it: var tools = await mcp.ListToolsAsync(); var builder = new ChatClientBuilder(ChatClient) .ConfigureOptions(options => { options.Tools = tools.ToArray(); }) .UseFunctionInvocation(); var chatClient = builder.Build(); and finally our 3 lines of skUnit to test the scenario: var scenarioText = await File.ReadAllTextAsync("TestScenario.md"); var scenario = ChatScenario.LoadFromText(scenarioText); await ScenarioAssert.PassAsync(scenario, chatClient); And just like that, here’s the outcome! Test Scenario But how did we specify the test scenario, and what was in the TestScenario.md file that we loaded as our scenario? As you can see in this scenario, we expect the answer to "What time is it?" should be any sentence or paragraph that has this semantic condition: Semantic Condition: It should mention a time. ### CHECK SemanticCondition It mentions a time. We also expect it to utilize the current_time tool in MCP, and by following the assertion, we are ensuring this. Function Call Check: It should call a tool. ### CHECK FunctionCall { "function_name": "current_time", } Then, by asking "How many days are in January this year?", we test its other capability, ensuring it correctly calls days_in_month. ## [USER] How many days are in this year's january? ## [AGENT] ### CHECK SemanticCondition It mentions 31 days. ### CHECK FunctionCall { "function_name": "days_in_month", } Finally! Using skUnit, it's super easy to test your AI blocks like MCP Server, ChatClient, Kernel, and more. The full source code is available on my GitHub: Demo.TddMcp.

Apr 15, 2025 - 22:47

Imagine you have an MCP server; it's likely you'd want to create unit tests to ensure it functions as intended.

In this post, I'll walk through how to write unit tests for MCP servers. We'll be testing an existing MCP Server available in Smithery: Time Server MCP.

As shown, it includes the following tools:

Write Unit Test

First, we create a new xunit project and add skUnit from NuGet.

Next, according to the skUnit documentation, our only task is to configure it in the test constructor.

public class TimeServerMcpTests
{
  ScenarioAssert ScenarioAssert { get; set; }
  IChatClient ChatClient { get; set; }

   public TimeServerMcpTests(ITestOutputHelper output)
   {
     var configuration = new ConfigurationBuilder()
                              .AddUserSecrets<TimeServerMcpTests>()
                              .Build();

     var apiKey = configuration["AzureOpenAI_ApiKey"];
     var endpoint = configuration["AzureOpenAI_Endpoint"];
     var deploymentName = configuration["AzureOpenAI_Deployment"];

     ChatClient = new AzureOpenAIClient(
                    new Uri(endpoint),
                    new System.ClientModel.ApiKeyCredential(apiKey))
                  .GetChatClient(deploymentName)
                  .AsIChatClient();

            ScenarioAssert = new ScenarioAssert(ChatClient, output.WriteLine);
  }

  [Fact]
  public async Task Tools_MustWork()
  {
    // Our tests will be here...
  }
}

Let's start by writing the body for our test method. The first step is to establish a connection to the MCP Server.

var clientTransport = new StdioClientTransport(new StdioClientTransportOptions
{
    Name = "Time MCP Server",
    Command = "cmd",
    Arguments = [
        "/c",
        "npx",
        "-y",
        "@smithery/cli@latest",
        "run",
        "@yokingma/time-mcp"
    ],
});

// Here's our MCP ready for testing.
await using var mcp = await McpClientFactory.CreateAsync(clientTransport);

Now, we built a ChatClient on it:

var tools = await mcp.ListToolsAsync();

var builder = new ChatClientBuilder(ChatClient)
              .ConfigureOptions(options =>
              {
                  options.Tools = tools.ToArray();
              })
              .UseFunctionInvocation();

var chatClient = builder.Build();

and finally our 3 lines of skUnit to test the scenario:

var scenarioText = await File.ReadAllTextAsync("TestScenario.md");
var scenario = ChatScenario.LoadFromText(scenarioText);
await ScenarioAssert.PassAsync(scenario, chatClient);

And just like that, here’s the outcome!

Test Scenario

But how did we specify the test scenario, and what was in the TestScenario.md file that we loaded as our scenario?

As you can see in this scenario, we expect the answer to "What time is it?" should be any sentence or paragraph that has this semantic condition:

Semantic Condition: It should mention a time.

### CHECK SemanticCondition
It mentions a time.

We also expect it to utilize the current_time tool in MCP, and by following the assertion, we are ensuring this.

Function Call Check: It should call a tool.

### CHECK FunctionCall
{
  "function_name": "current_time",
}

Then, by asking "How many days are in January this year?", we test its other capability, ensuring it correctly calls days_in_month.

## [USER]
How many days are in this year's january?

## [AGENT]

### CHECK SemanticCondition
It mentions 31 days.

### CHECK FunctionCall
{
  "function_name": "days_in_month",
}

Finally!

Using skUnit, it's super easy to test your AI blocks like MCP Server, ChatClient, Kernel, and more. The full source code is available on my GitHub: Demo.TddMcp.