Introduction Link to heading

The .NET ecosystem for testing has been quite stable for a while now. It all began with NUnit and MSTest, and then xUnit came along and became the most popular choice for .NET developers when it became the default testing framework for .NET Core projects. Personally, I don’t really have any big complaints or gripes about these platforms. They all do what they need to do, and they do it well. However, I recently stumbled upon a new (open source) testing framework called TUnit, and I thought it was worth checking out, especially since it’s a new contender in a space that hasn’t seen much change in a while!

So, what can TUnit offer that the other testing frameworks can’t? Let’s find out!

What is TUnit? Link to heading

TUnit is a new testing framework (for .NET 8 and up) that aims to address pain points in existing testing frameworks. Furthermore, it wants to be more extensible and a whole lot faster! It’s created by Tom Longhurst, and it’s still in preview at the time of writing. However, this doesn’t mean you shouldn’t follow or even try out this new framework!

I believe that TUnit changes the game because of 4 things:

• It uses source generators to locate and register your tests instead of using reflection. This slightly increases your build time, but it (dramatically) decreases the time it takes to run your tests, which is way more important to me!
• It’s AOT compatible because it uses source generators instead of reflection.
  • AOT stands for Ahead-Of-Time compilation, which means that your code is compiled before it’s run, instead of being compiled just-in-time (JIT) when it’s run. This is relatively new in .NET. You can learn more here.
  • AOT compatibility is great for applications that want to start up quickly, as the code is already compiled and doesn’t need to be compiled when the application starts.
• It uses the new Microsoft.Testing.Platform instead of the older VSTest platform. This new platform is more modern, faster and more extensible!
• The benchmarks show that TUnit, with and without AOT mode, can make your tests run up to 10x faster, and some benchmarks are even 200x faster! 🤯

Even though it has “unit” in the name, you’re not limited to unit tests. You can write integration tests, acceptance tests, and more with TUnit! Finally, it also has great documentation. In this post I want to document the process of getting started with TUnit, and show you how to write your first test with it and compare it to other platforms.

I hope this has convinced you to read on and learn more about TUnit! Let’s get started!

Installing TUnit Link to heading

Installing TUnit is easy; it’s just a NuGet package that you add to your projects, just like XUnit, NUnit etc.. One important caveat is that you should not install Microsoft.NET.Test.Sdk in your test project, as it will stop test discovery from working properly.

We’ll start by creating 2 projects, one for source code, one for tests. I’m using .NET 9 here, but you can also use .NET 8! This assumes you’ve already installed .NET. We’ll start from a clean slate.

# Set up your project folder
mkdir tunit-playground
cd tunit-playground

# Set up your solution and empty projects
dotnet new sln -n tunit-playground
dotnet new console -n SampleProject
dotnet new console -n SampleProject.TUnit

# Add the projects to the solution file
dotnet sln add SampleProject
dotnet sln add SampleProject.TUnit

# Let's make the test project reference the source project
cd SampleProject.TUnit
dotnet add reference ../SampleProject/SampleProject.csproj

# We don't need a Program.cs in the test project, let's remove it!
rm Program.cs

# Let's add TUnit to the test project
dotnet add package TUnit --prerelease

# Let's go back to the root folder and build successfully!
cd ../
dotnet build --configuration Release

We’ve now created 2 projects, one for source code, one for tests. The test project references the source project, and we’ve added TUnit to the test project. We’ve also built the solution successfully. Now let’s write our first test!

Writing your first test Link to heading

This blog post is just an introduction to TUnit. Therefore, I will keep things simple. Later on in this post we’ll introduce more TUnit features. For now, let’s write a simple test that checks if a method that adds two numbers together works correctly.

Let’s add a class called Calculator to the bottom of the Program.cs file of the SampleProject. Feel free to add some fun input/output code so theSampleProject does something fun when you run it. Now, let’s add a method called Add that takes two integers and returns their sum:

public class Calculator
{
    public int Add(int a, int b)
    {
        return a + b;
    }
}

Mind-blowing, isn’t it? 😉 This is good enough for now!

Let’s now write a test for this method in the SampleProject.TUnit project. Create a new file called CalculatorTests.cs in the SampleProject.TUnit project, and add the following code:

namespace SampleProject.TUnit;
public class CalculatorTests
{
    [Test]
    public async Task Add_WhenCalled_ReturnsTheSumOfArguments()
    {
        // Arrange
        var calculator = new Calculator();

        // Act
        var result = calculator.Add(1, 2);

        // Assert
        await Assert.That(result).IsEqualTo(3);
    }

    [Test]
    public async Task Add_WhenOverflowing_ReturnsIntMinValue()
    {
        // Arrange
        var calculator = new Calculator();

        // Act
        var result = calculator.Add(int.MaxValue, 1);

        // Assert
        await Assert.That(result).IsEqualTo(int.MinValue);
    }
}

You might notice that the test methods are async, even though the Add method is synchronous. This is done because TUnit asks us to use await Assert.That(...) to verify test results. Of course, there are also other methods available like ThrowsAsync and more. Again, if you’re familiar with xUnit or NUnit, you’ll feel right at home, even if the syntax is slightly different.

We can now run our tests by calling dotnet test or dotnet run in the SampleProject.TUnit folder:

# dotnet build --configuration Release
# dotnet test --configuration Release --no-build
SampleProject.TUnit test succeeded (0,2s)

Test summary: total: 2; failed: 0; succeeded: 2; skipped: 0; duration: 0,2s
Build succeeded in 0,4s

# dotnet run --configuration Release --no-build
████████╗██╗   ██╗███╗   ██╗██╗████████╗
╚══██╔══╝██║   ██║████╗  ██║██║╚══██╔══╝
   ██║   ██║   ██║██╔██╗ ██║██║   ██║   
   ██║   ██║   ██║██║╚██╗██║██║   ██║   
   ██║   ╚██████╔╝██║ ╚████║██║   ██║   
   ╚═╝    ╚═════╝ ╚═╝  ╚═══╝╚═╝   ╚═╝   
   
   TUnit v0.4.43.0 | 64-bit | Microsoft Windows 10.0.22631 | win-x64 | .NET 9.0.0 | Microsoft Testing Platform v1.4.3

Test run summary: Passed! - bin\Release\net9.0\SampleProject.TUnit.dll (net9.0|x64)
  total: 2
  failed: 0
  succeeded: 2
  skipped: 0
  duration: 62ms

ASCII art always manages to put a smile on my face! Anyway, we’ve successfully run our tests, and they’ve all passed! 🎉

One important thing to note is that code coverage (--coverage) and TRX test reports (--report-trx) need an extension to work. You’ll first need to install the Microsoft.Testing.Extensions.CodeCoverage and Microsoft.Testing.Extensions.TrxReport, respectively.

IDE Support Link to heading

Currently, we’ve been using the dotnet cli to run our tests, but a lot of developers, including myself, like to run our tests in our IDEs. This is possible, but requires a few steps to enable. TUnit has support for Visual Studio, Visual Studio Code, and JetBrains Rider. Depending on your choice of IDE, you might need to install the latest (Preview) version of your IDE to see the options below.

• Visual Studio
  • Enable “Use testing platform server mode” in Tools -> Options -> Environment -> Preview Features.
• Visual Studio Code
  • Install the C# Dev Kit extension.
    • While you’re at it, why not install my Readme Auto Open extension as well? 😉
  • Go to the extension settings.
  • Select “Use Testing Platform Protocol”.
• JetBrains Rider
  • Select “Enable Testing Platform support” in Settings -> Build, Execution Deployment -> Unit Testing -> VSTest.

Feature comparison with other testing frameworks Link to heading

I’ve mentioned that TUnit is faster than other testing frameworks, but how does it compare feature-wise? I’ve taken a look at some features I use quite often in xUnit and compared them to TUnit!

• You can use the [Arguments] arguments to pass in arguments to your test methods, similar to xUnit’s [Theory] and [InlineData] combo.
• You can use [ClassDataSource] and reference a class to inject it into your test method. When your test executes, it will instantiate the class and pass it to your test method. This is similar to xUnit’s [ClassData] attribute.
  • You can even set up a Shared property, which allows you to determine how the instance of that class should be shared between other tests. There are lots of exciting options like sharing test instances for the whole class, for the entire assembly, or even keyed, so you can share instances between tests that have the same key! 🤯
• Tests run in parallel by default, just like xUnit. However, TUnit has a lot more flexibility around parallelism! If you want to disable parallelism, add a [NotInParallel] attribute to your test class or method.
  • You can even apply several keys to the NotInParallel attribute to allow tests to run in parallel, but not with other tests that have the same key! 🤯
  • Alternatively, you can use Parallel Groups to group tests together that should run in parallel, and exclude parallelization for tests that are not in the same group!
  • You can also set up an Order property to determine the order in which your tests run.
  • You can even limit the amount of parallelization by using ParallelLimiter!
• In the same vein, you can use [DependsOn] to specify that a test depends on another test, which can be useful for integration tests that build on top of each other! In XUnit, this would require a lot more setup!

This is just a small batch of comparisons. I could make many more! The more I look at TUnit, the more I see that it’s not just a faster testing framework, but also a more feature-rich one! Some of these features are quite difficult to use in other testing frameworks, so this is definitely a breath of fresh air.

After writing this section, I found the official documentation from TUnit about the comparisons and differences from other testing frameworks. Take a look!

Benchmarks Link to heading

Alright, I believe I’ve hyped up TUnit enough. Let’s see if the benchmarks hold up! 🔍

TUnit has already published benchmark results. Therefore, I want to keep mine simple and compare TUnit to XUnit, with the exact same test setup.

Setting up the xUnit test project Link to heading

Let’s execute some commands to set up a new xUnit project in the root of our project, add it to the solution, and reference the source project. We’ll then build the solution.

# Let's create a new xUnit project in the root of our project
dotnet new xunit -o SampleProject.XUnit

# Now we need to add it to the solution and reference the source project
dotnet sln add SampleProject.XUnit
cd SampleProject.XUnit
dotnet add reference ../SampleProject/SampleProject.csproj

# Let's remove the default test file
rm UnitTest1.cs

# Let's go back to the root and build the solution
cd ../
dotnet build --configuration Release

Now, let’s recreate the CalculatorTests class in the SampleProject.XUnit project:

namespace SampleProject.XUnit;
public class CalculatorTests
{
    [Fact]
    public void Add_WhenCalled_ReturnsTheSumOfArguments()
    {
        // Arrange
        var calculator = new Calculator();

        // Act
        var result = calculator.Add(1, 2);

        // Assert
        Assert.Equal(3, result);
    }

    [Fact]
    public void Add_WhenOverflowing_ReturnsIntMinValue()
    {
        // Arrange
        var calculator = new Calculator();

        // Act
        var result = calculator.Add(int.MaxValue, 1);

        // Assert
        Assert.Equal(int.MinValue, result);
    }
}

Finally, let’s run the tests:

# dotnet build --configuration Release
# dotnet test --configuration Release --no-build
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.2+699d445a1a (64-bit .NET 9.0.0)
[xUnit.net 00:00:00.03]   Discovering: SampleProject.XUnit
[xUnit.net 00:00:00.05]   Discovered:  SampleProject.XUnit
[xUnit.net 00:00:00.05]   Starting:    SampleProject.XUnit
[xUnit.net 00:00:00.07]   Finished:    SampleProject.XUnit
  SampleProject.XUnit test succeeded (0,5s)

Test summary: total: 2; failed: 0; succeeded: 2; skipped: 0; duration: 0,4s
Build succeeded in 0,6s

We now have XUnit tests that run exactly the same tests as the TUnit tests. Let’s set up some benchmarks!

Setting up the benchmarks Link to heading

To run the benchmarks, we’ll want to use the BenchmarkDotNet package in another console project. Let’s set it up:

# Let's create a new console project in the root of our project
dotnet new console -o SampleProject.Benchmarks
dotnet sln add SampleProject.Benchmarks

# Let's add the necessary packages
# This is all inspired by TUnit's own benchmarks on their repo:
# https://github.com/thomhurst/TUnit/tree/main/tools/speed-comparison/Tests.Benchmark
cd SampleProject.Benchmarks
dotnet add package BenchmarkDotnet --version 0.14.0
dotnet add package CliWrap --version 3.6.7

# Let's build and see if all is well!
cd ../
dotnet build --configuration Release

The Program.cs in the SampleProject.Benchmarks project should contain the following::

// Thanks to TUnit for the inspiration
// https://github.com/thomhurst/TUnit/tree/main/tools/speed-comparison/Tests.Benchmark
using System.Runtime.InteropServices;

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;

using CliWrap;

BenchmarkRunner.Run(typeof(Program).Assembly);

public class RuntimeBenchmarks : BenchmarkBase
{
    [Benchmark]
    public async Task TUnit_AOT()
    {
        // Build the TUnit project using `dotnet publish -c Release --framework net9.0 --output aot-publish-net9 --property:Aot=true --runtime win-x64`
        // Change the runtime to your desired platform or omit it.
        await Cli.Wrap(Path.Combine(UnitPath, $"aot-publish-{Framework.Replace(".0", "")}", GetExecutableFileName()))
            .WithStandardOutputPipe(PipeTarget.ToStream(_outputStream))
            .ExecuteAsync();
    }

    [Benchmark]
    public async Task TUnit()
    {
        // Build the TUnit project using `dotnet build -c Release --framework net9.0`
        await Cli.Wrap("dotnet")
            .WithArguments(["run", "--no-build", "-c", "Release", "--framework", Framework])
            .WithWorkingDirectory(UnitPath)
            .WithStandardOutputPipe(PipeTarget.ToStream(_outputStream))
            .ExecuteAsync();
    }

    [Benchmark]
    public async Task XUnit()
    {
        // Build the xUnit project using `dotnet build -c Release --framework net9.0`
        await Cli.Wrap("dotnet")
            .WithArguments(["test", "--no-build", "-c", "Release", "--framework", Framework])
            .WithWorkingDirectory(XUnitPath)
            .WithStandardOutputPipe(PipeTarget.ToStream(_outputStream))
            .ExecuteAsync();
    }
}

[SimpleJob(RuntimeMoniker.Net90)]
public class BenchmarkBase
{
    protected readonly Stream _outputStream = Console.OpenStandardOutput();

    protected static readonly string UnitPath = GetProjectPath("SampleProject.TUnit");
    protected static readonly string XUnitPath = GetProjectPath("SampleProject.XUnit");

    protected static readonly string Framework = GetFramework();

    private static string GetFramework()
    {
        return $"net{Environment.Version.Major}.{Environment.Version.Minor}";
    }

    private static string GetProjectPath(string name)
    {
        var folder = new DirectoryInfo(Environment.CurrentDirectory);

        while (folder.Name != "tunit-playground")
        {
            folder = folder.Parent!;
        }

        return Path.Combine(folder.FullName, name);
    }

    protected string GetExecutableFileName()
    {
        return RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? "SampleProject.TUnit.exe" : "SampleProject.TUnit";
    }
}

We can now run this by calling dotnet run --configuration Release in the SampleProject.Benchmarks folder. This will run the benchmarks and show the results!

Benchmark results Link to heading

BenchmarkDotNet v0.14.0, Windows 11 (10.0.22631.4541/23H2/2023Update/SunValley3)
AMD Ryzen 9 7950X3D, 1 CPU, 32 logical and 16 physical cores
.NET SDK 9.0.100
  [Host]   : .NET 9.0.0 (9.0.24.52809), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
  .NET 9.0 : .NET 9.0.0 (9.0.24.52809), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

Job=.NET 9.0  Runtime=.NET 9.0

| Method    |     Mean |    Error |   StdDev |
| --------- | -------: | -------: | -------: |
| TUnit_AOT | 180.1 ms |  2.56 ms |  2.40 ms |
| TUnit     | 458.0 ms |  8.16 ms |  7.63 ms |
| XUnit     | 765.8 ms | 14.90 ms | 21.84 ms |

There we go! We can see that the TUnit tests are quite a bit faster! The AOT version is even faster than the regular version, which is quite impressive! We’re talking milliseconds here in a project with 2 small tests, but take a second to consider a big project with a lot of tests: This difference between TUnit and xUnit will save you a lot of time!

Keep in mind that I ran these tests on my own machine, so your results might be different. However, I believe that TUnit will keep being quite a bit faster!

For more benchmarks, take a look at the official ones by TUnit.

Finishing up Link to heading

That’s it for now! I wanted to keep this post a bit shorter and “introductory” compared to my other posts. December is a busy enough month already 😉.

What do you think of TUnit? Are you going to check it out?

If you’d like to learn more about TUnit, let me know in the comments below!

This post did quite well on Reddit, so if you want to read some comments and discussions, check it out on r/csharp or r/dotnet!

Finally, I’d like dedicate this post to the C# Advent Christmas 2024 Calendar, which is organized by Matthew Groves! These initiatives are always a lot of fun, and I’m glad to be a part of it!