Maximize .NET LINQ performance using ZLinq

LINQ is one of the standout features of .NET and is loved by many C# developers, for whom it has become second nature in their day-to-day work. The declarative nature of LINQ allows for expressive queries over collections to be composed, resulting in highly readable and maintainable code.

However, while LINQ provides powerful abstractions and can provide a major boost in developer productivity, it often doesn’t lead to the highest performance solution due to inefficiencies such as memory allocations. This can be particularly noticeable in performance-critical applications that are connected to large datasets and where many LINQ operations need to be chained.

In cases where we need to optimise our applications further, wouldn’t it be nice if we didn’t need to sacrifice the benefits of LINQ by rewriting code in a more procedural manner for the sake of performance? That’s where ZLinq comes in, offering practically all the same functionality as LINQ (and more), while providing a performance boost at the same time.

In this article, I will introduce you to ZLinq, show you how easy it is to get started with it, and share the results of a simple benchmark that compares the performance with standard LINQ.

What is ZLinq?

ZLinq is a performance-focused version of LINQ developed by Yoshifumi Kawai, currently the Founder/CEO/CTO of Cysharp, and author of several other notable open-source projects, including MemoryPack and ZString.

The ZLinq GitHub repository features an impressively detailed ReadMe that highlights the following features.

• 99% compatibility with .NET 10’s LINQ (including new Shuffle, RightJoin, LeftJoin, Sequence, InfiniteSequence operators)
• Zero allocation for method chains through struct-based Enumerable via ValueEnumerable
• LINQ to Span to full support LINQ operations on Span<T> using .NET 9/C# 13’s allows ref struct
• LINQ to Tree to extend tree-structured objects (built-in support for FileSystem, JSON, GameObject)
• LINQ to SIMD to automatic application of SIMD where possible and customizable arbitrary operations
• Optional Drop-in replacement Source Generator to automatically accelerate all LINQ methods

A key point to note is the 99% compatibility with standard LINQ, which is backed by over 9000 passing tests from System.Linq.Tests in addition to ZLinq-specific tests.

It’s also pretty cool that ZLinq makes the newest LINQ methods, such as Shuffle and LeftJoin available to older versions of .NET, including .NET Framework 4.8!

So it seems that there’s a lot to love about ZLinq, and with that, let’s get started with it in the next section.

Getting started

To get started with ZLinq, you must first install the ZLinq NuGet package as follows.

dotnet add package ZLinq

Then import the ZLinq namespace.

using ZLinq;

Here’s a straightforward example of using ZLinq in a C# Console App to filter and sort a list of numbers.

using ZLinq;
 
var numbers = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
 
var evenNumbers = numbers
    .AsValueEnumerable()
    .Where(x => x % 2 == 0)
    .OrderDescending()
    .ToList();

As you can see from the above code sample, this looks just like regular LINQ. The only difference is the call to the AsValueEnumerable method, which converts existing collections to a type that can be chained with ZLinq methods.

Surprisingly, that’s all there is to it! You can begin transitioning parts of your codebase by selectively replacing or supplementing standard LINQ calls where needed.

In addition to the selective approach, it’s worth pointing out that ZLinq offers the concept of drop-in replacements if you want to apply ZLinq optimisations across your codebase, targeting arrays or lists, etc. It achieves this via a Source Generator which adds extension methods that take priority over the LINQ methods on each type.

If you want to go down the drop-in replacement route, you’ll need to install the ZLinq.DropInGenerator NuGet package. and follow the drop-in replacement instructions to enable the overrides.

Measuring performance

So, how much performance gain can we reasonably expect from ZLinq?

We can use the ever-popular BenchmarkDotNet NuGet package to run performance benchmarks that will measure the performance of ZLinq versus standard LINQ.

Below is an example of some basic benchmarks that we can set up within a C# Console App to test things out.

using BenchmarkDotNet.Attributes;
using ZLinq;
 
[MemoryDiagnoser(displayGenColumns: false)]
public class Benchmarks
{
    private int[] _numbers = [];
 
    [GlobalSetup]
    public void Setup()
    {
        var random = new Random(12345);
 
        _numbers = Enumerable.Range(0, 10_000)
            .Select(x => random.Next(0, 10_000))
            .ToArray();
    }
 
    [Benchmark]
    public long Linq()
    {
        return _numbers
            .Where(n => n % 2 == 0)
            .Select(n => n * 2)
            .Sum();
    }
 
    [Benchmark]
    public long ZLinq()
    {
        return _numbers
            .AsValueEnumerable()
            .Where(n => n % 2 == 0)
            .Select(n => n * 2)
            .Sum();
    }
 
    [Benchmark]
    public long ForLoop()
    {
        long sum = 0;
 
        for (int i = 0; i < _numbers.Length; i++)
        {
            int n = _numbers[i];
 
            if (n % 2 == 0)
            {
                sum += n * 2;
            }
        }
 
        return sum;
    }
}

In the above code, the MemoryDiagnoser attribute is applied to the Benchmarks class to enable memory allocation diagnostics for the benchmarks, while suppressing the detailed generation breakdown (Gen0/Gen1/Gen2 columns) via the displayGenColumns parameter.

The Setup method, which is decorated with the GlobalSetup attribute, configures the data that will be used by each of the tests, creating the same array of random integer numbers every time the tests are run by specifying a specific seed value to the Random constructor.

Each benchmark method is decorated with the Benchmark attribute to flag them as part of the performance test suite.

The Linq, ZLinq, and ForLoop methods are all doing essentially the same thing: filtering the array to only even numbers, squaring each number, and returning the sum.

The methods have been declared in the order of expected performance, from worst to best.

Running the benchmarks

To run the performance tests, we can add the following code to the ‘Program.cs’ file of a C# Console App.

using BenchmarkDotNet.Running;
 
BenchmarkRunner.Run<Benchmarks>();

The above code will run the benchmarks and output results in a format that is similar to the following.

| Method  | Mean     | Error    | StdDev   | Allocated |
|-------- |---------:|---------:|---------:|----------:|
| Linq    | 64.58 us | 0.432 us | 0.361 us |     104 B |
| ZLinq   | 51.05 us | 0.799 us | 1.267 us |         - |
| ForLoop | 42.33 us | 0.070 us | 0.062 us |         - |

The above results show that ZLinq offers a noticeable performance improvement over LINQ in this simple example, sitting between the performance of LINQ and a For Loop implementation at around 51 microseconds. One of the key details to notice is that while LINQ allocated 104 bytes, ZLinq did not allocate any memory for the operation.

.NET 9.0+

The benchmarks demonstrate a nice improvement in performance, and one can imagine how the gains could add up when dealing with more complex code and larger volumes of data.

However, what if we could get even more gains by simply upgrading to a newer version of .NET?

Well, with .NET 9 and C# 13, ZLinq is able to automatically add allows ref struct to the ValueEnumerable class definition and ZLinq method constraints. A ref struct is a special kind of struct that has restrictions to ensure it always stays on the stack and never gets boxed or allocated on the heap, enabling better overall performance.

Running the same tests with the project converted from .NET 8.0 to .NET 9.0 yields the following results.

| Method  | Mean     | Error    | StdDev   | Allocated |
|-------- |---------:|---------:|---------:|----------:|
| Linq    | 53.34 us | 0.196 us | 0.164 us |     104 B |
| ZLinq   | 45.51 us | 0.172 us | 0.153 us |         - |
| ForLoop | 46.23 us | 0.101 us | 0.089 us |         - |

Interestingly, this time, while LINQ and ZLinq are both faster than before, the For Loop is slower, and ZLinq is the fastest of all three methods.

Of course, results may vary slightly each time the benchmarks are run. However, shaving another 5.5 microseconds off by simply upgrading the version of .NET is pretty nice, and .NET 10.0 is not far away, with further improvements likely.

Summary

In this article, I provided an introduction to ZLinq, what it is, and how to get started with it.

I then demonstrated how to set up a basic set of benchmarks using BenchmarkDotNet and showcased the benefits of upgrading to the latest .NET version to realise additional performance gains.

There’s a lot more to ZLinq than meets the eye. In this blog post, I’ve only scratched the surface, and there is a great deal of helpful information on the ZLinq GitHub page that covers many of the different ways you can use ZLinq.

If you’re interested in the technical architecture, I recommend that you check out this blog post by Yoshifumi Kawai that goes into the details of how ZLinq was developed.

Let me know in the comments if you’re using ZLinq for any production projects!