Benchmark Codes

Performance and Benchmarking#

To get us started, first, we need to talk a little bit about performance and benchmarking. When we are evaluating software, we are looking for code quality, use of design patterns, and performance. Evaluating and improving performance is probably one of the hardest parts of developing software, but it’s what makes our product or service valuable. Performance is how well an application or a unit within the application performs (usually in terms of responsiveness) under a particular workload.

Good software performs well. In other words, good software is “fast” under heavy workloads.

So how do we know how “fast” our code is? Benchmarks.

We can think of a benchmark like a timer. When we are attempting to optimize our code, we can use benchmarks to verify whether our code has truly been optimized. Suppose we have an ASP.NET Core REST API and we wish to optimize our project by migrating our REST API to gRPC. Is this optimization worth it? There are many other uses we can consider. Should we use Razor pages or MVC patterns? Should we use a custom ORM or EF Core? These are questions that can be difficult to answer without running proper benchmarks. In our example for this article, we are going to look at how the REST protocol compares to the gRPC protocol.

We can use benchmarks to compare the performance of our original code versus the performance of our optimized code. Based on those benchmarks, we can make the appropriate decision.

That’s enough theory for now. Let’s take a look at what benchmarking really looks like.

Benchmarking in C##

BenchmarkDotNet is a lightweight, open source, powerful .NET library that can transform your methods into benchmarks, track those methods, and then provide insights into the performance data captured. It is easy to write BenchmarkDotNet benchmarks and the results of the benchmarking process are user friendly as well.

Demo#

Let’s continue with our example of searching an item in dictionary.

Imagine we have a dictionary which have lot of strings and we have to get a value of given key.

Create a console application BenchmarkingDemo in Visual Studio Code.

dotnet new console -o BenchmarkingDemo

Add the reference of BenchmarkDotNet nuget package.

dotnet add package BenchmarkDotNet

Now add a class DictionaryLookup and add the following code.

public class DictionaryLookup
{
   readonly Dictionary<string, string> MyDictionary = new();

   public DictionaryLookup()
   {
     for(int i =0;i < 100; i++)
     {
       MyDictionary[i.ToString()] = $"test_{i}";
     }
   }
}

Add a method to get the value by key.

public string GetValueByKey(string key)
{
    if (MyDictionary.ContainsKey(key))
    {
      return MyDictionary[key];
    }
    return null;
}

This will be our benchmark method.

Add some more ways of getting the value by key.

Now we have to add benchmark class. Let’s add BenchmarkDictionaryLookup.cs

[MemoryDiagnoser]
[Orderer(BenchmarkDotNet.Order.SummaryOrderPolicy.FastestToSlowest)]
[RankColumn]
public class BenchmarkDictionaryLookup
{
    DictionaryLookup obj = new DictionaryLookup();

    [Benchmark(Baseline = true)] // mark this is as baseline method.
    public void GetValueByKey()
    {
        obj.GetValueByKey("test_99"); // i am doing it for worst case by searching the last item.
    }
    [Benchmark]
    public void GetValueByKeyWithLinq()
    {
        obj.GetValueByKeyWithLinq("test_99");
    }
    [Benchmark]
    public void GetValueByKeyWithTryGet()
    {
        obj.GetValueByKeyWithTryGet("test_99");
    }
    [Benchmark]
    public void GetValueByKeyManual()
    {
        obj.GetValueByKeyManual("test_99");
    }
}

Finally add the code in Program.cs

BenchmarkRunner.Run<BenchmarkDictionaryLookup>();

Run the application in release mode. open cmd prompt and go to root of the application.

dotnet run -p BenchmarkingDemo.csproj -c Release

After benchmarking is completed you will see the following summary result.

Results are also stored inside folder “BenchmarkDotNet.Artifacts” under the application’s root folder

Result Explanation#

From results it is seen that our benchmark method GetValueByKey and GetValueByKeyWithTryGet ranks 1 with 8 ns to lookup, where as GetValueByKeyWithLinq ranks last with 1773 ns.

Also the GetValueByKeyWithLinq method have some memory footprint as seen the screenshot.

By looking at the timing results you might say how it is going make difference but if this operation performed 1 millions times in day then it would make the difference.

So next time when you have to write the code to get value from dictionary then you know which method to use and which one to avoid.

Learn More#

• Overview | BenchmarkDotNet
• Introduction to Benchmarking in C# and ASP.NET Core Projects
• Benchmarking in C# Using BenchmarkDotNet