/ Benchmarking

Is WCF faster than ASP.NET Core? Of course not! Or is it?

I was casually browsing Reddit when I came across a comment that triggered me. I’m paraphrasing[1], but the gist of it was that ‘WCF is faster than Web API or ASP.NET Core’. Surely, that was a mistake.

Are you coming to bed? - I can’t. This is important. - What? - Someone is wrong on the internet.
"Duty Calls" by xkcd is licensed under CC BY-NC 2.5.

Boy, I’ll show them. What are they claiming, actually? ‘The response times of a WCF service are much lower than those of ASP.NET Web API or ASP.NET Core MVC.’ Pfft. I’ll just write a small benchmark using trusty ol’ BenchmarkDotNet. Stand up a local web server, measure how long it takes to create a request, send it, deserialize it, generate a response, send that back, and deserialize the response. One method will use WCF, the other will use ASP.NET Web API. To quote a colleague of mine: ‘how hard could it be?’

If you’re only interested in the complete picture, take a look at the full table of results and the graphs.

First steps

I’m sending 100 very simple objects (a single property that is a GUID) to the API and it’s sending 100 items back. Writing the benchmark wasn’t hard, but processing the outcome was, kind of. WCF was faster than ASP.NET Web API.

Method Mean
Wcf 830,1 μs
WebApi 2 614,2 μs

And not just a little bit faster; WCF is putting Web API to shame by taking less than a third of the time. Alright, but Web API is a pretty obsolete technology. Surely the new and shiny ASP.NET Core MVC will do much better. Right?

Method Mean
Wcf 830,1 μs
WebApi 2 614,2 μs
AspNetCore 2 524,8 μs

Well, it’s a little faster, but WCF still takes less than a third of the time.

Different formats

What’s so different between WCF and the other two options? Well, there is a pretty obvious difference: WCF is serializing data to and from XML (SOAP, to be precise), while for Web API and ASP.NET Core MVC, I was defaulting to JSON. What if I force the APIs to use XML?

Method Mean
Wcf 830,1 μs
WebApiJson 2 614,2 μs
AspNetCoreJson 2 524,8 μs
WebApiXml 1 982,7 μs
AspNetCoreXml 1 933,5 μs

There’s a definite improvement there. WCF is still a lot faster, but at least this shows we can get better results by picking a different serializer.

MessagePack

So what’s the fastest serializer we can find? According to their own claims, MessagePack is a small and fast format, and Yoshifumi Kawai has written a high-performance MessagePack serializer for .NET that beats protobuf-net, the default Protobuf serializer for .NET. Protobuf was designed to be easy to serialize and is known for its performance. Let’s see.

Method Mean
Wcf 830,1 μs
WebApiJson 2 614,2 μs
AspNetCoreJson 2 524,8 μs
WebApiXml 1 982,7 μs
AspNetCoreXml 1 933,5 μs
WebApiMessagePack 1 615,7 μs
AspNetCoreMessagePack 1 483,8 μs

It’s going in the right direction. But can we go any faster? Probably not without extensive customization. However, these benchmarks have all been about serializing and deserializing 100 very simple objects. In the real world, objects are usually more complex than just a single GUID property.

Larger objects

Let’s try a more complex object.

public class LargeItem
{
    public Guid OrderId { get; set; }
    public ulong OrderNumber { get; set; }
    public string EmailAddress { get; set; }
    public Address ShippingAddress { get; set; } = new Address();
    public Address InvoiceAddress { get; set; } = new Address();
    public DateTimeOffset RequestedDeliveryDate { get; set; }
    public decimal ShippingCosts { get; set; }
    public DateTimeOffset LastModified { get; set; }
    public Guid CreateNonce { get; set; }
    public List<OrderLine> OrderLines { get; set; }
}

public class OrderLine
{
    public string Sku { get; set; }
    public int Quantity { get; set; }
    public string Product { get; set; }
    public decimal Price { get; set; }
}

public class Address
{
    public string Name { get; set; }
    public string Street { get; set; }
    public string HouseNumber { get; set; }
    public string PostalCode { get; set; }
    public string City { get; set; }
    public string Country { get; set; }
}

This is an order with order lines, copied from one of my other projects. There is a reasonable number of properties there, so let’s see how it goes.

Method Mean
LargeWcf 9 890,2 μs
LargeWebApiJson 25 425,6 μs
LargeAspNetCoreJson 40 312,9 μs
LargeWebApiXml 16 193,5 μs
LargeAspNetCoreXml 36 767,1 μs
LargeWebApiMessagePack 8 834,9 μs
LargeAspNetCoreMessagePack 8 813,8 μs

Bingo. When serializing and deserializing 100 ‘large’ items, MessagePack on either ASP.NET Web API or ASP.NET Core MVC beats WCF by a small margin.

#itdepends

As usual, the answer to the question ‘which is faster’ is: it depends. When performance is absolutely critical, and your service doesn’t have to be a public API that anyone can consume, WCF is probably your best bet, up to a certain point. When you need to communicate large amounts of complex data, MessagePack on top of ASP.NET Core MVC is probably a better solution. Also worth considering is the developer-friendliness, where WCF doesn’t score very high marks.

What about JSON? The default library for working with JSON in .NET is Newtonsoft.Json. It’s great when you have to have absolute control over how your JSON looks, but it’s by far the slowest option I’ve examined. Can we have our cake and eat it too?

Utf8Json

It turns out that, yes, we can. Sort of. Yoshifumi Kawai, who you’ll remember as the author of the very fast MessagePack serializer for .NET, has also written a performance-oriented JSON serializer for .NET called Utf8Json. It’s not as customizable and it doesn’t support DOM operations, but man, is it fast.

Method Mean
LargeWcf 9 890,2 μs
LargeWebApiJson 25 425,6 μs
LargeAspNetCoreJson 40 312,9 μs
LargeWebApiXml 16 193,5 μs
LargeAspNetCoreXml 36 767,1 μs
LargeWebApiMessagePack 8 834,9 μs
LargeAspNetCoreMessagePack 8 813,8 μs
LargeWebApiUtf8Json 13 787,9 μs
LargeAspNetCoreUtf8Json 13 961,1 μs

It’s not as fast as WCF, and definitely not as fast as MessagePack, but it easily takes less than half the time that Newtonsoft.Json does. It does this by avoiding memory allocations (similar to the MessagePack serializer) and by treating JSON as a binary format; it doesn’t serialize to a string which is then converted into bytes, but instead it serializes straight to UTF-8 bytes[2].

Conclusion

As I said before, it depends. If round-trip time performance is more important than pretty much anything else, but you still want to use a managed language, use WCF. If you care about performance, but also about legible and easy to use code, or you want to create a proper REST-ful service, use MessagePack or Utf8Json on top of ASP.NET Core MVC or ASP.NET Web API.

The code I’ve used to benchmark these libraries is available on GitHub. Feel free to play around with it, and maybe find even better options.

Data

Below you’ll find the ‘raw’ output from the benchmark I’ve run. Some annotations:

  • A method prefixed with ‘Small’ means it serializes and deserializes a small class with only a single GUID property. ‘Large’ means it serializes and deserializes the large class described above.
  • All WCF methods are hosted using WebServiceHost;
  • All WCF methods use a ChannelFactory client, while the Web API and ASP.NET Core MVC methods use System.Net.HttpClient[3];
  • WcfText uses a BasicHttpBinding with the message encoding set to Text;
  • WcfWebXml and WcfWebJson use a WebHttpBinding, with, respectively, XML or JSON as the message format;
  • JsonNet methods use Newtonsoft.Json to serialize objects;
  • I’ve included ‘0 items’ as an indication of how fast the client and web server/framework are;
  • The ‘P95’ column represents the 95th percentile value of the response time;
  • The ‘Gen 0’, ‘Gen 1’ and ‘Gen 2’ columns represent the number of garbage collections per 1 000 invocations;
  • The ‘Allocated’ column represents the total amount of memory allocated for each invocation;

Some interesting data points that jump out:

  • At 0 items, ASP.NET Core makes significantly less allocations, and as a result, it is the only option that results in no Generation 1 (or higher) allocations;
  • Even with 0 items, the options using Newtonsoft.Json are a lot slower than any of the other options. There must be a lot of initialization happening;
  • For larger item sizes, Utf8Json is frequently allocating the most memory of any option, but it still manages to be pretty fast. That just goes to show - avoiding memory allocation isn’t the only thing you need for speed;
  • ASP.NET Core’s XML serializer is a lot slower than Web API’s - this is particularly visible for large items;
  • WCF does very well in terms of allocated memory. If memory is constrained, WCF might be a viable option;

A note on the chart: I’ve tried my best to make it look good on desktop and on mobile, but the viewing experience is still best on a large screen.

Method ItemCount Mean P95 Gen 0 Gen 1 Gen 2 Allocated
SmallWcfText 0 365,8 μs 383,1 μs 8,3008 1,4648 - 30,08 KB
SmallWcfWebXml 0 378,9 μs 392,0 μs 9,2773 1,9531 - 33,28 KB
SmallWcfWebJson 0 396,1 μs 421,4 μs 9,7656 1,9531 - 34,6 KB
SmallWebApiJsonNet 0 1 802,4 μs 1 904,6 μs 35,1563 9,7656 - 122,74 KB
SmallWebApiMessagePack 0 931,8 μs 955,5 μs 17,5781 5,8594 - 62,95 KB
SmallWebApiXml 0 997,7 μs 1 067,0 μs 29,2969 7,8125 - 105,28 KB
SmallWebApiUtf8Json 0 925,5 μs 939,4 μs 17,5781 5,8594 - 66,17 KB
SmallAspNetCoreJsonNet 0 1 753,1 μs 1 776,1 μs 31,2500 - - 99,3 KB
SmallAspNetCoreMessagePack 0 853,2 μs 898,7 μs 14,6484 - - 46,21 KB
SmallAspNetCoreXml 0 955,9 μs 1 014,9 μs 31,2500 - - 98,16 KB
SmallAspNetCoreUtf8Json 0 914,4 μs 957,6 μs 12,6953 - - 41,6 KB
LargeWcfText 0 353,1 μs 361,0 μs 8,3008 1,4648 - 30,11 KB
LargeWcfWebXml 0 374,8 μs 382,4 μs 9,2773 1,9531 - 33,27 KB
LargeWcfWebJson 0 386,9 μs 398,0 μs 9,7656 1,9531 - 34,58 KB
LargeWebApiJsonNet 0 1 860,5 μs 1 900,6 μs 50,7813 7,8125 - 183,1 KB
LargeWebApiMessagePack 0 924,2 μs 937,6 μs 17,5781 5,8594 - 62,22 KB
LargeWebApiXml 0 1 000,3 μs 1 110,8 μs 32,2266 10,7422 - 109,81 KB
LargeWebApiUtf8Json 0 929,3 μs 939,1 μs 19,5313 6,8359 - 66,79 KB
LargeAspNetCoreJsonNet 0 1 848,1 μs 1 915,2 μs 50,7813 - - 159,7 KB
LargeAspNetCoreMessagePack 0 881,4 μs 918,2 μs 14,6484 - - 46,27 KB
LargeAspNetCoreXml 0 943,4 μs 974,8 μs 33,2031 - - 104,96 KB
LargeAspNetCoreUtf8Json 0 943,7 μs 991,1 μs 12,6953 - - 41,61 KB
SmallWcfText 10 465,0 μs 534,4 μs 9,7656 1,9531 - 35,29 KB
SmallWcfWebXml 10 472,3 μs 516,5 μs 11,2305 2,4414 - 38,44 KB
SmallWcfWebJson 10 457,7 μs 467,2 μs 11,7188 2,4414 - 39,84 KB
SmallWebApiJsonNet 10 1 856,1 μs 1 890,8 μs 42,9688 11,7188 - 152,07 KB
SmallWebApiMessagePack 10 930,9 μs 940,6 μs 17,5781 6,8359 - 67,73 KB
SmallWebApiXml 10 1 267,1 μs 1 463,6 μs 33,2031 5,8594 - 124,98 KB
SmallWebApiUtf8Json 10 960,2 μs 1 027,8 μs 21,4844 7,8125 - 74,07 KB
SmallAspNetCoreJsonNet 10 1 899,6 μs 1 950,9 μs 41,0156 - - 128,06 KB
SmallAspNetCoreMessagePack 10 902,6 μs 933,2 μs 15,6250 - - 50,96 KB
SmallAspNetCoreXml 10 1 051,0 μs 1 186,4 μs 37,1094 - - 118 KB
SmallAspNetCoreUtf8Json 10 996,0 μs 1 057,8 μs 13,6719 - - 46,21 KB
LargeWcfText 10 1 471,8 μs 1 731,5 μs 35,1563 9,7656 - 125,85 KB
LargeWcfWebXml 10 1 430,8 μs 1 564,6 μs 42,9688 13,6719 - 160,92 KB
LargeWcfWebJson 10 1 669,1 μs 1 822,5 μs 41,0156 9,7656 - 145,14 KB
LargeWebApiJsonNet 10 10 642,6 μs 11 767,8 μs 250,0000 31,2500 - 776,28 KB
LargeWebApiMessagePack 10 1 945,1 μs 2 036,7 μs 74,2188 11,7188 - 255,49 KB
LargeWebApiXml 10 2 487,6 μs 2 522,3 μs 128,9063 19,5313 - 421,66 KB
LargeWebApiUtf8Json 10 1 949,1 μs 2 098,9 μs 82,0313 15,6250 - 278,19 KB
LargeAspNetCoreJsonNet 10 10 367,4 μs 10 987,8 μs 203,1250 62,5000 - 746,2 KB
LargeAspNetCoreMessagePack 10 1 946,0 μs 1 975,3 μs 72,2656 1,9531 - 228,89 KB
LargeAspNetCoreXml 10 2 463,8 μs 2 521,2 μs 117,1875 27,3438 - 392,83 KB
LargeAspNetCoreUtf8Json 10 1 977,8 μs 2 084,4 μs 74,2188 1,9531 - 231,07 KB
SmallWcfText 100 830,1 μs 871,4 μs 27,3438 3,9063 - 91,15 KB
SmallWcfWebXml 100 961,7 μs 1 062,7 μs 29,2969 3,9063 - 94,82 KB
SmallWcfWebJson 100 1 188,1 μs 1 359,0 μs 27,3438 3,9063 - 94,06 KB
SmallWebApiJsonNet 100 2 614,2 μs 2 688,8 μs 82,0313 11,7188 - 297 KB
SmallWebApiMessagePack 100 1 615,7 μs 1 762,6 μs 39,0625 7,8125 - 133,62 KB
SmallWebApiXml 100 1 982,7 μs 2 020,6 μs 78,1250 11,7188 - 270,55 KB
SmallWebApiUtf8Json 100 1 816,0 μs 1 860,2 μs 41,0156 7,8125 - 146,35 KB
SmallAspNetCoreJsonNet 100 2 524,8 μs 2 608,3 μs 82,0313 - - 263,7 KB
SmallAspNetCoreMessagePack 100 1 483,8 μs 1 792,1 μs 31,2500 - - 101,22 KB
SmallAspNetCoreXml 100 1 933,5 μs 1 960,5 μs 74,2188 - - 239,64 KB
SmallAspNetCoreUtf8Json 100 1 777,2 μs 1 831,4 μs 37,1094 - - 114,71 KB
LargeWcfText 100 9 890,2 μs 10 865,7 μs 250,0000 125,0000 15,6250 1 261,97 KB
LargeWcfWebXml 100 10 136,1 μs 11 027,3 μs 265,6250 140,6250 31,2500 1 266,48 KB
LargeWcfWebJson 100 12 813,6 μs 13 895,2 μs 328,1250 171,8750 15,6250 1 417,78 KB
LargeWebApiJsonNet 100 25 425,6 μs 26 789,2 μs 906,2500 468,7500 62,5000 3 588,29 KB
LargeWebApiMessagePack 100 8 834,9 μs 9 149,7 μs 578,1250 328,1250 62,5000 2 465,68 KB
LargeWebApiXml 100 16 193,5 μs 16 481,3 μs 750,0000 406,2500 62,5000 2 934,26 KB
LargeWebApiUtf8Json 100 13 787,9 μs 14 252,9 μs 687,5000 437,5000 140,6250 3 677,67 KB
LargeAspNetCoreJsonNet 100 40 312,9 μs 43 191,0 μs 625,0000 312,5000 62,5000 3 685,57 KB
LargeAspNetCoreMessagePack 100 8 813,8 μs 10 180,6 μs 312,5000 187,5000 62,5000 2 292,35 KB
LargeAspNetCoreXml 100 36 767,1 μs 39 825,2 μs 562,5000 312,5000 62,5000 2 911,21 KB
LargeAspNetCoreUtf8Json 100 13 961,1 μs 14 168,9 μs 484,3750 343,7500 156,2500 3 673,72 KB
SmallWcfText 1000 5 299,2 μs 6 077,0 μs 195,3125 54,6875 - 646,64 KB
SmallWcfWebXml 1000 5 758,0 μs 7 026,5 μs 187,5000 54,6875 - 649,61 KB
SmallWcfWebJson 1000 6 747,8 μs 7 438,1 μs 195,3125 62,5000 - 686,42 KB
SmallWebApiJsonNet 1000 9 909,7 μs 10 125,5 μs 515,6250 203,1250 - 1 625,45 KB
SmallWebApiMessagePack 1000 4 187,4 μs 4 299,4 μs 238,2813 85,9375 - 768,83 KB
SmallWebApiXml 1000 9 301,4 μs 9 428,8 μs 484,3750 171,8750 15,6250 1 652,37 KB
SmallWebApiUtf8Json 1000 4 849,8 μs 4 906,9 μs 250,0000 93,7500 - 841,2 KB
SmallAspNetCoreJsonNet 1000 18 942,1 μs 19 746,0 μs 406,2500 125,0000 - 1 566,54 KB
SmallAspNetCoreMessagePack 1000 3 682,6 μs 3 756,6 μs 171,8750 54,6875 - 694,65 KB
SmallAspNetCoreXml 1000 25 635,3 μs 26 039,3 μs 406,2500 125,0000 - 1 627,79 KB
SmallAspNetCoreUtf8Json 1000 4 896,0 μs 4 979,0 μs 179,6875 54,6875 - 741,7 KB
LargeWcfText 1000 111 347,5 μs 113 444,9 μs 3 187,5000 1 500,0000 875,0000 26 256,54 KB
LargeWcfWebXml 1000 110 655,4 μs 119 280,8 μs 3 062,5000 1 312,5000 625,0000 26 240,84 KB
LargeWcfWebJson 1000 134 667,6 μs 144 168,1 μs 3 687,5000 1 187,5000 500,0000 19 557,27 KB
LargeWebApiJsonNet 1000 173 466,7 μs 201 296,3 μs 6 812,5000 1 312,5000 625,0000 28 508,8 KB
LargeWebApiMessagePack 1000 77 229,8 μs 79 016,2 μs 5 812,5000 1 500,0000 625,0000 26 610,34 KB
LargeWebApiXml 1000 155 180,8 μs 156 540,0 μs 6 937,5000 1 562,5000 937,5000 32 755,87 KB
LargeWebApiUtf8Json 1000 113 577,7 μs 115 954,0 μs 6 562,5000 1 937,5000 1 312,5000 36 599,17 KB
LargeAspNetCoreJsonNet 1000 364 533,8 μs 366 468,4 μs 4 562,5000 1 250,0000 562,5000 29 998,47 KB
LargeAspNetCoreMessagePack 1000 75 584,5 μs 76 337,7 μs 3 125,0000 1 312,5000 625,0000 24 512,12 KB
LargeAspNetCoreXml 1000 355 358,8 μs 359 180,4 μs 4 000,0000 1 187,5000 625,0000 29 263,61 KB
LargeAspNetCoreUtf8Json 1000 105 684,4 μs 107 125,5 μs 4 000,0000 2 000,0000 1 312,5000 35 664,96 KB

  1. I’ve spent an inordinate amount searching for it, but I cannot find the original thread or comment. You’ll just have to take my word for it. ↩︎

  2. According to RFC 7159, valid encodings for JSON are UTF-8, UTF-16 or UTF-32, with the default being UTF-8. ↩︎

  3. I did try using RestSharp, but it’s horribly slow. ↩︎