A lot of new performance-related stuff landed in the F# world recently. As well as the release of .NET 6, F# 6 was recently released, with built-in, highly-optimized "Resumable Code" Tasks. So let's measure it.

This blog post provides benchmark results that answer the following questions:

• How much faster is .NET 6 for running F#, vs .NET 5?
• How much faster is F#6 vs F# 5 (both measured in .NET6)?
• How does the performance of  Tasks improve due to .NET 6, compared to the various different Task and Async libraries?
• How does F# 6 Resumable Code compare to these various Task and Async libraries, using F#6 and .NET6?
• How much overhead is Task performance costing us (versus a purely synchronous model), in F# 6/.NET 6?

F# 6's new Task support is especially interesting because it has a new compilation model for Tasks. Rather than using a standard Computation Expression, where let! and such turn into Builder.Bind method calls, F# 6 Tasks compile into Resumable Code. You don't need to understand Resumable Code to use it, but if you'd like to, there's an excellent video where Don Syme steps through how this all works. Essentially, there's a big switch statement added around your code, allowing you to very quickly goto the right place when a Task is finished. Lots of other languages do the same thing, including the await  keyword in JS and C#.

I'd like to know how much faster this is compared to existing work. But, there's a lot of work to choose from. There's F#'s built-in async of course, but there's also two existing Task libraries: TaskBuilder.fs and Ply. And then within Ply, there are four interesting implementations:

• task: Used to create regular tasks
• vtask: Uses ValueTasks instead of Tasks
• uvtask: An "unsafe" version of ValueTasks which doesn't sync async local and synchronization changes
• uply: An unsafe version which skips allocating Tasks at all in many cases.

Overall, the Task/Async implementations we'll be benchmarking are TaskBuilder.fs, Async, Ply's task, vtask, uvtask and uply, and F#6's Resumable Code.

F# 6 was also released as part of .NET6, though you can use F# 5 in .NET 6 if you choose. This allows us to see what performance improvements come from the improvements to .NET vs the improvements to F#!

The results

The start off with, here are the results in graph form and table form.

The benchmark used here is FizzBoom, which implements a cut-down version of the Darklang interpreter, interpreting a program which prints the Fizzbuzz results via JSON in a HTTP server. All tests used the same HTTP server (Giraffe).

The results show the number of requests per second, and more is better. Each result is measured on .NET 5 with F# 5, .NET 6 with F# 5, and .NET6 with F# 6 (except for F# 6 Resumable Code, which is only available on the latter). The servers are started and warmed up for two seconds before the measurement starts.

How much faster is F# in .NET 6?

When looking at raw compute performance (and I should say that there are probably some better benchmarks out there for this, but whatever), it looks like F#6 on .NET6 does about 4% better than F#5 on .NET 5 did. So a little improvement, but not a lot. (It looks like 1 of those 4 percentage points comes from F# speed improvements, and 3 from .NET).

This is just raw CPU improvement, so don't be too underwhelmed - it wasn't really expected to go up and this is just a freebie.

Performance of Tasks between .NET/ F# versions

Now let's look at Tasks. IO in general has been given a big performance boost in .NET6, so let's compare all the Task implementations across .NET5, .NET6, and F#6. (We're saving F#6 Resumable Code for later).

It seems like we've got some good performance results here. Async is 12% faster in .NET6/F#6, TaskBuilder.fs is 17% faster, Ply Tasks are 18% faster and Ply vtasks are 13% faster.

We also see the Ply Task-compatible CEs (uvtask and uply) have nice performance improvements of 18% and 9% respectively.

The performance of Ply remains interesting, with ValueTasks and Ply-specific Task implementations continuing to be very impressive. uply has impressive gains: in .NET 5, using uply was 8% slower than keeping the code synchronous, now that has dropped to a mere 2.9%!

Another interesting thing is the regression in Async code. There was a great improvement in .NET 6, but the "upgrade" to F# 6 dropped the performance of Async by 8% after an otherwise phenomenal 22% gain from .NET 6.

Performance of F# 6 Resumable Code

Now let's look at Resumable Code, F# 6's flagship feature. Compared to Ply' and TaskBuilder.fs's task CEs, which are pretty much the same performance as each other, F# 6's Resumable Code is massive 32% faster. If we add that to the overall performance improvements from .NET6 and F# 6, we see an almost unbelievable 55% improvement. If you were using Async before, that jumps to a staggering 118% improvement, almost twice as fast. So performance here is so good I'm basically running out of superlatives.

Room for improvement

That isn't to say that the F# 6 Resumable Code is the end of the line, performance-wise. Resumable Code is a performance improvement over Ply's task, but Ply's vtask) which uses ValueTasks instead of Tasks) is 14% faster than Ply's task, and there isn't a good reason to think that when we apply Resumable Code to vtask (ValueTasks), that we wouldn't get a similar improvement. I'm told that this work is planned for a future version of F#.

Ply's Task-compatible CEs, namely uply and uvtask, are extremely high performant. Though they drop in nicely with the same interface and functionality, they are not actually Tasks, and so do not produce values that are interoperable. They also do not resume the "execution bubble" (capturing and restoring async local variables and synchronization context), which is why they're called "unsafe".

I would expect some of the same improvement also to uvtask, and possibly some performance improvement to uply, though uply is so close to the performance of not using Tasks at all that's it's hard to improve on, but it feels like that's the potential here.

Nino Floris, who wrote Ply and was heavily involved in F# 6's Resumable Code, tells me the main advantage for uply from Resumable Code is that we can make a safe version of uply, keeping the "execution bubble" intact.

Summary

To summarize the findings here:

• CPU-bound code is 4% faster in .NET 6/ F# 6
• In addition, .NET 6 provides a 15% performance boost for Tasks over .Net 5
• F# Resumable Code offers a performance boost of over 30% over using existing Tasks in F# 5
• Performance of Async regressed in F# 6 by 8% after speeding up by 22% due to .NET 6
• Ply's unsafe CEs, uvtask and uply, still retain a significant speed advantage over Tasks
• Combining Resumable Code with ValueTasks and Ply's unsafe CEs seems an exciting area for future work.

Thanks to Nino Floris and Don Syme for reading a draft of this post. Published as part of #FsAdvent.

This was all part of performance tests for Darklang - a programming language, editor, and cloud infrastructure, aiming to make it 100x easier to write cloud services. Test it out at https://darklang.com.