Welcome again HN! Dark is a programming language, structured editor, and infrastructure—all in one—whose goal is to make it 100x easier to build backend services. Check out the website, our What is Dark post, and How Dark deploys in 50ms for more. Thanks for checking us out!

This is the third or a 3-part series: Leaving OCaml and Dark's new backend will be in F#. You can enjoy this without reading the previous posts.

With the election in the state it is, I'm going to stop pretending that I can do work right now. So instead, I'll just milk the success of my last two posts and hope that none of you are really working right now either.

As discussed in the two previous posts, Dark is moving to F#. This has been a bit of a surprise to people, including me. We've spoken for years about the inevitable Rust rewrite; we have a CLI written in Rust and two services, so I was pretty sure that it was going to be Rust.

Why not Clojure/Haskell/Scala?

People asked about a few other languages as well, so let's get them out of the way first.

Clojure

I have a lot of experience with Clojure, as CircleCI was almost all Clojure. However, we spent a whole lot of time with dealing accidental complexity, specifically "what type is this field" and nulls all over the place. So I deliberately chose not to have a dynamically typed language, even though Clojure is a lovely language. A side benefit is to escape the Cult of Rich in the Clojure community. Hearing his Maybe Not talk really cemented for me how deep down the dynamically typed rabbit hole they are over there, and how much I disagree with that.

Haskell

I had previously tried to love Haskell, trying to write an interpreter in it while I was at the Recurse Center, and I did not like it. HN user momentumtop's explanation match my feelings exactly:

The Haskell community, in my experience, is far more academic. A recent post to the Haskell libraries mailing list began with:

"It was pointed out to me in a private communication that the tuple function \x->(x,x) is actually a special case of a diagonalization for biapplicative and some related structures monadicially.

It received 39 pretty enthusiast replies.

Scala

I have no experience with Scala, but my overwhelming sense of the language and the community is that the whole thing is a mess. So I didn't consider it, and still wouldn't.

Ok, why not Rust?

I actually wrote quite a bit on why I didn't like Rust a few weeks ago. I think those main reasons stand, so I'll just link to them rather than repeat the 1800 words again. As a quick summary, the good parts were:

• tooling is great
• library ecosystem is great
• community is great
• macros are nice (though I feel I was overusing them to cover problems in the language)

and the bad parts were

• having to do memory management sucks
• pattern matching doesn't work all that well
• too many ways to do things (Arc vs Rc, async vs sync, different stdlibs)
• the language isn't immutable
• having to fight the compiler

Again, for more on those, have a read of the previous post.

Ultimately, when it came time to decide, it came down to a few major things: missing a GCP library, and the low-level nature of the language.

Libraries

Rust has a ton of libraries, and they work really well and are nicely integrated. They have 3rdparty libraries for Honeycomb, LaunchDarkly, and Rollbar, which are important services for us. However, the library for GCP seems super sketch. It's autogenerated and the issues imply that they've gone as far as they can using this technique. So that seemed very risky to take on, given that the whole point was to have a much richer library ecosystem.

Async

When I wrote the previous post, I had just gotten the synchronous version of the benchmark to work in Rust. Then I tried to make it async. I really struggled with making things async. Apparently, recursion adds a new level of complexity to async. But the thing that killed me was pinning.

Let's see if I can explain this. When you're writing an async, multi-threaded server in using the tokio runtime, async processes can be moved between threads. This means the memory can be copied, and so you need to ... pin things? OK, that's as much as I remember. Look in the HN comments after I publish this and I'm sure someone will explain better. The code is over here if you're interested.

I tried to get my head around this for some time, before deciding that this was a waste of time. Apparently, this boxing and pinning is what you get when you don't have a GC, and that when you do have a GC, you simply don't need to deal with it. So that was the final straw for me.

Rust is a low-level language

I'm implementing a language that's basically F#/OCaml. So it makes sense that it's easier to implement in F#/OCaml. A few people pointed out that I'm trying to write OCaml in Rust, and that's not really what it was designed for. I think that's right. Rust's semantics makes many things easy, but not what I'm trying to do.

I think most of us don't need Rust. I think Rust is a wonderful community, ecosystem, and tooling, wrapping a language that nicely solves a problem very few of us have. It's just so nice over there, until you actually write code.

It's easy to forget, given how nice everything is with the error messages and the docs, that Rust is a very low-level language. We're so attracted to the community and the tooling that we forget that low-level languages suck. Maybe Rust has a better story than most low-level languages, but remember that garbage collectors are great. By having a GC, we don't have to do any of the stuff that causes all these problems in Rust. Maybe that costs performance, but I need the ability to quickly write code a lot more than I need the extra performance.

And ultimately, that's why I picked F#.

You can sign up for Dark here. For more info on Dark, follow our RSS, follow us (or me) on Twitter, join our Slack Community, watch our GitHub repo, or join our mailing list.

Thanks to Joël Franusic, Jonny Sywulak and Luca Palmieri for feedback on drafts of this post.