If you look at stuff on this site you'll notice a trend: newer apps tend to be written in D programming language. There's a photo editing app, disk space visualizer and analyzer, a tool for automatic video processing, a tool for reading DirectShow graph files as text, all in D (and all free and open source btw). We also have many small internal tools written in D, even this very blog is powered by a hundred lines of D now. One of our main products, a video processing app called Video Enhancer, is currently written in C++ but the next major version is going to be in D for most parts except for the heavy number crunching bit where we need the extremes of performance only reachable by C++ compilers today. So obviously I must like D, and here are my personal reasons for doing it:

1. From infrared to ultraviolet. D covers very wide spectrum from "low-level" to "high-level". Some tasks require precise memory layout control and byte-level direct memory manipulations, and doing it without much overhead in terms of CPU instructions. This is where people usually turn to C. In high-level languages you either cannot do it at all (like in Idris) or (like in Haskell or Scala) you can attempt to but the code will look ugly and work slowly, you'll be fighting the language. So usually people there use FFI and delegate the low-level work to C. In D you don't need to rewrite parts of the program in C, you can write in D as in C, you can go even lower with its standardized inline assembly, you can tweak and generate low-level code using string mixins without resorting to external code generation tools and error-prone C macros.

On the other end are high-level languages with powerful abstractions and type-level wizardry like higher kinded types, higher kinded polymorphism, type families and associated types, type-level arithmetic and value-to-type propagation. D can do it too, some of it you could see in my previous post with some code abstracted over "any type constructor that happens to be an endofunctor". D's powerful templates can abstract over very different things including other templates, giving us higher-kinded polymorphism. Its static-if support and compile-time function evaluation fulfills the need for type-level arithmetic, value-to-type propagation and associated types.

It also shines in the visible light, i.e. between those extremes of infrared and ultraviolet, low and high levels. Many tasks do not require low-levelness of C/C++ and sophistication of Haskell and people don't want to pay with their time and efforts for those qualities. Payment for low-levelness of C and C++ is having to write a lot of boilerplate code, mental efforts of thinking about manual memory management, slow compilation times and cumbersome process of building. Payment for Haskell's sophisticated type system and control over effects is having to deal with monads and nontrivial ways to combine them (monads do not compose naturally). Compilation time is pretty bad in Haskell too, and very bad in Scala and Idris. So for these mid-level tasks people want something simple and convenient, like Python. Well, D can do it too: being simple and conveninent without paying with slow compilation and unwanted mental efforts, while retaining benefits of statically typed compiled language (type checking and fast execution), see my earlier post D as a scripting language for evidence. Apps listed in the beginning of this post, how simple and pleasant they were to write, serve as another evidence, I suppose.

2. D has a great story for compile-time function evaluation (most of the language is available for running at compile time, allowing things like building regexp automata during compilation or making parsers from EBNF definitions and immediately applying them during the same compilation phase) and on this stage types are very much like values, allowing to loop over them and branching on them. Here's some neat examples from my recent experiments with Robin Hood hashing. Consider this code:

alias types = TypeTuple!(bool, int, double, char, string,
                         TestStruct!false, TestStruct!true,
                         TestClass!false, TestClass!true);
foreach(t1; types)
    foreach(t2; types)
        testRB!(t1, t2, false)(num);

Here a generic function is called with 81 combinations of types. What other statically typed language (apart from dependently typed ones where types are values) can do this?

Another fragment from the same project:

alias MakerType(T) = T delegate();
void testHashesHisto(K, V, Hs...)(size_t num, staticMap!(MakerType, Hs) makers) {
...
    foreach(i, H; Hs) {
        auto h = makers[i]();
        measure("# " ~ H.stringof ~ ".make_histo", (){
        ...

Here a function takes an integer num and an arbitrary number of functions that produce values of types specified with the call. Type of each particular function from the makers tuple is formed by applying type-level function MakerType to corresponding type from the Hs type tuple. In the body of testHashesHisto there is a loop over the tuple of functions, on different iterations of this loop different functions from this tuple are called and return values of different types (!), and these values are used later together with names of their types. So on one iteration the h value can be an int, on another iteration it can be a string, on yet another - an object of some class and so on. This is, again, something you won't see in ordinary statically typed languages (apart from dependently typed ones). But D does it without dependent types and without resorting to dynamic types, all the types here are checked at compile time. And the nice thing is that I don't need to think about such long scary words as "metaprogramming", it's all just code, quite easy and natural to write in D.

3. Compile-time introspection. This is a killer feature of D. In a generic function (or class, template, etc.) you can not just blindly use type parameter but also analyse the passed type in all details: is it a class, a struct, a basic type or a function? What fields and methods does it have? What types do those fields have? What are their attributes (including user-defined ones)? What arguments do the passed function or methods of passed class have? Basically anything known to compiler at the call site (i.e. not where you define your generic but when someone uses it and passes some type or value as generic argument) you can know inside that generic thing you define, and you can define things differently or generate some code depending on this introspection information. This allows things like converting arbitrary classes into REST services in one line, simple and universal serialization to different formats, turning class/struct definitions to SQL queries or HTML pages etc. etc. etc. And all this completely at compile-time, no overhead for introspection at all!

I could probably go on but I better stop here. During my career I had a chance to use many different languages, so I can compare. Following are my short impressions from different languages, copied from my reddit answer with one little edit:

I've used C++ for many years (and still do, it's inevitable in some fields), the pain, the pain.
I've used Perl (it still serves my company web site) but it's really insane.
I've used Ruby and loved it at first (we all were young), but then got tired of slowness and dynamic typing.
I've used C# and it's ok for some fields but not really suitable for others. And CLR is Windows-only while Mono sucked everywhere.
I've used OCaml and loved it (my previous long-time language of choice for most tasks), but got tired of verbosity, bad Windows support and single-threadedness.
I've used Haskell a bit and it's great but sometimes too restricting and not really suitable for low-level stuff.
I've used Haxe and it's fine for its niche but quite limited in others.
I've used ATS and it's great (they made everything Rust makers dreamed of before it was cool) but too hard to use in practice. And Windows version required Cygwin too.
I've used Elm a bit but it's obviously a very niche thing.
I've used Idris and I'm using it right now for one project, it's the least mature language among them all, unfortunately, although one of the most interesting.
And I've used D. While it has its problems, it's still pretty close to the best offer.

tags: programming