My concerns about D programming language

I have announced my stepping away from most D upstream participation a while ago. At that time I have not provided any specifics about my reasons to do so and that was quite intentional - that wouldn't do anything but create another hot debate that helps no one.

Still, I know that more than few developers have been wondering about that and, most importantly, I feel like I owe some explanations to everyone who has been hearing D evangelism from me before.

Please note that this is purely my perception of the matter lacking any proofs or objectivity and as such it should be taken with appropriate skepticism. My goal is to explain, not to convince.

History

My interest in D programming language started in 2009 when I have read The Case for D article by Andrei Alexandrescu, language co-author. It has hit the right spot - by that time I got tired of infinite baggage of historical reasons in C++ and was looking for something that could replace it as my natively compiled language of choice. D looked promising and I was very impressed by what Andrei did with "Modern C++ Design" so it was worth trying at least.

It is hardly a secret that early adopter experience was rather terrible back in 2009. It took me about 20 minutes of "Hello World" style experiments to get my first compiler crash and reading standard library sources was pretty much the only way to learn how to use it. That has not discouraged me of course - I was not attracted by the language as it was in 2009 but by the promise of what it could become if some effort was put into it.

Naturally is has ended with contributing to both standard library and compiler on certain occasions because I am not the person who is going to wait until someone else fixes my problem or quickly resorts to workaround. It wasn't much but just enough to become more and more familiar with internal development process over time.

At some point I started working for Sociomantic Labs, one of companies using D in production and after that things got a bit more serious - there was a major push to move from maintaining own fork of D compiler (ancient version) to directly using up to date upstream toolchain, and I was responsible for much of upstream interaction on company behalf. And the more committed I was to make it happen, the more disappointed I have become with result-to-effort ratio.

What is wrong with D development?

Many users who decide to try using D programming language praise it for pragmatism and ease of use. There is a certain tendency to favor simple and practical solutions as opposed to theoretically sound ones that makes it simply feel "nice" from the start. I think a lot of that can be attributed to D programming language creator, Walter Bright, who values "get it done" hacker mentality a lot.

There is inherent dark side of that attitude though. Anything that does not immediately benefit some specific technical purpose tends to be dismissed as overly bureaucratic and formal. The history of D development is full of cases when it took huge amount of time to adopt something that would be considered a standard practice by that time (like version control, issue tracking or release maintenance) because someone had to "sell" it to Walter in a convincing manner.

It is not limited to Walter, of course. Language naturally attracts developers with hacker mentality and one often has a hard time selling to the community anything that imposes constraints on their creativity. It also matters a lot that neither of D programming language dictators (neither Andrei Alexandrescu nor Walter Bright) are not particularly interested in any kind of management activities preferring instead to focus on technical challenges.

As a result D programming language for many years remains consistently immature. Don't get me wrong, you can do great things with it, pushing both productivity and performance to the limit and getting immediate competitive advantage. I would even say that it is among best choices available for technological startup which will need to write major part of its software stack from scratch - it was a great success story for Sociomantic Labs in that sense.

But when it comes to be boring and predictable, D fails terribly. You can't assume that released features will become 100% usable (no bugs, sufficient docs and tests, well-defined integration with other language features) even few years after initial release. You can't assume that next compiler upgrade won't suddenly break your project or any of its transitive dependencies. You can't assume that any compiler version will be maintained for more than few months. You can't assume there is any control over how declared vision documents get executed in practice. You can't trust any promises from language authors because they don't keep any track of those. It is anarchy driven development in all its glory.

Something I have observed a lot over last years is that companies that get most positive experience of using D commercially are ones that are very independent in their tool stack - maintaining own compiler builds and/or own library sets without relying much on what happens with upstream. It is more of a building base for own tools than a product in that context. Other extremely successful niche is "fast scripting language" because maintenance doesn't matter much there and D is very productive.

Solutions?

Now the thing that makes me really sad is that all problems I talk about are not unsolvable. Many aren't even hard and have solutions tested and tried in other projects. Reason why they are not solved is because leadership doesn't view them as important problems worth focusing on and sometimes even actively resists external attempts to do so. You can easily get Walter to talk hours about memory safety or Andrei about optimizing one function in standard library. Ask them about something like following SemVer in compiler and "and you'll see Kryptonite at work" (c).

Also one perfectly reasonable point of view can also be that nothing really needs to be solved and all problems that frustrate me so hard are actually features and language can be successful among developers who fit such development methodology. Perfectly reasonable but sadly making it rather bad choice for both myself and job I have to do.

What I would love to happen is for both Walter and Andrei to focus on decision making and quality control exclusively while 100% of actual development would happen by other parties. That would require major time investment on their side of course to get required funds and/or developers and organize the process but in then end it will make a difference that no technical novelty will ever possibly do.

I don't really expect it to happen though.

What to do, what to do

For now I am going to put some more effort into studying Rust which has caught my attention last year but I am not terribly excited about it (will probably write about my Rust concerns too at some point later). Still, that will be a good way to refresh ones perspective on programming language landscape until something promising appears.

It is surprising how many programming languages exist out there and how disappointing most of them are if you try to stick to some standards :)