The Making of Epoch, Part 2 - WTF?

The Making of a Pragmatic Programming Language for the FuturePart 2: WTF is up with <insert thing> about Epoch?

Welcome to part 2 of the series on the Epoch language's creation and development. In this installment, I'd like to present a sort of FAQ, or more accurately, a list of "frequent WTFs" about Epoch. These are in no particular order and garnered largely from various bits of reaction and interaction I've seen relating to the project.

WhyTF would you ever do this?Typically, this response comes from people who are very comfortable with existing languages and tools. The question is usually framed from the perspective of "Language X (and maybe Y, Z, and Q) solves all the problems I face. Why do yet another new language?"

My response is twofold. First, if you're not in the situation where you feel daily pain using the languages you use, I envy you. I do not have that luxury. I am forced, for a number of reasons, to use C++ day in and day out. I loathe it. My desire here is partially to create something that alleviates much (but probably not all) of the pain I feel using a language like C++. A lot of this is articulated in the first installment of this series.

Secondly, I strongly feel that while C++ in particular is sufficient for now for doing what we need it to do, it will soon stop being sufficient at all. C++ is a language with a comparatively small standardized feature set (as far as libraries go) and this feature set does not adequately address things like parallel programming, distributed computation, GPGPUs, and so on. There are two options that I see here: continue to bolt vendor-specific extensions onto C++ until the heat death of the universe, or solve the actual problem for once and for all in a new language paradigm that's designed from the start to cope with sophisticated and ever-evolving computational models.

WhoTF are you?This is a totally fair and not entirely unexpected reaction of many people. The fact of the matter is, I am not a widely known languages expert (or really a languages expert at all). I am not a hugely reputable programmer, although that's not to say I am disreputable ;-) I am not a member of an existing language committee or implementation effort or anything of that nature. I've never committed code to a major compiler or language implementation project before.

Basically, I have zero credentials in this field, and I'm coming in making a lot of bold promises and claims, with little in the way of track record to prove that I can pull off this massive feat of engineering.

The skeptics are right to be cautious. It is not uncommon for someone to have big ideas about programming languages and fail to ever carry them out. The best answer I can provide to this question is, "I'm just this guy, you know?" In all seriousness, I'm just a programmer who feels the need for better tools. I don't see anyone else making the kinds of tools I really want (although frankly projects like Rust are getting closer to the mark) so I'm willing to make the commitment and put in the time to make those tools myself.

Time alone will tell if I can succeed, but if nothing else, I've found a very engaging hobby project and a fascinating venue for learning huge amounts of cool stuff about programming in general.

Five years in the making? WTF?This is another fair objection. Five years is a long time, and it only makes sense to look at the output of the Epoch effort over that span and find it a bit lacking. In all honesty, compared to efforts like Clang, Epoch's compiler implementation has been glacially slow, to be generous.

However, there's a simple reason for this: five years does not represent the actual amount of time I've invested in the project. If I were to cram all the hours I've spent into chunks of 40-hour work weeks, it'd be closer to five or six months of dedicated effort. For a lone programmer hacking without any external input or support, working around time constraints imposed by having a full-time career and lots of other interests, and often interrupted for months at a time by real life, I think I've done fairly well.

Epoch is moving forward, albeit slowly, and it may be easy to look at the project and conclude that it can never really go anywhere if it's taken five whole years to reach the stunted embryonic form it's in today. This is not an unreasonable conclusion by any means. The reality, though, is that I hope to be able to give Epoch an ever-increasing timeslice of my future, and I'm confident that things will accelerate as they gain critical mass.

Another language that tries to solve everyone's problems? WTF?This is the first criticism that I think just plain misses the mark. Never do I claim to solve "everyone's" problems, nor am I delusional enough to think that Epoch would be suitable for even a sizable majority of domains out there. The fact is we live in a software world where many different tools are necessary to accomplish the wide array of jobs that need to get done. Epoch will live in a small subset of that world, and I am in no way trying to advertise it as a panacea or cure-all solution to every programming issue ever.

I fully expect that Epoch will inevitably have its share of flaws, drawbacks, weaknesses, and areas where it just isn't appropriate. Granted, I will do my best to minimize those issues, but they will exist nonetheless. The important thing isn't that "everyone" uses Epoch - the important thing is that Epoch be available for anyone who could potentially benefit from its way of thinking.

It is true that Epoch has ambitious goals, and aims to solve a lot of outstanding issues in the way we write code today. However, this is far from staking a claim to universality. Just because I want Epoch to accomplish a lot does not mean I'm foolish enough to believe it will be the ultimate solution to everything.

A complete reinvention of <linguistic wheel that already exists>? WTF?This is a bit more subtle. While it may superficially appear that Epoch is just another attempt at retreading well-covered ground, this is not the case. I have specific desires for the language that differ greatly from any other language effort on the planet (at least that I know of). As I mentioned before, Rust comes close in a lot of areas, but compromises in a few ways that I think are not necessary.

I'll go into more detail about this later in the series, but for now I'd like to emphasize that Epoch is a unique ideal. It represents the confluence of many different concepts, and in my mind, is something of a playground or sandbox for experimenting with all kinds of ways to improve real-world programming as effectively as possible.

Until such time as I've elaborated on my mental image of what the language should be, it is entirely fair to conclude that it's just another bad rip-off of stuff that's been done (or is being done) already. But hopefully as I spell out the full scope of what I want the language to be like, it will become apparent that nobody else is really doing quite what Epoch aims to achieve.

It took you this long to (badly) rediscover what compiler writers have been doing for decades? WTF?A while ago I circulated a fairly popular article on how I managed to improve compiler performance for Epoch by a factor of over 1000. One of the most common reactions to this paper could be paraphrased as "dear God, this noob knows nothing about basic compiler design."

This is a fair point. I don't. I dropped out of high school and never took a single proper, formal CS class. I never learned all the great stuff that compiler implementers discovered ages ago, at least not in an institutional setting.

However, I didn't approach the Epoch project in pure ignorance. I chose the initial implementation strategy for a number of pragmatic reasons, which I hope to elaborate on later in the series. It does make for a rather awkward-looking start, and I'll be the first to admit that the language implementation has huge room for improvement.

The simple reality, though, is that Epoch has as yet not attracted the attention of anyone who does have the credentials and experience to radically improve on what foundations I've managed to establish. I would dearly love to get some real compiler hackers working on the project and helping to shore up the basis of the language, but that has so far failed to happen. Until such time as those people arrive on the Epoch scene, we'll have to make do with what I personally can manage.

The longer and more thoughtful response is that there are several ways in which this effort can be beneficial. In the absolute worst case, I learn a lot for myself and move on, more or less defeated in the Epoch campaign but a much better programmer for having waged the battle.

Another possible outcome is that I discover that someone else can do a better job of making the language I want to program in. This is fine with me; I'd just as soon have that language today and be able to play with it. I don't really care about being the one to make this language a reality. I'm just willing to do the work if nobody else is.

What I see as the most likely outcome is that Epoch will make interesting inroads into certain areas of programming language design, and then inspire more concerted and expert efforts by others. Whether or not I remain involved in programming language creation past this point is mostly irrelevant. Again, I just want to have the nifty tools; the only reason I'm doing it myself is because nobody else seems to be making the exact tool that I want.

And of course in my happy fantasy land, Epoch does succeed wildly, and manages to capture a large audience by virtue of how awesome it is.

We can all dream.

A virtual machine for a runtime? WTF?The VM is a bootstrapping effort. It exists solely so that I can make Epoch programs that run on my computer today. If I had to write a native machine code back-end, this whole thing would be going even slower than it already is.

Epoch by design and by fiat is intended to be execution-model agnostic. The VM is not a permanent part of the implementation; indeed, someday I'd like to retarget to LLVM or even direct native code generation. But for now, running against a VM is a useful shortcut for getting the thing into working shape.

Your <implementation/tools/installer/documentation/etc.> really sucks. WTF?Again, keep in mind that this is a volunteer effort by just myself and that it takes a backseat to a lot of other real life concerns for me. For various reasons (some of which I've discussed in my more personal journal entries in the past) it is very important for me to manage my stress load. Working a demanding full-time job and trying to create a programming language on the side is a recipe for immense stresses. To mitigate this, and to ensure I don't end up quite literally hospitalized, I have to resign myself to not being as productive on Epoch as I would otherwise like.

As a direct result, a lot of things just don't get polish time. The implementation is crufty in many areas and even with the ongoing compiler redesign for Release 12 there's some icky corners of the code base. The tools are barely more than half-cooked prototypes under the most forgiving of light; frankly they're not even really tools yet. The installation experience is pretty lame and the documentation is woefully inadequate.

These are all things I wish I could change, but at this point in the project, just getting the language implemented and specified is top priority. It is more important to get things working and establish the features and scope of the language than to make sure every tiny little bit of the ecosystem is perfect. There will be plenty of time to polish things once they have taken shape and stabilized a bit.

You don't maintain branches of old releases? WTF?This is a deliberate decision. I don't want to spend time back-porting fixes to old releases. I don't want to have to worry about regression testing. I want to have the freedom to work on the language and keep it malleable - even fluid - as long as possible. I promise that as soon as Epoch starts to gain traction - and the construction dust has settled a bit - I'll start treating it like a more typical product with real release cycles and back maintenance and so on.

You didn't answer <my pet question>? WTF?I am, alas, not psychic, and also rather forgetful to boot. If there's anything else you'd like to know more about, drop off a comment, and I'll be happy to try and expound on it.