Succeeding as a programming language

Succeeding as a programming language

In my last post I speculated that we can infer from history that a necessary precondition for toppling our existing programming language champions is that there should be a significant expansion in computing.

A number of reasons come to mind why this may be true. One is that such an expansion represents a big new market with huge potential pay-offs for companies that succeed in serving the market. This creates an appetite for investment and a greater tolerance for risk taking. It also creates demand for programmers and spreads them out. A programmer with a personal interest in a new programming language might now find himself/herself in a position of authority, exposed to many people with diverse non computing backgrounds, and saddled with the responsibility of quickly and cheaply coming up with software that address new kinds of problems.

In a more traditional setting, our protagonist will quickly be shouted down and told to stick with practices that are known to work (kind of). In the midst of a gold rush, however, anything goes and a programming language with something new to offer is likely to be tried and if it is worthy, word will spread like wild fire. In a few exceptional cases the language goes on to become the new champion. (It certainly also helps a lot if the language has a powerful sponsor.)

Of course, even in a gold rush most people do not take risks that they can avoid. It should also be true that the old champion languages should be significantly less suited for the new environment than the aspiring champions. Since existing languages can and do evolve (sometimes quite quickly), it should be asked why, in the past, champion languages were so slow to adapt to new environments.

I think the answer is that it the design space for language innovations is quite huge. Existing champions have to be very careful about how they evolve and they carry a lot of baggage that must be cared for by huge and expensive armies of porters. Aspiring champions are a dime a dozen and the vast majority of them will quickly die in utter obscurity. It is really hard to know which innovations are the right ones. By the time it is obvious what is to be done, an existing champion may already have become a former champion.

If you find all this plausible and you're a programming language enthusiast like I am, it is really nice to know that we are not perpetually condemned to put ever more layers of lipstick on the language pigs that are our current champions. It is also quite daunting to consider the really long odds against coming up with one or more of the innovations that will get us to a better place.

In searching for such innovations, it really helps to understand the problems of the new environment.

So, just what is the new environment? Sadly, I am less than sure that I know. While computing is still expanding at quite a clip, there is little evidence that we are in the midst of a gold rush like during the dot com boom.

Some trends stand out, though: Computing devices are becoming ever more ubiquitous. At some point, things may reach a critical mass, precipitating a gold rush. In such a world, computing will be massively distributed, network latencies will be significant, and security will be huge concern. On the one hand, these devices will generate huge amounts of data to process in high performance data centers. On the other hand, we'll be faced with computers that are tiny and slow, that need to resort to drastic measures to conserve battery life, that are not constantly connected and that may be impossible or at least very difficult to update.

The kinds of applications that are likely to be very important will not just be traditional IT, serving up web pages and searching them. Bioinformatics will be huge. And if Robotics really takes off, a lot of AI computing will come along for the ride.

All in all, computing will become much more diverse and much more challenging. Along the way, our current pain points will become unbearable burdens. The winners in this brave new world will be those companies that figure out how eliminate these pain points altogether.

So what are the current pain points? I can only speak from personal experience, but here is my list:

Programmers have far too much control over the execution of their programs. This may seem like a good thing, but it is a nightmare when you have to worry simultaneously about network and memory latency, energy consumption, race conditions, deadlocks and failures. It is far better if programmers need to specify only what is absolutely essential, and have automated tools figure out the rest.

Security is a nightmare. There are far too many ways to do things and far too few iron clad guarantees provided by the underlying platforms. The only way forward is to automate this to the point where the vast majority of programmers never need to think about security.

Complexity is overwhelming and getting worse at an increasing rate.

Versioning is still an unsolved problem.

It is still far too hard to reason about the correctness of our programs. And when we can't be sure what our programs do, the above problems are greatly magnified.

Perhaps my language enthusiasm is clouding my good sense, but I can see no way in which incremental changes to our current champions can eliminate these pain points.

Which is perhaps just another way of saying: the innovations that we need are not yet obvious.