Research in Programming Languages

Is there still research to be done in Programming Languages? This essay touches both on the topic of programming languages and on the nature of research work. I am mostly concerned in analyzing this question in the context of Academia, i.e. within the expectations of academic programs and research funding agencies that support research work in the STEM disciplines (Science, Technology, Engineering, and Mathematics). This is not the only possible perspective, but it is the one I am taking here.

PLs are dear to my heart, and a considerable chunk of my career was made in that area. As a designer, there is something fundamentally interesting in designing a language of any kind. It’s even more interesting and gratifying when people actually start exercising those languages to create non-trivial software systems. As a user, I love to use programming languages that I haven’t used before, even when the languages in question make me curse every other line.

But the truth of the matter is that ever since I finished my Ph.D. in the late 90s, and especially since I joined the ranks of Academia, I have been having a hard time convincing myself that research in PLs is a worthy endeavor. I feel really bad about my rational arguments against it, though. Hence this essay. Perhaps by the time I am done with it I will have come to terms with this dilemma.

Back in the 50s, 60s and 70s, programming languages were a BigDeal, with large investments, upfront planning, and big drama on standardization committees (Ada was the epitome of that model). Things have changed dramatically during the 80s. Since the 90s, a considerable percentage of new languages that ended up being very popular were designed by lone programmers, some of them kids with no research inclination, some as a side hobby, and without any grand goal other than either making some routine activities easier or for plain hacking fun. Examples:

PHP, by Rasmus Lerdorf circa 1994, “originally used for tracking visits to his online resume, he named the suite of scripts ‘Personal Home Page Tools,’ more frequently referenced as ‘PHP Tools.’ ” [1] PHP is a marvel of how a horrible language can become the foundation of large numbers of applications… for a second time! Worse is Better redux. According one informal but interesting survey, PHP is now the 4th most popular programming language out there, losing only to C, Java and C++.

JavaScript, by Brendan Eich circa 1995, “Plus, I had to be done in ten days or something worse than JS would have happened.” [2] According to that same survey, JavaScript is the 5th most popular language, and I suspect it is climbing up that rank really fast. It may be #1 by now.

Python, by Guido van Rossum circa 1990, “I was looking for a ‘hobby’ programming project that would keep me occupied during the week around Christmas.” [3] Python comes at #6, and its strong adoption by scientific computing communities is well know.

Ruby, by Yukihiro “Matz” Matsumoto circa 1994, “I wanted a scripting language that was more powerful than Perl, and more object-oriented than Python. That’s why I decided to design my own language.” [4] At #10 in that survey.

Compare this mindset with the context in which the the older well-known programming languages emerged:

Fortran, 50s, originally developed by IBM as part of their core business in computing machines.

Cobol, late 50s, designed by a large committee from the onset, sponsored by the DoD.

Lisp, late 50s, main project occupying 2 professors at MIT and their students, with the grand goal of producing an algebraic list processing language for artificial intelligence work, also funded by the DoD.

C, early 70s, part of the large investment that Bell Labs was doing in the development of Unix.

Smalltalk, early 70s, part of a large investment that Xerox did in “inventing the future” of computers.

Back then, developing a language processor was, indeed, a very big deal. Computers were slow, didn’t have a lot of memory, the language processors had to be written in low-level assembly languages… it wasn’t something someone would do in their rooms as a hobby, to put it mildly. Since the 90s, however, with the emergence of PCs and of decent low-level languages like C, developing a language processor is no longer a BigDeal. Hence, languages like PHP and JavaScript.

There is a lot of fun in designing new languages, but this fun is not an exclusive right of researchers with, or working towards, Ph.Ds. Given all the knowledge about programming languages these days, anyone can do it. And many do. And here’s the first itchy point: there appears to be no correlation between the success of a programming language and its emergence in the form of someone’s doctoral or post-doctoral work. This bothers me a lot, as an academic. It appears that deep thoughts, consistency, rigor and all other things we value as scientists aren’t that important for mass adoption of programming languages. But then again, I’m not the first to say it. It’s just that this phenomenon is hard to digest, and if you really grasp it, it has tremendous consequences. If people (the potential users) don’t care about conceptual consistency, why do we keep on trying to achieve that?

To be fair, some of those languages designed in the 90s as side projects, as they became important, eventually became more rigorous and consistent, and attracted a fair amount of academic attention and industry investment. For example, the Netscape JavaScript hacks quickly fell on Guy Steele’s lap resulting in the ECMAScript specification. Python was never a hack even if it started as a Christmas hobby. Ruby is a fun language and quite elegant from the beginning. PHP… well… it’s fun for possibly the wrong reasons. But the core of the matter is that “the right thing” was not the goal. It seems that a reliable implementation of a language that addresses an important practical need is the key for the popularity of a programming language. But being opportunistic isn’t what research is supposed to be about… (or is it?)

Also to be fair, not all languages designed in the 90s and later started as side projects. For example, Java was a relatively large investment by Sun Microsystems. So was .NET later by Microsoft.

And, finally, all of these new languages, even when created over a week as someone’s pet project, sit on the shoulders of all things that existed before. This leads me to the second itch: one striking commonality in all modern programming languages, especially the popular ones, is how little innovation there is in them! Without exception, including the languages developed in research groups, they all feel like mashups of concepts that already existed in programming languages in 1979, wrapped up in their own idiosyncratic syntax. (I lied: exceptions go to aspects and monads both of which came in the 90s)

So one pertinent question is: given that not much seems to have emerged since 1979 (that’s 30+ years!), is there still anything to innovate in programming languages? Or have we reached the asymptotic plateau of innovation in this area?

I need to make an important detour here on the nature of research.

<Begin Detour>

Perhaps I’m completely off; perhaps producing innovative new softwareis not a goal of [STEM] research. Under this approach, any software work is dismissed from STEM pursuits, unless it is necessary for some specific goal — like if you want to study some far-off galaxy and you need an IT infrastructure to collect the data and make simulations (S for Science); or if you need some glue code for piecing existing systems together (T for Technology); or if you need to improve the performance of something that already exists (E for Engineering); or if you are a working on some Mathematical model of computation and want to make your ideas come to life in the form of a language (M for Mathematics). This is an extreme submissive view of software systems, one that places software in the back sit of STEM and that denies the existence of value in research in/by software itself. If we want to lead something on our own, let’s just… do empirical studies of technology or become biologists/physicists/chemists/mathematicians or make existing things perform better or do theoretical/statistical models of universes that already exist or that are created by others. Right?

I confess I have a dysfunctional relationship with this idea. Personally, I can’t be happy without creating software things, but I have been able to make my scientist-self function both as a cold-minded analyst and, at times, as an expert passenger in someone else’s research project. The design work, for me, has moved to sabbatical time, evenings and weekends; I don’t publish it [much] other than the code itself and some informal descriptions. And yet, I loathe this situation.

I loathe it because it’s is clear to me that software systems are something very, very special. Software revolutionized everything in unexpected ways, including the methods and practices that our esteemed colleagues in the “hard” sciences hold near and dear for a very long time. The evolution of information technology in the past 60 years has been _way_ off from what our colleagues thought they needed. Over and over again, software systems have been created that weren’t part of any scientific project, as such, and that ended up playing a central role in Science. Instead of trying to mimic our colleagues’ traditional practices, “computer scientists” ought to be showing the way to a new kind of science — maybe that new kind of science or that one or maybe something else. I dare to suggest that the something else is related to the design of things that have software in them. It should not be called Science. It is a bit like Engineering, but it’s not it either because we’re not dealing [just] with physical things. Technology doesn’t cut it either. It needs a new name, something that denotes “the design of things with software in them.” I will call it Design for short, even though that word is so abused that it has lost its meaning.

<Suspend Detour>

Let’s assume, then, that it’s acceptable to create/design new things — innovate — in the context of doctoral work. Now comes the real hard question.

If anyone — researchers, engineers, talented kids, summer interns — can design and implement programming languages, what are the actual hard goals that doctoral research work in programming languages seeks that distinguishes it from what anyone can do?

Let me attempt to answer these questions, first, with some well-known goals of language design:

Performance — one can always have more of this; certain application domains need it more than others. This usually involves having to come up with interesting data structures and algorithms for the implementation of PLs that weren’t easy to devise.

Human Productivity — one can always want more of this. There is no ending to trying to make development activities easier/faster.

Verifiability — in some domains this is important.

There are other goals, but they are second-order. For example, languages may also need to catch up with innovations in hardware design — multi-core comes to mind. This is a second-order goal, the real goal behind it is to increase performance by taking advantage of potentially higher-performing hardware architectures.

In other words, someone wanting to do doctoral research work in programming languages ought to have one or more of these goals in mind, and — very important — ought to be ready to demonstrate how his/her ideas meet those goals. If you tell me that your language makes something run faster, consume less energy, makes some task easier or results in programs with less bugs, the scientist in me demands that you show me the data that supports such claims.

A lot of research activity in programming languages falls under the performance goal, the Engineering side of things. I think everyone in our field understands what this entails, and is able to differentiate good work from bad work under that goal. But a considerable amount of research activities in programming languages invoke the human productivity argument; entire sub-fields have emerged focusing on the engineering of languages that are believed to increase human productivity. So I’m going to focus on the human productivity goal. The human productivity argument touches on the core of what attracts most of us to creating things: having a direct positive effect on other people. It has been carelessly invoked since the beginning of Computer Science. (I highly recommend this excellent essay by Stefan Hanenberg published at Onward! 2010 with a critique of software science’s neglect of human factors)

Unfortunately, this argument is the hardest to defend. In fact, I am yet to see the first study that convincingly demonstrates that a programming language, or a certain feature of programming languages, makes software development a more productive process. If you know of such study, please point me to it. I have seen many observational studies and controlled experiments that try to do it [5, 6, 7, 8, 9, 10, among many]. I think those studies are really important, there ought to be more of them, but they are always very difficult to do [well]. Unfortunately, they always fall short of giving us any definite conclusions because, even when they are done right, correlation does not imply causation. Hence the never-ending ping-pong between studies that focus on the same thing and seem to reach opposite conclusions, best known in the health sciences. We are starting to see that ping-pong in software science too, for example 7 vs 9. But at least these studies show some correlations, or lack thereof, given specific experimental conditions, and they open the healthy discussion about what conditions should be used in order to get meaningful results.

I have seen even more research and informal articles about programming languages that claim benefits to human productivity without providing any evidence for it whatsoever, other than the authors’ or the community’s intuition, at best based on rational deductions from abstract beliefs that have never been empirically verified. Here is one that surprised me because I have the highest respect for the academic soundness of Haskell. Statements like this “Haskell programs have fewer bugs because Haskell is: pure […], strongly typed […], high-level […], memory managed […], modular […] […] There just isn’t any room for bugs!” are nothing but wishful thinking. Without the data to support this claim, this statement is deceptive; while it can be made informally in a blog post designed to evangelize the crowd, it definitely should not be made in the context of doctoral work unless that work provides solid evidence for such a strong statement.

That article is not an outlier. The Internets are full of articles claiming improved software development productivity for just about every other language. No evidence is ever provided, the argumentation is always either (a) deducted from principles that are supposed to be true but that have never been verified, or (b) extrapolated from ad-hoc, highly biased, severely skewed personal experiences.

This is the main reason why I stopped doing research in Programming Languages in any official capacity. Back when I was one of the main evangelists for AOP I realized at some point that I had crossed the line to saying things for which I had very little evidence. I was simply… evangelizing, i.e. convincing others of an idea that I believed strongly. At some point I felt I needed empirical evidence for what I was saying. But providing evidence for the human productivity argument is damn hard! My scientist self cannot lead doctoral students into that trap, a trap that I know too well.

Moreover, designing and executing the experiments that lead to uncovering such evidence requires a lot of time and a whole other set of skills that have absolutely nothing to do with the time and skills for actually designing programming languages. We need to learn the methods that experimental psychologists use. And, in the end of all that work, we will be lucky if we unveil correlations but we will not be able to draw any definite conclusions, which is… depressing.

But without empirical evidence of any kind, and from a scientific perspective, unsubstantiated claims pertaining to, say, Haskell or AspectJ (which are mostly developed and used by academics and have been the topic of many PhD dissertations) are as good as unsubstantiated claims pertaining to, say, PHP (which is mostly developed and used by non-academics). The PHP community is actually very honest when it comes to stating the benefits of using the language. For example, here is an honest-to-god set of reasons for using PHP. Notice that there are no claims whatsoever about PHP leading to less bugs or higher programmer productivity (as if anyone would dare to state that!); they’re just pragmatic reasons. (Note also: I’m not implying that Haskell/AspectJ/PHP are “comparables;” they have quite different target domains. I’m just comparing the narratives surrounding those languages, the “stories” that the communities tell within themselves and to others)

OK, now that I made 823 enemies by pointing out that the claims about human productivity surrounding languages that have emerged in academic communities — and therefore ought to know better — are unsubstantiated, PLUS 865 enemies by saying that empirical user studies are inconclusive and depressing… let me try to turn my argument around.

Is the high bar of scientific evidence killing innovation in programming languages? Is this what’s causing the asymptotic behavior? It certainly is what’s keeping me away from that topic, but I’m just a grain of sand. What about the work of many who propose intriguing new design ideas that are then shot down in peer-review committees because of the lack of evidence?

This ties back to my detour on the nature of research.

<Join Detour> Design experimentation vs. Scientific evidence

So, we’re back to whether design innovation per se is an admissible first-order goal of doctoral work or not. And now that question is joined by a counterpart: is the provision of scientific evidence really required for doctoral work in programming languages?

If what we have in hand is not Science, we need to be careful not to blindly adopt methods that work well for Science, because that may kill the essence of our discipline. In my view, that essence has been the radical, fast-paced, off the mark design experimentation enabled by software. This rush is fairly incompatible with the need to provide scientific evidence for the design “hopes.”

I’ll try a parallel: drug design, the modern-day equivalent of alchemy. In terms of research it is similar to software: partly based on rigor, partly on intuitions, and now also on automated tools that simply perform an enormous amount of logical combinations of molecules and determine some objective function. When it comes to deployment, whoever is driving that work better put in place a plan for actually testing the theoretical expectations in the context of actual people. Does the drug really do what it is supposed to do without any harmful side effects? We require scientific evidence for the claimed value of experimental drugs. Should we require scientific evidence for the value of experimental software?

The parallel diverges significantly with respect to the consequences of failure. A failure in drug design experimentation may lead to people dying or getting even more sick. A failure in software design experimentation is only a big deal if the experiment had a huge investment from the beginning and/or pertains to safety-critical systems. There are still some projects like that, and for those, seeking solid evidence of their benefits before deploying the production version of the experiment is a good thing. But not all software systems are like that. Therefore the burden of scientific evidence may be too much to bear. It is also often the case that over time, the enormous amount of testing by real use is enough to provide assurances of all kinds.

One good example of design experimentation being at odds with scientific evidence is the proposal that Tim Berners-Lee made to CERN regarding the implementation of the hypertext system that became the Web. Nowhere in that proposal do we find a plan for verification of claims. That’s just a solid good proposal for an intriguing “linked information system.” I can imagine TB-L’s manager thinking: “hmm, ok, this is intriguing, he’s a smart guy, he’s not asking that many resources, let’s have him do it and see what comes of it. If nothing comes of it, no big deal.” Had TB-L have to devise a scientific or engineering assessment plan for that system beyond “in the second phase, we’ll install it on many machines” maybe the world would be very different today, because he might have gotten caught in the black hole of trying to find quantifiable evidence for something that didn’t need that kind of validation.

Granted, this was not a doctoral topic proposal; it was a proposal for the design and implementation of a very concrete system with software in it, one that (1) clearly identified the problem, (2) built on previous ideas, including the author’s own experience, (3) had some intriguing insights in it, (4) stated expected benefits and potential applications — down to the prediction of search engines and graph-based data analysis. Should a proposal like TB-L’s be rejected if it were to be a doctoral topic proposal? When is an unproven design idea doctoral material and other isn’t? If we are to accept design ideas without validation plans as doctoral material, how do we assess them?

Towards the discipline of Design

In order to do experimental design research AND be scientifically honest at the same time, one needs to let go of claims altogether. In that dreadful part of a topic proposal where the committee asks the student “what are your claims?” the student should probably answer “none of interest.” In experimental design research, one can have hopes or expectations about the effects of the system, and those must be clearly articulated, but very few certainties will likely come out of such type of work. And that’s ok! It’s very important to be honest. For example, it’s not ok to claim “my language produces bug-free programs” and then defend this with a deductive argument based on unproven assumptions; but it’s ok to state “I expect that my language produces programs with fewer bugs [but I don’t have data to prove it].” TB-L’s proposal was really good at being honest.

Finally, here is an attempt at establishing a rigorous criteria for design assessment in the context of doctoral and post-doctoral research:

Problem: how important and surprising is the problem and how good is its description? The problem space is, perhaps, the most important component for a piece of design research work. If the design is not well grounded in an interesting and important problem, then perhaps it’s not worth pursuing as research work. If it’s a old hard problem, it should be formulated in a surprising manner. Very often, the novelty of a design lies not in the design itself but in its designer seeing the problem differently. So — surprise me with the problem. Show me insights on the nature of the problem that we don’t already know.

Potential: what intriguing possibilities are unveiled by the design? Good design research work should open up doors for new avenues of exploration.

Feasibility: good design research work should be grounded on what is possible to do. The ideas should be demonstrated in the form of a working system.

Additionally, design research work, like any other research work, needs to be placed in a solid context of what already exists.

This criteria has two consequences that I really like: first, it substantiates our intuitions about proposals such as TB-L’s “linked information system” being a fine piece of [design] research work; second, it substantiates our intuitions on the difference of languages like Haskell vs. languages like PHP. I leave that as an exercise to the reader!

Coming to terms

I would love to bring design back to my daytime activities. I would love to let my students engage in designing new things such as new programming languages and environments — I have lots of ideas for what I would like to do in that area! I believe there is a path to establishing a set of rigorous criteria regarding the assessment of design that is different from scientific/quantitative validation. All this, however, doesn’t depend on me alone. If my students’ papers are going to be shot down in program committees because of the lack of validation, then my wish is a curse for them. If my grant proposals are going to be rejected because they have no validation plan other than “and then we install it in many machines” or “and then we make the software open source and free of charge” then my wish is a curse for me. We need buy-in from a much larger community — in a way, reverse the trend of placing software research under the auspices of science and engineering [alone].

This, however, should only be done after the community understands what science and scientific methods are all about (the engineering ones — everyone knows about them). At this point there is still a severe lack of understanding of science within the CS community. Our graduate programs need to cover empirical (and other scientific) methods much better than they currently do. If we simply continue to ignore the workings of science and the burden of scientific proof, we end up continuing to make careless religious statements about our programming languages and systems that simply will lead nowhere, under the misguided impression that we are scientists because the name says so.

104 Responses to Research in Programming Languages

“we need to be careful not to blindly adopt methods that
work well for Science, because that may kill the essence
of our discipline”

Indeed! One of my favorite quotes is Einstein’s

“Not everything that can be counted counts,
and not everything that counts can be counted”

I think this is especially important to bear in mind when considering PL/Design.

(And yes, program committees are terrible at evaluating language designs — partly
because they are very difficult to evaluate! — I have many anecdotes about this, all
quite objective IMO as none of them are about my own papers, but thats another story…)

That said, I believe the situation is not as dire as you indicate. As you point out:

1. “…one striking commonality in all modern programming languages, especially the
popular ones, is how little innovation there is in them!”

2. “…there appears to be no correlation between the success of a programming language
and its emergence in the form of someone’s doctoral or post-doctoral work.”

The explanation for this is *not* that all the interesting/innovative PL work was done
in the 70s, and that the work now is simply too technical etc. Instead, the explanation is
that it *takes decades* to really figure out what the truly useful, valuable and
implementable designs/features are, and the best way to integrate them with
“mainstream” languages. In other words, the “tech transfer” process for language
design has proven to be decades long…

So yes, there is little correlation with programming *languages*, but I think the
picture is quite different if you look at *features* not entire languages.

Recall that it took GC nearly 50 years to go mainstream! As you point out, there are
few definitive “scientific” studies about why even GC is better. Once you move to
things like static typing, I think it becomes an impossible endeavor, for the reasons
Einstein pithily describes. Thus, instead of nifty scientific charts, what we have are
series of anecdotes and narratives built up over a long span (maybe decades), at
which point the features inch into the mainstream.

Every time someone says “there has been no new innovative work in languages” since
so-and-so’s (Turing award) winning work in the 60s, 70s, one simply has to point
to the explosion of cleverness in Haskell, much of which has already
“gone mainstream”. The single most astonishing and influential feature is
probably Typeclasses — see Simon Peyton-Jones’ graph
(http://research.microsoft.com/en-us/um/people/simonpj/papers/haskell-retrospective/HaskellRetrospective.pdf)
which enabled a slew of other things (monads, FRP, generic programming,
quickcheck etc.) I doubt any of this could be anticipated when typeclasses came out,
and indeed there are, to my amazement, those who still question the value of this
feature. Nevertheless, two-and-a-half decades on, these are all features
(local type inference, lambdas, LINQ,…) seeping into mainstream languages
C# has been particularly progressive in this regard. Other recent examples
that pop to mind are the early academic work on SELF/OO (designs + optimizations)
which are now the basis for many of the performance improvements for JS.
And of course, there are the many innovations around Scala and F# which
are greatly informed by deep technical ideas that came out of the academic
community.

(TL;DR)
So, the upshot is that yes, a lot of work in PL (and SE) is indeed design, which
is difficult if not impossible to evaluate using the usual scientific method.
We should be looking for better ways to evaluate them, and not nipping ideas
in the bud before there is bullet proof “evidence” of merit. Nevertheless, all is not lost.
There does appear to be a (rather lengthy) social process, where thanks to
anecdotal narratives there is a steady stream of academic ideas that eventually
seriously influencing mainstream languages. We should be thinking of ways to
shorten this process, and in the meantime, be patient

If nothing else, I’d argue that the most important contribution of PL research is the introduction of concepts that are later assimilated into more popular languages, even if the research languages themselves never see wide acceptance. Python owes its list comprehensions to Haskell (similarly with Rust’s typeclasses), and I’d like to think that AspectJ had a non-negligible influence on Python’s decorators. For a more extreme example, consider the fact (the fact!) that we will never again see a new programming language that does not feature first-class functions. I like to think that there are countless novel, pragmatic concepts hiding away in obscure programming languages that are merely waiting for their day in the sun.

Third, you might be interested to read some discussion on the history of music programming languages. Max/MSP and Pd, both dataflow-oriented, offer what I would estimate to be an order of magnitude of productivity gain for certain tasks in building one-off multi-media systems. They’re a bit like a UNIX for real-time multi-media + control signals. This essay reminded me a bit of the anti-academic and organic approach that Miller Puckette took in building them despite being trained as a mathematician and developing them in an academic setting. This serves as a good lesson that successful software isn’t necessarily designed by having good principles, but rather the proper environment, namely, one with energy and a need.

Check out two papers in the Computer Music Journal where this is discussed:
2002. Miller Puckette, “Max at Seventeen”. Computer Music Journal, 26(4)
2002. Eric Lyon, “Dartmouth Symposium on the Future of Computer Music Software: A Panel Discussion”. Computer Music Journal, 26(4)

Generally, computer music is one of the more interesting fields to look at if you’re interested in ascertaining the future of HCI, computer science and psychological research since from the beginning they have not been accorded the luxury of forgoing certain constraints, such as that everything must happen in real-time, data must be of a certain resolution (in time and ‘space’) and that non-tech-savvy practitioners from other fields (musicians) must be able to use the tools as experts.

Oh, and I would add that if you are not familiar with Bill Buxton’s career, it may prove interesting reading for you. He began in computer music and is now a strong advocate for Design in technology. One insight that he often emphasizes, which I don’t claim is his originally, is that new technologies take 20-30 years to be adopted. According to this view, new ideas in software design should expect to lie dormant for at least 20 years, echoing what @Ben wrote above.

I fully agree with your viewpoint re. human productivity. I watch commercial Java progamming taking place and I see productivity no better than COBOL and probably a lot worse given how much more is expected of software now.

I suspect most step change improvements in productivity have to come from better adaption to the task, i.e. from languages that are to a greater or lesser degree domain-specific.

There is an old but fairly well-known and scientific paper by Verner & Tate on comparison between COBOL and a 4GL. The 4GL was ~5x more productive. Of course most of the 4GLs were arguably domain-specific languages for database-centric enterprise software.

The paper is available via IEEE but I don’t find any public copy. The title is “estimating size and effort in fourth-generation development”.

If you have Bob Glass’s book Software Conflict it’s highlighted on p.98, something I was amused to find recently as I used to work with and on the 4GL technology in question 20 years ago

Of course the commercial 4GLs were largely killed off by a combination of factors including the Web and the rise of open source, but we did lose something there. That ~5x productivity is eyecatching but it’s consistent with my personal experience with the technology. Those who do not study history, etc.

@Ben.
– comprehensions didn’t originate in haskell (probably not even from Miranda, as they were available in smalltalk, and probably that was taken from somewhere else)
– aspect oriented programming didn’t start with aspectJ. I think Kiczales started his experiments using common lisp. at least the early papers on aop used that.

There will be a day when the programming universe accepts the fact that LISP is by far the best programming language in the world, a language that can actually “think” and make decisions, logical decisions.

The shortest path algorithm can be written in LISP in a few lines – I challenge any programmer out there to do it in a few pages in C, without using any dependencies.

LISP has been underestimated for quite so long, and it’s nice to see that someone (like you) acknowledges the contributions that LISP has on the programing ecosystem.

Very interesting. I’m in the industry developing software. I’m not sure what to think about the 3 goals you state: for performance, we have grids, clusters, GPUs, it seems there is more and more hardware so that even if the language itself is not “fast”, the grid will compensate (as an aside, I’m seeing horrible uses of grids from a resource point of view: people don’t care about writing efficient software, because they know they can have 1,000 more nodes on the grid anyway…) ; for productivity, what I’m seeing everyday is that: either the problem is (partially) solved by the use of libraries, or what’s really getting you is the environment, such mundane things as repositories, build systems, deployments (just a matter of organization, more than science, I guess)… and testing – I think each day of coding results in 2 or 3 days of testing, that testing being a kind of “proof” that the system is not going to… crash and make you lost money. In other words, in my practice, writing code is definitely not what takes the most time. Reducing the amount of testing we have to do, or reducing the possibilities of bugs would in the end be the most useful (to me), and I’m constantly looking for automated proving tools, but those are not forthcoming for C++ or Java.

I would like to echo Frank’s comment. I work in a 50 person team all working on the same codebase. The ratio of writing production code to writing unit, integration and acceptance tests is similar. There is nothing in C# that assists us perform this testing.
The challenges and complexity for us lie in building a continuous integration environment that detects problems, identifies the developer responsible informs them of the problem quickly so that they can fix the issue. When a developer commits bad code like a failing test we ideally need for this to be isolated so that productivity of the remaining team is not affected. This last requirement is proving particularly tricky.

I was doing my phd in PL and felt the same way about AOP. I’m relieved someone involved with it ginally said agreed. Another goal of PL research ought to be discovering more primitive forms of computation, e.g. continuations, closures, type theories. Concurrency is still a mess and could use some innovation. Also distributed programming, reliable systems, and module systems. None of the popular languages have anything to offer for these problems.

I have been using and advocating literate software for a decade.
I claim that it improves software due to three effects:
1) the developer has to explain the code and, as a side-effect, discovered
corner cases, missed cases, bad design, etc. before submission to review.
2) the team reviewer have text that explains the design decisions and the
rationale behind the code. they are able to critique the design as well as
the code. They will better understand the code which leads to better
review which leads to higher quality.
3) the code “lives” because it is embedding in human-to-human
communication. There are over 100,000 dead piles of code on Sourceforge
because the author left and nobody has a clue about how to maintain and
modify the code.

I have been trying to find a researcher at a University interested in
creating studies to confirm or deny the above assertions which are
based only on my experience.

Claim 2, for instance, could be tested by taking previously published
software (e.g. cryptographic software). Give one group the book
“Implementing Cryptographic Software” which contains the actual
source code. Give a second group just the source code from the book.
Have a group review and post-review test. See which group has a
better understanding of the code, e.g. why some constant has the
value it does.

We need studies like this to “put some science” behind the opinions.
Literate programming is a fundamentally important technology but
nobody will touch it unless we do the studies.

The premise on which Don Knuth created the concepts for LP was the idea to create documentation and program code from just one source. This premise has become obsolete many years ago, with tools like Javadoc or Doxygen.

These latter tools also come with a big advantage, as they don’t require a pre-source code version of your programs, and therefore can directly interoperate with any developer tool of your choice. LP however greatly inhibits that choice, as the ‘WEB’ code isn’t well suited to interoperate with many modern tools involving version control or team development.

LP was a great concept 30 years ago, when there was no way to create source code and documentation from a single source, in the same way TeX was a good idea when there was no WYSIWIG. These times are past now.

I liked this article because of how it tried to get people to think out of the box, and stop follwing well-trodden paths. I don’t think going down a 30 year old path will lead to any new insights.

Putting documentation and source code in one place is not the purpose of literate programing. The main merit of literate programming is to rearrange the code in what best for human mind to follow.

TeX is still a good idea now. The current WYSIWIG sucks. What I like to see is an editor that build for dual monitors where you edit TeX on one screen and have the rendered output on the other display. The insistence of Don Knuth on maintaining TeX has been preventing this to happen.

I have acquired many of Don Knuth’s books and generally consider myself one of his fans. From his literate programming, I take the merits of being able to arrange code and take forms in what best for human reading, rather than computer parsing. And often, what is best for human reading is what best for human writing.

However, I am not big fan of documentations. Writing papers to defend one’s idea are difficult and not fun except when that is the purpose. It has its merit in academics, but in practical situations of programing, we are trying to get things done rather than to propagate an idea. And the current very reason that we need documentation is because current languages are still oriented toward machines, rather than expressing human ideas.

What I want to see in the direction of programming research is a system that takes in what is intuitive for humans and translate them into what is ready for machines. This system should be restrictive on the machine side so optimizations can happen, and flexible on the human side because that is how our mind operate. Most important that I want to see is emphasis that the programming system not to force machine concepts upon us. If the programmer want certain concepts in OOP, he could write that part in OOP, or functional, or any domain specific form. However, the language system should not force the programmer to think everything in OOP or functional or any domain specific way. Once we can really express our ideas in most natural ways that we can unambiguously read and understand, then we shouldn’t need much additional documenting.

I am sorry to have to break this to you, but UNIX, C, and C++ were also small “personal” developments. Ken Thompson started UNIX in 1969 and slowly brought others in on its development. Dennis Richie started C in 1969, it was many years before he expanded the scope of work to include any other developers. Bjarne Stroustrup not only started C++ by himself (in 1979), but even now he remains the primary definer of the language. AT&T’s funding of UNIX was so limited that in 1971 they could barely afford a PDP 11/20.

Instead of trying to mimic our colleagues’ traditional practices, “computer scientists” ought to be showing the way to a new kind of science — maybe that new kind of science or that one or maybe something else. I dare to suggest that the something else is related to the design of things that have software in them. It should not be called Science. It is a bit like Engineering, but it’s not it either because we’re not dealing [just] with physical things. Technology doesn’t cut it either. It needs a new name, something that denotes “the design of things with software in them.” I will call it Design for short, even though that word is so abused that it has lost its meaning.

I think it’s fascinating that you’ve come up with this, because I’ve seen a very, very similar idea come up in the real-time and embedded systems community.

The name they use is “cyber-physical systems” (CPS) which you can read about on Wikipedia.

Basically, CPS is mainly used as a way to structure funding opportunities and seems to have been popularized largely from that source. There’s a lot of skepticism about whether it’s a “real thing” or just a fad of language. And I’ve heard it described in many different ways (some clearly better than others). The best way I’ve heard it described is thus (you can imagine I’m giving a spiel at a conference, trying to sell you on the idea):

Engineers used to build things. Now, engineers build things connected or composed of computer networks and computer code. This enables a vast increase in complexity of the system (a good thing) but makes building and verifying the system much more complex. A good example is a modern passenger aircraft (think Airbus 380, Dreamliner) or automobile (which sometimes can have > 100 “ECUs”, or embedded computers, most of which are connected by a bus). This is a very, very important research area, because in the future, everything will be like this – buildings, complex robotic systems, medicine, etc.

Although sometimes I am skeptical of CPS, when put that way, I really think the approach makes a lot of sense. Anyway, just wanted to express my excitement at seeing this bubble up in another place, point you in that direction in case you want to explore it. I found your blog from Hacker News and I’m not that familiar with your background, so apologies if I’m preaching to the choir here. Hope you see the connection I’m trying to make.

I consider CSP concurrency very useful and innovative. Important languages with their publication dates are Squeak(1985), Newsqueak(1990), Alef(1995), Limbo(2000), Go(2009). This doesn’t seem “no innovation” to me.

Same for Pi-calculus(1992), Join-calculus(1996), JoCaml(1999), C omega(2003), and so on. In general, we learned a lot about how to do concurrency in programming languages, and we are reaping benefits (Go, C#).

Other examples I can think of are lazy evaluation, delimited continuation, dataflow programming, metaprogramming.

Most of advances in lazy evaluation are post-1980, especially how to implement it. The Implementation of Functional Programming Languages is from 1987. Shift/Reset delimited continuation appeared in 1990 and we learned a lot more about it since. For dataflow programming, SISAL is from 1983, Oz is from 1991. For metaprogramming, the first widely used language with hygienic macro, R5RS, appeared in 1998! Both MetaOCaml and Template Haskell postdate 2000!

Is there still anything to innovate in programming languages? Yes, there are *a lot* to innovate in programming languages.

Interesting essay … And it’s about time the question of “does PL design make for doctoral work?” and “if yes, how do we evaluate?” gets asked.

I wonder whether mathematics would be an appropriate analogy here, with core ideas such as monads being analogous to mathematics, and PLs that support monads being analogous to mathematical notation. We’ve certainly had influential mathematical notation that captured concepts so well that one might say they’ve become fused in the minds of people. Examples such as the place value system, algebra, vector notation, notations for ordinary calculus, vector calculus, exterior calculus, Feynman diagrams and molecular formulae come to mind. Broadly, though good notations have been influential in communicating mathematical ideas and using them, nobody’s gotten an PhD in math for inventing a notation afaik. At best, these inventions come in the form of a paper or a note.

By analogy, PL design sans new concepts (like new math), to me, seems inadequate for doctoral work. This criterion rules out purely syntactic contributions and I think that’s a valid criterion – unless one wishes to study Whorfian issues like impact of syntax on cognition.

– “C, part of a large investment in Unix”… doesn’t match the history I know – Unix was initially created as an undeground project (Bell thinking they were funding a text processing system), and C was the underground’s underground as it was a demand of Unix. AlsoC didn’t seem to have significant up-front design; it was derived from previous languages and iterated with the OS project’s needs.

– Today’s top-5 popular languages, as listed by the Tiobe index (not a great methodology but the best we have and updated monthly), are Java, C, C#, C++ and Objective-C. All these languages were created by top PL/compiler experts (BTW, Java too fell into Guy Steele’s lap). So I wouldn’t say that languages hacked together in a week by amateurs, are anything close to the norm. Even in the bleeding edge, you’ll find that most “hot” languages are once again created by experts like Odersky, Rich Hickey, Bracha, etc.

– Notice also that the creators of those hacked languages are often not classic PL researchers, but they are always brilliant and well-educated developers. Example: Larry Wall had a BS in “natural and artificial languages” and followed with graduate studies in human linguistics–this mix, with the stronger focus in the human languages, easily explains his approach with Perl. Larry’s classic article “Wherefore Art, Thou?” is essential for this discussion: http://www.linuxjournal.com/article/2070.

– Languages are expensive tools; the switch to a new language is a huge investment, from each developer’s learning curve to the enormous weight of legacy code and the wide and deep ecosystem of supporting toolchain and libraries. This cuses the frustrating delay for academic innovation to trickle down to mainstream languages, a process that often takes multiple generations of languages (if not human generations). “Universal” platforms, from Microsoft’s .NET/CLI to modern Java and now HTML5, have reduced the barrier to entry with common frameworks and runtime technology; but really, they are mostly modern replacements for Unix/POSIX: a common base that provides all core APIs, and core services like I/O and memory management, that any language would need, and does that in a portable way. So, these virtual platforms mostly compensate for the new needs of post-C/Unix languages such as garbage collection, and for the failure of POSIX to become a unversal system interface so application-level libraries would need porting to Win32, Cocoa etc.

– Because languages are tools, their success is the result of adoption by millions of rank-and-file professionals, 90% of those as unable to distinguish the qualities of “properly-designed” PLs as I am unable to distinguish a $1000 champagne from a $20 sparkling wine. This is critically different from most academic work, e.g. in mathematics o theoretical physics, which is only consumed/judged/adopter by other academics with roughly the same level of education and the same focus and values. Well, it’s not that simple because the first stage of adoption is typically driven by some kind of elite, still the language eventually needs to scale to Joe Developer, which never happened and will never happen with languages like Haskell, regardless of its significant adoption some years ago and very mature implementation.

Thanks for all these comments! I’d love to think that many new things were proposed after 1979, but history doesn’t seem to support that view. Here are some concepts mentioned in this discussion:

– Dataflow programming: late 60s
– Actors: early 70s by Carl Hewitt
– CSP: late 70s by Hoare
– Pi-calculus: not terribly different from earlier work on CCS. The CCS book was published in 1980, the work was done before that
– Lazy evaluation: early 70s within work in lambda calculus
– Metaprogramming: early 70s
– Continuations: mid 60s
etc.

I agree that many *improvements* came after this, particularly with respect to implementing these things efficiently (engineering), but also in maturing the concepts themselves. Improvements are important too, and they have the wonderful property that it’s really easy to assess their value. But it seems that the *innovations* (i.e. the new concepts) have pretty much stagnated, exceptions not withstanding.

I’d love to be proven wrong.

“But it takes 20 years for design ideas to come to the masses.” Maybe, maybe not. Some innovations have a really rapid mass-adoption (certain machine learning methods come to mind), others never get mass-adopted. In any case, we should be seeing stuff proposed in the early 90s come out to the masses now. Where is it? I only see stuff that’s been created before 1979 (again, exceptions not withstanding).

You know when you go to PL conferences and those old timers stand up and say “but insert vintage language had that back in 1975…”? I used to find it really annoying. But they’re right, for the most part. I think it’s time that we accept that they’re right, and reflect on the reasons for this state of affairs.

Thanks also for those who point out that C was an underdog project. I’ll update the essay one of these days. I guess the point should be that before the PC era, this kind of work was exclusive of the very few lucky ones who had access to very expensive computers — and that tended to happen only in University labs and Industrial Research labs. Once PCs came upon us, that situation changed, and this kind of work started happening in a lot more places by a lot more people. Design of things with software in them was democratized.

Hi Crista — I don’t have time to take apart your essay, but I’d like to add a couple of
public comments.

1. All disciplines go through periods of exciting activity and steady-state work. See
Thomas Kuhn’s short book “The Structure of Scientific Revolutions.” Even if you do
not accept his judgement (on paradigm shifts), you should read it for the collection
of historical work on the evolution of disciplines.

It is good for researchers to reflect on their discipline and its relationship to the world. It is dangerous to think our own discipline is facing unique problems, and it is even more dangerous not to know the history and philosophy of science.

[[Example: I agree with you that continuations and delimited continuations
— as I proposed them at POPL ’88 after working them out for 4 years —
fall straight into the steady-state part of PL research. Indeed, as you say,
Stoy had a similar idea in the 1970s for his OS work, though I do claim what
prompt and control/c/callcc did was much more. Nevertheless, it’s small
potatoes and yet, I enjoyed working on delimited continuations for years,
and I enjoyed it even more getting them just right in a production system
a few years back (ICFP). It is fun when you see all the pieces fall into place.]]

2. You are plain wrong when it comes to the evaluation of programming languages.
When IBM switched to Java whole-sale, it had gathered a large amount of data on
the productivity of programming in Java (with memory safety, type safety) vs C++
(lack of both). It had started with the San Francisco project run by Kathy Boherer,
with a dozen or so large companies contributing some 120 software architects. These
people determined that Java improved the productivity of average programmers by
a factor of 3 or more. As someone mentioned, PLs are a major infrastructure investment
and switching infrastructure is expensive. Hence when a major, large company does
switch, we should pay attention. Sadly, we also need to accept that they perceive such
data as a competitive advantage and will therefore not release it.

3. As far as ‘design’ is concerned, I agree with you. Standard PL conferences — perhaps
with the exception of OOPSLA — give way too little credit to design. When they do,
including OOPSLA, it is in a strange fashion. Find 20 people to run a symposium on PL
design. I’ll attend.

4. I have heard the lament about hobbyists designing PLs many times now, and I have
formulated it myself in semi-public spaces since 2000. I have found myself to be wrong
in two ways:

4a. These languages tend to inject one or two new ideas into the discussion. In addition,
unlike PLs designed by academic researchers at universities and labs, the break-thru
languages address dire needs of practical work and design — immediately and on the spot. Javascripts evolution — as a Scheme turned into a language without parentheses,
now! — is a classic example. Brendan Eich should write it down for history’s sake, and not just the sanitized version that leaves his superiors blameless.

4b. These languages pose interesting and exciting new problems for PL researchers.
I have worked on adding types to untyped languages for 20 years; since the web placed
languages such as Python and Javascript into the center of new software designs (1998ish
latest), this work has become tremendously relevant. I am actually pretty sure that it
will evolve into a nearly-big idea that people will pay attention to.

5. Last but not least, don’t escape. If you find other areas more challenging, do pursue problems there. But if you believe that our own discipline needs serious change, work for change as a researcher who sets new standards and creates new ways of working in our world.

I didn’t escape At some point I felt the need to go and explore other parts of town to see what’s all about and how things are done there. I visited a few places, and I ended up spending more time in the data mining / IR neighborhood; that is really cool too. Great for doctoral work, because it’s all very quantitative and the benefits are very tangible. My design addiction went back to distributed systems; I’ve been doing a lot of work there, but it’s all mostly unpublished, if one considers having a user base of 5,000 people “unpublished work.” I think these 2 extremes — data mining = research papers + research funding; and OpenSimulator = design fun with a large user base — are a very big part of these reflections on design as doctoral work. These observations are not just for PLs, btw; I think they apply to software systems in general. Languages are particularly good to reflect upon.

Hope to continue this conversation with you some time! As for this essay, please do break it apart if you have time. Maybe then it can become a real paper… heheh. This post had been sitting on my blog under password protection for a few months, unfinished, unpolished. Some students started asking for it, so I freed it from the password.

On another note, I think there is rather too much cultural emphasis on
“innovation” in the sense that Crista describes here (“big ideas”). After
all taken to the extreme, *everything* boils down to the lambda calculus
or state machines, so by this logic, we might as well have called it a day
by the end of the 1930s. One extremely negative consequence (or cousin?)
of this emphasis, is the need to find single herculean figures who cause
“paradigm shifts”.

Sure, that might happen once in a while, rather more rarely than you think.
In reality progress is rather more bottom up, in fits and starts with lots
of “little ideas” cancelling or building up on each other, lots of dead
ends (with, sadly, the final “credit” not going to the person who
“discovered” an idea, but with the person after whom the idea “stayed
discovered”.)

There needs to be *far* more importance placed on the critical “steady
state” work that Matthias refers to that is needed to refine/improve/fix
some idea till the point the “pieces to fall into place”. This might be
especially true in PL because there are so many different moving parts
that need to be reconciled.

And so, I have rather less patience for the “old timers”.

(Of course, like all other disciplines, we also have some wheel reinvention,
but thats another matter altogether…)

ps: as phrased, this question appears impossible to answer:

“Where is it? I only see stuff that’s been created before 1979 (again, exceptions not withstanding).”

I bet you in 10 years, we’ll have turned a full circle, and the above date will be updated
to 1989, and thats how we move forward 😉

I think your conclusion about the quote regarding Haskell is incorrect.

Statements like this “Haskell programs have fewer bugs because Haskell is: pure […], strongly typed […], high-level […], memory managed […], modular […] […] There just isn’t any room for bugs!” are nothing but wishful thinking. Without the data to support this claim, this statement is deceptive;

True, if you read the sentence “There just isn’t any room for bugs!” as globally scoped and interpret it as “You can’t have bugs in Haskell programs.”, this would certainly be a silly claim. And even though I don’t believe that this is the intent of the sentence, I think it would be better to either drop the sentence altogether or at least qualify it and say that there is no room for certain kinds of errors (such as type errors or memory errors).

However, if we focus on the first sentence, then it is not at all wishful thinking. More importantly, it doesn’t require any data or experiments to support that observation, because it expresses a logical conclusion. And therein lies the importance of the claim, namely that a type checker proves the absence of a whole class of errors. So it is a simple fact — an analytical, non-empirical fact — that, e.g., type-correct programs contain fewer errors than arbitrary programs.

Empirical validation is a requirement in Science. Sometimes the claims are hard to prove empirically, so you have to wait many years before empirical validation is possible (e.g. Physics). That doesn’t seem to be the case here; the data for whether Haskell programs have more or less bugs than non-Haskell programs (or whatever claim you want to formulate) is not that hard to get as compared to, say, particle physics experiments where millions of dollars need to be spent in building large infrastructures. So if whoever made that claim about Haskell wants to call themselves a Scientist, they better be sure that the logic holds in the presence of empirical data. Or change the claim to something less ambitious like “type checkers eliminate a whole class of errors; Haskell has a type checker, therefore, a whole class of errors is eliminated.” Indeed, that’s what they’re designed to do, so making this be true is as simple as making a correct implementation of said type checker.
Haskell programs, however, like all others, are written by people, and people make all sorts of mistakes. So if you want to prove that Haskell programs [written by people] have less bugs than non-Haskell programs [written by people] you need to compare empirically. You may be unpleasantly surprised with the results; or you may come out a winner, in which case the whole world will be convinced that type checkers are an absolute must-have in every programming language.
Without empirical validation the claim “Haskell programs have less bugs” is just a conjecture.

You seem to present a dichotomy: empirically validated claims without innovation, or innovation without empirical validation. I do wonder if these are the only alternatives? Even if they were, languages – along with tools, methodologies, nearly everything in the vicinity of software – are so much in the latter camp today, it would seem that a little more emphasis on empiricism – that which in most disciplines earns the term “science” – couldn’t hurt. I am entirely unconvinced that “It is also often the case that over time, the enormous amount of testing by real use is enough to provide assurances of all kinds.” Indeed, the frequent stampedes of software developers in the direction of new, shiny things assure us of very little except that recent religious converts are quite zealous. I take your assertion as more of a sigh. Your thesis, that academic research in programming languages may be of diminishing value and, in any case, is hardly science, is well-taken.

A few proofreading notes:

“worth endeavor” – you probably meant “worthy”.
“innovate new software” – you probably meant “innovative”.
“we have in hands” – I think the idiom should be “we have in hand” and not match number unless you wish to write “we have on our hands”.

Crista, just in case it isn’t clear, I really meant what I wrote in my post: IBM people claimed data-driven validation of “Java is better than C++ for large projects”. Their evaluation method uses dollars-spent, i.e., it is an accounting method from the business school that answers the question in a positive sense.

Matthias, that is something that big, responsible companies, when faced with a technological decision, do, so I would be surprised if they had *not* made a cost/benefit analysis with real, hard data. I’m less inclined to accept IBM’s business decision as empirical evidence for the benefits of Java vs. C++, in general, unless they open up their data and methodology to scrutiny. That would be very interesting to see!
In any case, CS academics should take note of such data-driven practices

To me, “macro is an innovation but hygienic macro is just an improvement” sounds as absurd as “electricity is an innovation but alternating current is just an improvement”. Ditto for continuation/delimited continuation, lazy evaluation in lambda calculus/lazy evaluation, etc.

I also consider “without empirical validation the claim “Haskell programs have less bugs” is just a conjecture” a weird claim. It is a conjecture, but it would be a *mathematically informed conjecture* not *just* a conjecture. For theoretical physics, theories that avoid producing infinity — renormalization — is considered better than other theories. Why not apply same for type safety?

Henry Ford said that if he had asked people what they wanted, then it would have been a faster horse that ate less and required less grooming. PL research stuck in that space, dreaming up of flying horses.

There are basically two issues to consider:
1. Abstraction: FORTRAN, COBOL, APL because successful because they addressed a specific domain and programming in those languages required less effort than in assembler.
2. Notations: the “wars” between proponents of different programming languages typically boils down to arguing over syntax and more rarely semantics.

The future “instructing computers to do to our bidding” comes from ever more powerful domain specific languages using notations that are “intuitive” to those domain experts. Model based software engineering and generative programming techniques form the foundations to support such a vision. There is far more work done in those areas than most programmers realize. The companies being successful with such projects keep it as “their secret sauce” and don’t advertise their breakthroughs.

Yes, the new paradigm of using iPad like internet access devices and using touch and drag boxes of language structures to write programs for software agents will define new PLs. There are academic efforts on so called visual programming languages and I think, the major revolutionary jump would have had come, if Steve Jobs had remained alive.

Speaking as the developer of an aggressively non-academic language (Objective-C), I’d like to suggest a project for some like-minded individual.

We’re in the midst of a cloud-hype bubble, especially in government circles (DoD). Doing that the old (current) way, each and every application handles identity management (authentication) and access control (authorization) itself, so that who can access what is under the control of the administrator for that application.

So there’s a lot of attention being devoted to doing that in some more centralized way; basing access control on explicit policy instead of each administrator’s whims. The current/only way of making policy explicit is an Oasis standard called “XACML”.

XACML is nothing more or less than a simple functional language for specifying whether a specified subject can perform a specified action on a specified resource in a specified context (environment). So far so good. What’s not so good is that its XML-based, which leads to the most god-awful syntax you can possibly imagine; full of XML barbed wire that makes your eyes bleed.

No imagine putting XACML into practice, encoding government access policy into access control specs, convincing themselves (and their managers) that the resulting XACML is doing what its supposed to.

So here’s my proposal. Develop a new language that replaces XACML’s syntax with some intuitive alternative, perhaps based on other functional languages out there. Scala is one good candidate. Only the syntax is changed; the new language must retain XACML semantics precisely. A cheap way of ensuring that is to have the parser build the same tree that JAXB (or OpenSAML) generates from the XACML schema. Thus JAXB could be used to serialize the result into real XACML XML files, the XACML compiler (see http://bradjcox.blogspot.com) could be used to turn it into java for runtime, or Sun’s XACML interpreter could be used to interpret it on the fly. All that changes is the syntax; everything else works unchanged.

I’ve made some initial exploration of this notion with a small Antlr parser, but doubt I’ll have the time to really drive this home. If some one does, drop me a line.

Does Jeeves (a Scala DSL) come close to meeting your criteria? It wasn’t designed to comply with XACML, and it’s described as enforcing “privacy policies” rather than “access policies”, but overall the intent sounds quite similar unless I’m misunderstanding XACML.

Thanks for the pointer to Jeeves. Just had a look. Its targeted at privacy, not access control. What I had in mind was far simpler, a straight/simple translation from a friendly syntax into XML or Java (or Scala etc) with an emphasis on raw execution speed and strict compliance with OASIS-defined semantics. Jeeves seems to go far beyond XACML compliance. Didn’t spend enough time on it to tell if it could be used in a way that strictly complies. Those are first impressions based on a quick skim, so easily wrong.

I agree with Joe. There seems to be an inability to accept the empirical evidence in the ‘cool stuff’ graph and move on to new ways of doing things. So much for science.

Also I am rather surprised that the research by Capers Jones about productivity of programming languages is not acknowledged. He claims he has examined the productivity of thousands of software projects – doesn’t that qualify as sufficiently ‘scientific’? BTW you may all be shocked to see what ranks high on his list.

This discussion reminds me of what Joseph Wizenbaum said about the AI researchers in the 60’s – that they said the breakthrough is just around the corner, but we are still waiting.
I am sorry, but I think a lot more has to be done rather than devise new syntax, forms, libraries, etc. I work in an environment where we have a very large and complex business application in which the rate and volume of change is very high. Productivity is very important to us – the PLs do not deliver enough so we have to augment them with all sorts of home grown tools – the main ones being software management, metadata management, test environment generation and code generation. But we need more, and we should not have to do this ourselves.

I think there are two sides to a programming language. The hardware side and the human side. Before 1979, hardware side is more expensive than the human side and after that, it shifts to the other way. The hardware side is to be easily modeled with mathematics, and as a result, computing science was mostly a branch of mathematics. Expensive hardware, abstract models, and a narrow base of experts restricts the language design to be a “serious” activity. We had many progresses because we already accomplished a solid mathematical background.

Afterwards, especially after 1990s, hardware side is no longer critical, so the attention shifts to the human side, which is more of a cognitive science. It is hard for academic rigor but easy for intuitive insights.

FORTRAN, COBOL, C, LISP, all emphasize on the hardware efficiency. LISP goes to the extreme of making the language itself a data structure. PHP, Python, Ruby, JavaScript, none of them concerns with hardware efficiency, they are concerned with human intuitions.

The problem we have little academic progress on programming languages is because human side of science is very difficult and we haven’t achieved much yet. Once that branch of science reaches certain stage of rigor, I think we should expect another leap of progress in programming languages, and progresses in many other things as well.

I agree with the poster that most new languages today appear to be mash-ups of existing concepts and languages. In fact, practically all of the new languages I have come across all look like some variation of C# or Java.

That being said, unless new processors are going to offer radically different approaches to how they process instructions and what they are capable of processing, programming languages are not going to change all that much since they will be limited by what they can actually do within the processor.

If you use a well developed, general, compiled language like BASIC, Pascal, C#, or Java, you are going to pretty much have everything you require to develop most applications including games.

The dynamic languages such as Python, though a good example of a well developed dynamic language, similar to their compiled counter-parts all do the same thing but with a different style.

Should research continue into programming languages? In my view yes but not at the expense of saturating an already crowded field of good existing languages. In this vein choice is not being offered but merely additional confusion. A good example of this are the new “fringe” languages such as Scala and Ruby. With the exception of some different functionality (functional programming in Scala and dynamic generation in Ruby) you are really not getting all that much for efforts made into adopting these languages unless you like living on the periphery of the programming world. Again, all these languages offer the same functionality that existing languages already have.

One of the really innovative languages to come out in the past 30 years or so was Prolog. It promoted a completely different type of design paradigm for development but it was only viable for expert systems development for which there are only approximately 2000 different applications to which the language can be applied.

Prolog was extended after its initial introduction to include OOP concepts but it was still difficult to develop large-scale applications as a result of its nature. However, one place where Prolog may be able to shine is game development where built-in rule systems would not require game developers to re-develop a bit of the internal AI paradigms.

(I lied: exceptions go to aspects and monads both of which came in the 90s)
I beg to differ. Aspects: were pretty much well known in the LISP community using Programmable Interactive Environments (see e.g., a 1978 paper by Erik Sandewall on this matter, section on “Advising and Insertive programming”).Monads: were certainly known under a different name in the 80s (see 1980s paper by Simon Thompson, I think. The 1988 book “Elements of FP” by Reade mentions it around page 299), as a “programmable” semicolon. The last one was invented by Peter Landin in the 60s, I believe. See http://okmij.org/ftp/Computation/IO-monad-history.html. There is nothing new on earth, really…

This is the goal of Language Design. Everything can be derived from this.

Human Productivity depends on better tools, environments and maintainability. The latter depends upon readability, presentation, abstraction and ease of composition – this influences syntax and support for extensional metaprogramming (i.e. “Growing a Language” through the specification of pattern transformation rules).

Computer Productivity depends on optimising performance to get results quicker for the human user and to boost the overall productivity of the system of which it is a part. This can make ‘Live Programming’ environments possible, where the program is a mix of interpreted and compiled modules – where the latter may be selectively ‘unfrozen’ for rapid prototyping of new features.

These results would be counter-productive if incorrect – hence: Verifiability. All work on separation of concerns, constraints, equational reasoning, type theory and dynamic manipulation control interfaces that avoid weaving independent aspects into a stateful model aid comprehension, testing and state-of-the-art proof techniques.

Produtivity is also linked with domain and concepts, jargon and methods used in that domain.
Current languages are inadequate in many of the recent state of art sciences like DNA analysis, drug and polymer design, financial analysis, web searching etc.

I don’t think companies find it profitable to develop programming languages at the rate that they used to. Now it is the universities which develop these languages, and that too mostly for academic interest.

Thank you for writing this. As a software practitioner without formal education in computer science, and a programming language design enthusiast with a stupid hobby project in the works, it’s heartwarming to know that somebody in academia shares my sense of how to go about what I’m doing.

So it’s fundamentally a methods issue. “What is a rigorous programming language or piece of PL research?”

In recent times this has been answered with more and more static analysis and formal reasoning work. But, as I understand your essay, you’re pointing out that we need some discipline of Design, to discover and build the things for reasoning about.

“Necessity is the mother of invention.” So, what is it really that we NEED from our programming languages? In PL we definitely have a Sapir-Whorf problem: without a programming language in which to formally express something, we can only grasp about in the air at what we want to say. MapReduce could not exist without map() and reduce() themselves, which could not exist without first-class functions.

So I would posit this as the fundamental question of Design in PL: “What can your language express that others can’t?”

“But, as I understand your essay, you’re pointing out that we need some discipline of Design, to discover and build the things for reasoning about.”

Yes! Pretty much! I’m not the first one to suggest this, btw. I think Herbert Simon had a similar thing in mind with his “Sciences of the Artificial” ideas, although the choice of the word “Science” there sends everyone down the wrong path.

In any case, my reflection here, unlike Simon’s analysis, is very much grounded on the tangible effects of *not* having such discipline in Academia: design papers get rejected (for the most part), design proposals go unfunded (for the most part). I’m not lamenting it, I’m just making this observation. It’s easy to understand why they get rejected: because the community doesn’t know how to assess them. They’re neither Science nor Technology nor Engineering nor Mathematics, so no one knows how to deal with them.

Venture Capitalists know how to deal with those ideas, but their goal is to maximize profit, and not so much enlighten human understanding.

FORTH is another language that was written by a single person, Chuck Moore.

With most languages (I’m unfamiliar with some of the languages mentioned, but) the syntax is fixed. Control structures are predefined. Basically all you can do is create new functions.

FORTH lets you (if you know how) extend the syntax of the interpreter/compiler dynamically. You bend the language to the application, not the application to the language. Being a stack based language it has been stuck with word sizes (16, 32, …). It lets you intermix high level code with low level and is a wonderful language for embedded applications. But I feel it has potential if rewritten for this object oriented age.

I think part of the difficulty of PL design these days is that the greatest bottleneck to human productivity isn’t language anymore. Library availability and quality is probably the most important deciding factor in using a language: that is, it’s more about what work you can *avoid* doing, and these days, you can avoid a whole lot.

Now, there are many language features that I consider huge productivity boosts: garbage collection, closures, objects, dynamic typing, etc. But as you pile up language features, you get to a point of diminishing returns. Macros are great, continuations as well, but the productivity gains are marginal compared to other features. The best one can expect from developing new language features is to inspire improvements to a new or existing popular language, but the magnitude of the actual impact is far from clear. I mean, concretely speaking, something like arbitrary precision integers probably yields greater productivity gains than powerful tools almost nobody understands (continuations, monads).

Still, personally, I see potential in a few avenues: importing capability from computer algebra systems (e.g. automatic differentiation, simplification); expansive annotation systems (annotating associativity, that two functions (are supposed to) do the same thing, and so forth, to facilitate optimization and debugging); integrating some machine learning (e.g. you could define a measure M, like the time taken for the program to run, and tag variables as “to optimize with respect to M”, leaving it to a JIT to figure out optimal values for performance, memory usage or a criterion of the programmer’s choosing); probably others I haven’t thought of.

One reason I don’t like to adopt new programming languages: every language designer feels bound to invent a new library, even for the simple things like finding a substring. This is really tiresome.
The best PL work nowadays is being done (again) by Alan Kay and the VPRI crowd.

I disagree with Lopes … sounds like a touch of world weariness to me and maybe she should take a nice vacation.

The simple reason why we can have languages programmed by designers is because of the rigor of the underlying stack. An alternative example, would be HTML5. Just look at the insanity that is HTML5 which basically says that we will take all the anachronistic parsing behavior that is common across browsers and make that ‘the standard’ (btw thats trollish of me, but I hope everyone understands) … its a race to the bottom but its underpinned by the fact that lower abstractions have a solid foundation.

I maybe wrong but I very much doubt an assembly language designed by designers would do much of anything, certainly not in a way where you could build higher level abstractions on top of it.

Lopes is not recognizing the fact that programming (and software) is emerging from its ‘hobbyist’ phase and that today’s developer are much less likely to choose proper programming languages (after all software is psychology in the end).

An analogy is with the auto industry … there was a time where we could all maintain our cars and do significant amount of work and customization to them. The 50’s and 60’s even up till the 80’s was the auto industry ‘hobbyist’ phase … pop open a hood today and most of us don’t want to get involved. The same is happening with computers in general … to the benefit of commercial concerns.

Back to the real problem which I believe is two fold … firstly education is a problem in that parts of academia is still catching up with industry but this is changing … almost to the point where many of the best bits of software incubate in a university somewhere to be commercialized by students leaving.

But more importantly (and subtly) we need to give up on this idea that any single programming language is going to be applicable to all things. Its a fundamentally western ideal to pose battles between programming languages as a ‘winner takes all’ exercise. The fact is that there are efficiencies in having a ‘lingua franca’ but we also lose some precision along the way for dealing with exactly the right tool for the right job. We need to embrace heterogeneity and apply principles of convergence judiciously … not just bet on programming ‘horses’ and blindly espouse their benefits.

Having been a programmer for so long I have seen my various pet languages go through the adoption curve … I cringe when I see the computer media obsessed with ‘memifying’ everything creating hype which in turn forces people to use any specific tech far beyond its original intent which is followed by the eventual backlash where people say ‘INSERT HERE is dead’ and actually the technology goes on to live for another 20, 30 years.

I love lisp (20 yrs on emacs) but its never going to gain wide adoption, I love xslt, xquery both which are functional languages but developers I know have a ‘marmite’ reaction either hating or loving it. Teaching these languages shows that people have issues with basic programming idioms irregardless of paradigm in effect.

Like any actor, musician who wants to play to an audience or mother who wants the world to know of their child’s genius or even a soldier who wants to get a chance to illustrate their devotion to duty … its understandable that all the hard work that Lopes does results in how she feels.

But this is very common in science where hard work and graft support and underpin each little micro step which eventually leads to future breakthroughs … its highly annoying that ‘crowd think’ results in us doing unholy things with javascript but I’m not going to worry about it anymore, think back far enough and things were much worst in computers (and if javascript killed flash thats enough for me).

Nice post! I’m also an academic, in experimental particle physics however. Students only get Ph.D.’s going after physics results – measuring physical constants, etc. Which I love doing. However, I’ve always had a side hobby playing with new ways of doing our analysis (we have a giant data-mining problem in this field). Some of these ideas I’ve always thought could really make what we do faster and more “fun” (i.e. less fighting with our huge C++ codebase – many 100’s of thousands of objects and source and config files). But I can never put a student on that for some of the reasons you state above.

I am glad you’ve found something that interests you. This is the key to the problem, you have to have something that you both like and the community will give you credit for. For me, my side hobby ends up as talks and posters at a large conference on computers in particle physics – in a small side parallel session for the about 10 of us that are interested in this stuff.

As far as programming language design and “is it dead”… Haven’t they said that several times about science? Watching new stuff steadily flow into the mainstream (I’m a heavy user of C++, C#, and python – C++ is just *too* slow on the uptake!), I can’t help but wonder what else is out there. Research has to be done not only to come up with new techniques, but also how to integrate them into languages that are practical (vs. pure). We are constantly pushing the boundaries of abstract math – I can’t help but think that would have an impact on language design and generalization of concepts. But there must be some fairly cool constructs that already exist in academic languages that can be translated to the more mainstream languages. Simlification…

At anyrate, good luck. And don’t stop pushing the boundaries. There are millions of interesting problems out there. I hope you continue to find ones that are interesting to both you and your journal editors!

Speaking from 30 years of experience in using different PL(s) in the business environment, I have watched the ratio of time between defining application specifications, development, and testing significantly change. Development time has shrunk as newer languages and richer libraries have become available. Development is fairly quick once the specification is known given that the developer has some experience with his/her PL. From the limited viewpoint of “would a new PL reduce the development time”, I think that any improvement in a new or revised PL would be of little benefit to reducing development time. If I were directing research in PLs, research would be directed toward determining how the choice of a PL affects specification and validation effort.

I feel compelled to point something out
I feel that the commercial success and ease of use and the ease of learning of a computer depended on it’s programming language
The personality of the computer depended on the programing language
I would say the language was the computer
One example of ease of use I would argue, and commercial success based on language I would argue was the zx spectrum
I argue we should do better and have the modern equivalent
I’m impressed with the integrated software from framework from Ashton Tate and it’s programming language that integrated with documents and outlines
I’m impressed with lisp and emacs
I’m impressed with smalltalk
Maybe I’m impressed with scratch visual programming language to a degree
But nothing was so simple as zx Sinclair tokenized basic with a token per key and good syntax error detection
And the language is the computer and commercial succes I believe or was in the 80’s
I don’t think much of java
I love unix / Linux
But the feeling of a language being a computer is maybe only framework or emacs besides the zx spectrum
Another interesting computer was the Jupiter ace a forth computer
Back then you turned the computer on and that was it
We could have had a lisp computer
Maybe with ssd’s we will have instant on languages
Things like the iPad are cripplelled as far as programming language potential
I think it’s a big deal that the programming language is the computer
So having a good one is important
I like the concepts of go
I thought programming languages would have evolved in different ways when I was a kid in the 80’s
Music is interesting in that it is a parallel language has loops and is real time
I was very impressed with framework implementing programming languages in outlines, spreadsheets and documents, a very powerful combination, easy to understand and very productive, think of it as emacs with outlines and spreadsheets instead of just buffers
Hope my insights are useful
Did the programming language stop being the computer in the 80’s? Or making or breaking it’s success or making it easy to learn and understand for the future programmer hacker

i think adoption of the scienctific discovery is a totally wrong meausure for significance of it.
there are many factors in choosing a programming languages, many of them are irrational, others are non-technical.one fine examples is list of benefits of PHP. most of the reasons are legal and financial. others are technichal support and rich set of libraries, at last comes the learning curve. nothing is said about productivity, reliabality or even performance.

scietific research is discovery of problems and solutions for them. tools are only made to faciliate the research. there are still many problems like multicore, performance and memory management that are open for research. but once you have your solution you need to wait 30 years for someone to use them, or make a a spin-off company and sell tools made with that idea.

what bothers me is having to use outdated tools and hacked languages for my everyday use (C++ and Ruby)

i am writing my own language hoping that i t will be useful for myself. i do it partly to have creative outlet but mostly out of frustration. C++ does not scale in complexity and Ruby does not scale in performance. i would be lucky if i can put all the innovations from PL research.

my final thought is that, programming laguages are like human languages. they are a culture. it is the language programmers speak. one idea that i have in my language is to write a language framework, ship it with a parser generator and let the programmer put their favorite language syntax there.

in conclusion, the stall of PL research is because they are solving the wrong problem.

Well said. As a fellow academic, I also find the situation distressing. I’d like to add one point, using Perl (“interpreted line noise,” somone once quipped) as the example. Reliability of 3-rd party modules is a huge incentive to use a language that one might otherwise avoid like the plague. I’ve no scientific study to backup my own very pleasant experience with contributed Perl modules, which is why I use the language a lot despite my many reservations about it.

Totally agree, and I was going to make the same point if no one else did. Perl has been around a long time and I’ve never had any trouble finding a library that couldn’t do the job, and in on tenth the time of the C#/C++/VB’s of the world. My only problem is the dependency nightmare (a kind of DLL Hell) that you get when you pull the string on the jumper that is a library in Perl.

I finished a PhD doing research in distributed systems / databases. This essay applies equally well there, at least if you replace the names of various programming languages with names of systems.

Part of what has driven me out of academia is the fact that the kind of work I like to do (design a system, then build it to explore if/how it is useful) is not well rewarded in academia. It could be that the focus on publishing papers is actually the right one, since industry seems to do a reasonable job of building interesting systems in my field, at least at the moment (see the explosion of various distributed databases that are now available). However, it certainly isn’t the right one for me.

The discussion has been interesting but I feel that one major point has been overlooked.

All the languages discussed are text based languages. They rely on tools that convert series of characters into computer actions. Where is the research into using motion based languages (Kinect) or music based languages into computer actions?

Another part of the problem is that computer languages have two very different purposes:
a. make the computer do something
b. allow another human being to understand what is being asked of the computer to do and to be able to modify that. In many cases, this second purpose is the more important. “Write only” languages have very limited application (see APL or Forth).
Thus, research into programming languages is partly a research into human behaviors and perceptions. Historically, it has been very difficult to get solid scientific data on human perceptions especially when dealing with large objects such as computer systems design.

It would be interesting to use a language like latex where the symbols mean something, no reason not to repent the things with symbols, like summation, everybody has bit mapped graphics, not just text, the languages could be more readable by having real math notation

As a young academic this article is speaking from my heart. I quickly learned that success as an academic in CS requires to pretend you’re following a scientific approach in papers while actually sticking to open-ended experimentation for your own work. I had the luck to do my PhD with a supervisor who never asked me for a proposal and even the less for a topic. I had no PhD topic for more than four of of my five years! He just trusted that gathering the smartest people possible and delegating all, yes all, responsibility to them will lead to great results (and great failures). Now as I am working at an American institution though I can see how the more formal American system with proposals, committees and (are you kidding me?) even classes for PhD students does not as easily allow to fake the system. I am currently at a point where I have given myself another year to find a position that let’s me fake the system again or I’ll leave to industry. I had been working in industry before so I know pretty much what to expect there, it’s not all sunshine either but at least it pays well 😉

Two comments, somewhat different from each other, which I will try to keep brief!

1. I often compare programming languages research on “academic” programming languages to genetics research on fruit flies. Fruit flies themselves are not hugely economically important (except, I guess, to fruit growers and vendors), but they have certain properties (such as quick turnaround of generations and low maintenance) that allow us to use them to explore concepts that are fundamental to all life.
Similarly, functional and logic programming languages don’t have a huge impact on the practice of programming (except in certain application areas), but they have certain properties (such as ease of parsing and absence of side-effects) that allow us to use them to explore concepts that are fundamental to all programming languages.
Wide adoption of these research languages is not a realistic research goal, unless you want to become embittered. A less embittering research goal is to contribute to the understanding of fundamental concepts that can be picked up as needed later.

2. Here’s a parachute-haystack-and-pitchfork story. Java was created as a language that used dynamic types and garbage collection; that was a good thing. However, it didn’t have any parameterized types; that was a bad thing. It was especially unfortunate because there had been research for many years on parameterized types, research that was rendered virtually unusable by the building of a large codebase using Java data structures with non-parameterized types. But then along came Odersky, Wadler et al. and created Pizza and GJ, a heroic and brilliant effort to harmonize classic parameterized types with an existing non-parameterized codebase (“making the future safe for the past” indeed); that was a good thing, and thankfully Sun recognized it as such.
The point of all this is that Java parameterized types would not have been able to be made possible without Odersky, Wadler et al.’s deep understanding of parameterized type systems. That deep understanding came from many years of exploration of those systems, involving a chain of researchers and teachers extending back to Church but certainly involving a lot of academic research work on languages like ML in the 1970s, 1980s and 1990s. Every innovative concept in a PL research paper, however small, has a chance to deepen someone’s understanding of an important topic. Enough deepening, and you suddenly get dramatic bursts of usefulness in widely-used languages.

Generics are great!… in my opinion. I love monads too.
But how should my opinion count wrt studies like this one:http://research.microsoft.com/apps/pubs/default.aspx?id=146635
Is this a case of giving Perls to undeserving developers? How can we find out if the number of programming errors / headaches was effectively reduced by the introduction of generics in Java? Or were developers doing just fine without generics?

These are the kinds of questions that I asked myself regarding AOP, too, so this is not just about other people’s language designs.

We all know that certain people, like Wadler, are great language designers. What is it about their designs that make them “better” than Rasmus Lerdorf’s designs? Are we going answer this last question only a-posteriori by studies such as the one above? How do we identify a fantastic design before wide deployment? — by the credentials of the designer?…
I have a lot more questions than answers, and the main purpose of this essay is to ask all those questions

Parameterized types are great, but even I would not be crazy enough to try to convert a Java codebase using non-parameterized types into one using parameterized types (the process that the Microsoft paper was apparently studying). The whole point of the Odersky/Wadler work was actually to make it possible for those codebases to remain exactly the same, while permitting programmers to introduce parameterized types as the fancy took them.

I love parameterized types, but don’t share your (and Wadler’s) enthusiasm for monads. (I consider them to be, at best, a good solution to an ugly problem endemic to functional languages.) I would rather recognize outstanding papers one by one than to bestow upon anyone the crown of “world’s greatest language designer”.

Thank you! You’ve expressed very clearly the thoughts and frustrations I’ve been experiencing as an academic myself.

Like you, I’ve often used Tim Berners-Lee as an example of someone who probably wouldn’t have been able to obtain a PhD thesis or get a journal paper accepted for his design of the web browser (Ward Cunningham, the inventor of Wiki Wiki, is another example of someone who got his glory in avenues other than academic ones). Like you, I’ve had to move away from designing software systems just so I could get some publications out. Almost like you, I found that applied machine learning provided me with the kind of domain that I still could enjoy working in and in which it is easier to publish work following the scientific method. So, yes, I can definitely play the game so I can get published and promoted! 😉

But I still find that my best work is in designing software systems. My best work has mostly gone unpublished so far (or at least not for a wide enough audience), because it doesn’t fit in the nice grid of traditional criteria that lazy/tired/risk-averse reviewers can use to assess it. It’s not the end of the world for me, as I can still publish other work, but obviously something is wrong here…

Does this new language/system allow me to think differently (i.e. is it introducing a useful new paradigm)? Does it allow me to do things I couldn’t do before? Or does it at least allow me to do certain things more easily than before? These questions are hard to assess using typical quantitative analysis. Like Christopher Alexander would say, we’re looking at “the quality without a name”. How subjective! How difficult to assess!

So what can we do? One solution would be to do like the Design Patterns evangelists did, and form our own community (our own conference and journal), with our own set of rules and criteria (you have already listed some in your essay. That’s a good starting point.). We need some reputed and risk-taking leaders. We need them coming from diverse backgrounds. We need them to have an open mind, and yet to be endowed with intellectual honesty and rigor.

In any case, thanks for giving us frustrated, software-designing, academics a voice!

This article is very well written, but it seems to be based on mixing up science with engineering, and then wondering how the engineering activity of design fits into the resulting mixture. Let me try and disentangle a few things, because I think Crista already knows the answer but just hasn’t laid the parts out clearly enough to make that answer obvious.

While I cannot claim to have discussed the philosophy of our discipline in my old department (which combined EE and CompSci), I don’t recall any faculty member ever having confused their engineering activities with their scientific ones, and we certainly were involved in both. Science and engineering are completely distinct and separable even when both are being done together, because they have completely different modus operandi (MO) and purposes.

The purpose of science is to understand something that is not currently understood, and it does so through application of its one and only MO, the extremely well known and very formal Scientific Method. Very briefly, it has two halves, a theoretical half in which mathematical theories are devised and testable hypotheses extrapolated, and an empirical half in which observations of the unexplained behavior are made and the measurements compared against the predictions in order to disprove the hypotheses. If after countless such cycles of the MO nobody around the world can disprove any of the predictions derived from a theory, then it gains credence in the scientific community as tentatively valid in the domain tested within the bounds of experimental error, despite no positive proof being possible through this MO.

Engineering is completely different. Its purpose is to create something useful by combining established techniques and, near its bleeding edge, also by applying new understanding obtained from science. Its MO is also completely different from that of science, involving the equally well known but less formal process of discovering requirements, evaluating alternative approaches, designing solutions, implementing and testing prototypes, and in the case of commercial production, devising the production systems as well. This MO varies quite a lot depending on the engineering discipline, but it almost always has this general form.

None of this is in dispute in the science and engineering communities, as their purpose and MO has not changed for many decades coming up to centuries, although the language used to describe them has changed somewhat. Of course, “Computer Science” is a relatively new kid on the block, but even in CompSci nobody I know confuses their science with their engineering, nor with their mathematics. CompSci embraces all three disciplines, but they are completely distinct at any given time, and I expect that every computer scientist is aware that the label “Computer Science” is a poor reflection of what they actually do.

Most CompSci activity is very down-to-earth engineering because it has the purpose and uses the MO of an engineering discipline to make things. Theoretical CompSci is a branch of mathematics, and quite rightly has its own label because its domain is so specific. And finally, only very rarely is the MO of science applied to investigate an as-yet unexplained phenomenon in CompSci — the computer scientist is then doing Science. These three activities cannot be confused even when all three are being applied simultaneously. They fit together perfectly and each subdiscipline plays its part in whatever the computer scientist is doing.

So now we get to the crux of the alleged difficulty, which I don’t think actually exists. “Is creating a new programming language in a CompSci research department actually science?” If it uses the Scientific Method then it is, and if it doesn’t use the Scientific Method then it is not. This is *by definition*. THERE CAN BE NO AMBIGUITY on this score, although of course it is possible that the MO of science is applied poorly by a computer scientist who is only a half-hearted or slipshod scientist. Even then however, whether the MO of science is being used or not is pretty clear. In virtually all cases the answer will be ‘No’, science is not being done because the MO of science is not being used, although the possibility of an exception cannot be excluded.

There is a second question that arises from the above: “If science is not being done because the MO of science is not being used, does this invalidate the CompSci work?” No, of course not! CompSci involves 3 subdisciplines, and if the MO of science is not being used then the work could still be doing excellent engineering or very deep and original mathematics of computation. Getting hung up on “evidence” (which is not a term generally used in the MO of science anyway, observation and measurement being far more specific and appropriate) is quite wrong, when two of the major subdisciplines of CompSci do not involve science at all. Note also that both science and engineering employ measurement as a very important tool, but for different purposes, which is another reason why focusing on “evidence” is not an effective way of determining whether science is being done.

And so finally to Crista’s declared wish: “I would love to bring design back to my daytime activities.” Do it! Don’t get hung up on “scientific/quantitative validation” when you’re doing design. Design is not science, it’s within the engineering subdiscipline of CompSci, easily recognized by any engineer through its distinctive purpose and MO. The same would apply if you were doing theoretical computer science: your domain of mathematics would require rigorous theoretical proofs if done formally, but as its name implies it is a theoretical subdiscipline and not science because the observational half of the MO of science is not present and not appropriate.

Try applying this acid test of “Is the MO of science being used?” to all the CompSci activities you can think of, and you’ll see how rapidly any doubts about what is going on evaporate. Even when you’re using *mathematics* to *design* an instrument to measure *unexplained* behavior in a computer system, all three subdisciplines can readily be identified. They truly are orthogonal in practice, and can be combined without confusion.

Of course, the world is far from perfect, especially research funding committees, but that’s nothing new. Human imperfection aside, the alleged conceptual problem concerning design of PLs doesn’t really exist from my experience of research faculty. Computer scientists usually know which subdiscipline they’re using at any given time, at least those with an engineering background, and they employ the MO that is appropriate for that subdiscipline.

Good essay though, provided much food for thought.

PS. Extending the topic of the essay a bit, while the MO of science is not appropriate when doing engineering, surely the MO of engineering is extremely appropriate. Yet, most software developers treat their MO almost with contempt. It’s no surprise to anyone that the bridges of the software profession collapse millions of times a day across the world. There used to be a term for this, ‘The Software Crisis’. Nowadays the word for it is ‘Normal’.

Well Opensim (Crista mentioned Virtual Worlds, and OpenSimulator is an open source toolkit for VWs in which we share a common interest) is a perfect example of an engineering project, and quite an ambitious one. Nobody would ever suggest though that what they’re doing is science when they’re designing and implementing it, even if their contribution were being done as part of a CompSci research project, because the MO of science — the Scientific Method — is not being used, nor appropriate.

Any science that they might be doing would be using Opensim as a tool (for example, writing simulator modules for 3D visualization of some scientific data), and their design and implementation work is engineering, even if completely original, because it has the purpose of engineering and uses the MO of engineering within their project. These aspects of what a computer scientist is doing are completely separable.

Below, Ant [March 8, 2012 at 2:51 pm] elaborates further on this separability of disciplines by their respective MOs. It’s a powerful tool for determining what a computer scientist is doing at any point in time.

I agree with comments by Richard and Joe that we can still gain
(major) improvements with domain-specific languages as then the language fits
better to the job we are solving. The challenge is then still how to study if a
particular language works better: Companies who develop their in-house
languages often do not have time to do that, albeit some exceptions exist (e.g.
Panasonic, Polar,USAF).

Perhaps one interesting area for the language research (programming
or modeling) is then to study the fit to the task.

1) There is no external incentive
Moore’s law (the hardware industry) took charge of the advancement of computing.
Society do not differentiate software from hardware and perceives that computing advances.
So it does not perceive that software is not advancing.
There is no need of software to advance.
There is no need of the software industry to advance.
There is no need of CS to advance in software.
There is no need of PL research.

2) There is no internal incentive
Academic research is driven by fashion and career advancement.
PL faded as fashion since the 70s.
Academic PL research is middle term to long term research.
Above 5 years, with a decade being normal.
PL is long term.
It is a professional suicide.
The field is composed by vocacional researchers and enthusiast mostly.

3) Interdisciplinary and pure
Interdisciplinary research is fashion and get the funds.
Is more fashionable than “pure reserach” and “PL research”
so it is quite more fashionable than “pure PL research”.
Interdisciplinary funds attract non-CS to relabel their research
projects as computational simulations,
so CS is full of research for the sake of other fields.
The computational non-CS fields advance greatly.
Pure CS advance slowly.
For instance bioinformatics.
In other words, the subdisciplines of CS that advances are
the ones that serves other disciplines.
CS does not have the aim of producing scientific knowledge
of its own discipline for its own purposes.
Sarcastically “CS is just the tech support of the rest of the sciences”.
For instance a PL research project will get funds
for GPU computing support PL features.

5) Academia is conservative
Even with the multicore challenge craving for a paradim shift
PL reserarch groups do not take big risks
and play safe bets with short term projects.
Projects that start from scratch ignoring
pre 90s concurrent PL research.

6) Polishing and Cocktail
PLs are created constantly.
The usual methodology employed by a PL designer
is to take his/her favorite PL and add some features from other PLs.
Essentially it polishes a PL, it completes what is missing.
More knowledgeable PL designers prefer to base their new PL
on many PLs so just put them all in a blender and synthesize a cocktail PL.
The nature of the methodology employed implies that no new
PL paradigm will be created.
The resulting PL will be of the same paradigm of the original PLs.
The features of PLs are memes.

7) Language scale and Paradigm scale
The Academic PL design field can be analyzed at the language level
or at the paradigm level.
So it would be convenient to consider the innovation of PL paradigm research
along the innovation of PL language research posed by Crista.
Imperative paradigm 1842 (Ada Byron, Charles Babbage)
Functional paradigm 1930s (Alonso Church) 1958 (LISP McCarthy)
OO paradigm 1963 (Simula 63 Nygaard Dahl)
Logic paradigm 1972 (Prolog Colmerauer)
Relational paradigm 1972 (Prolog Colmerauer) 1970 (Codd) 1976 (Chen)
The last profoundly new paradigm appeared in 1972 with Prolog,
a language of the logic and relational paradigms.
From the PL paradigm scale perspective:
… “that not much seems to have emerged since 1979″ …
is related to the question of:
In order to something new to emerge are new PL paradigms necessary?
In this case the word paradigm would correspond exactly
to the sense it is used in Thomas Kuhn “The Structure of Scientific Revolutions”
(mentioned previously by Felleisen).
So applying the Kuhn’s perspective to the PL scenario the exploration ended
in the 70s and it will not be unlocked until the next paradigm revolution
in PL research.

8) The scientific method
The study of the scientific method is done by the philosophy of science
or epistemology.
Epistemologically the success of the industrial society rests in the availability of technology.
Technology is produced by engineering.
Epistemologically the discipline of engineering is the solution of problems
by the application of science and mathematics.
Science lets engineers understand and predict their solutions.
Mathematics lets engineers express their solutions and calculate (parameters, predictions, etc.).
Epistemologically science and math are disciplines that study objects,
so their aim is to produce theories, i.e. scientific or mathematical knowledge about
their objects.
The object of study of a science is concrete, i.e. it exists in the natural physical reality.
The object of study of mathematics is abstract. It is a pattern that manifest in the reality.
The nature of the object of study determines the methodology.
The methodology of science is experimental verification.
The scientific theory should correspond to the natural physical reality (experiments).
The methodology of mathematics is proving theorems.

Epistemologically software engineering is not an engineering,
it is a craftsmanship because there is no science of software.
There is no theory of software in CS.
Suppose that a scientific theory of sofware of the kind needed by
software engineers exist, then their would be widespread adoption
and use by the software development community.
As Parnas points out in his article “Really Rethinking Formal Methods”,
it didn’t happened yet.
The rest of the engineerings are successful because they count with the
sciences and mathematics that they need, so they can
understand and predict the behaviour of their systems.
We software developers cannot understand even less predict
the behaviour of our systems and the corpus of knowledge of CS and SE
does not aid in to getting the profession close to the rest of the engineerings.

9) Back to the PL and Crista blog.
Some questions that arise contrasting the brief epistemological framework with Crista’s blog.
Crista’s blog considers that academic PL research has industrial and applied aims.
Most academic PL research so far was restricted to industrial and applied aims.
Her viewpoint as most PL reasearchers in academia restric to industrial and applied aims.
Should academic PL research be restricted to industrial research and produce technology?
Is this restriction what stops PL research of evolving?
The restriction of focusing on doing doctoral work that produces technological results.
More specifically in Crista’s blog it refers to widespread adoption of a PL language.
The success of an industrial PL should be judged by this criteria.
But should a research PL language be judged by the same criteria?
The aim of PL research should be to produce technology
and satisfy the needs of the industry?
or to produce theories of PL, scientific knowledge that lets us
understand PL better and more deeply?

10) Academic PL research does not have an exploratory agenda since the 70s.
The agenda was dictated by the hardware evolution through the
demands of continual adaptation of PL to succesive new generations
of hardware.
So far the continual patching of PLs worked and was enough.
Hardware evolution is still insufficient incentive for further
exploratory PL research.
Exploratory research is conducted marginally as a hobby.
For instance in an academically unrespectable site like
Cat’s Eye Technologies page about esoteric PLs.
PL names like brainfuck or funge will shun many
academic PL researchers.

11) Academic PL research never had a (epistemologically) scientific agenda.
One that is not restricted by industrial or applied aims.
But this is part of a general situation,
that CS research never had a scientific agenda or producing scientific
knowledge about software.
A discipline uninterested in producing a theory of software following
the scientific method will not produce a theory of PL.

Just because CS has the word science in it does not make it
a science in the epistemological sense.
In Crista’s blog the word scientific is used in “scientific evidence”,
but it is not scientific in the epistemological sense.
In Crista’s blog the purpose of a doctoral work is to produce technology.
So the evidence is about technological success, its effectivenes
or efficiency.
Epistemologically a scientific evidence validates experimentally
a scientific theory or some piece of it.

12) In other sciences there is an internal agenda of the discipline
and an external agenda. The internal agenda is to advance the state of scientific
knowledge of the discipline and the external agenda
is to apply its results in benefit of the society.
CS does not have an internal agenda in general.
Specifically in PL research there is no internal agenda.
In Crista’s blog the agenda is external and it is about doctoral proposals
fitting in ths STEM goal.
The lack of internal agenda means that epistemologically
CS does not have scientific goals and CS does not follow the scientific method.
Perhaps it is natural in Crista’s words “that not much seems to have emerged
since 1979 (that’s 30+ years!)”.

In the most mature sciences (epistemologically) like physics
most scientists are devoted to produce or verify scientific knowledge
and a minority to apply it and produce technology.
On the other hand CS is devoted to produce techniques
or technologies but not scientific knowledge (about computers,
or software or PL).
In the theoretical side, theoretical CS is pure mathematics.
They are mathematicians that write theorems and follow
the mathematical method: theoretical CS prove theorems.
So theoretical CS is not producing scientific theories.

On the other hand the most mature sciences and engineering
took centuries to develop. CS and SE have about half a century.
But the regard or disregard of the scientific method by CS
determines it to be a protoscience or pseudoscience
And the regard or disregard of the engineering method by SE
determines it to be a protoengineering or pseudoengineering.

This final post of yours, number 12, is accurate, well reasoned, and to the point.

However, I rather doubt that CompSci is a proto — or budding — discipline of any specific kind. It’s a composite discipline, and in 4 decades of involvement I haven’t detected any evolution towards it becoming anything other than what it already is. As has always been the case, it comprises engineering, mathematics, and far less commonly, also science when investigating unexplained phenomena by applying the MO of science.

Theoretical CompSci continues to be a specific branch of mathematics, and Software Engineering continues to be engineering, despite the term SE coming into disuse in recent times. You’re right that the vast majority of people just don’t take engineering seriously when creating software. Your term ‘pseudoengineering’ is harsh, but accurate.

The saddest part of this for me is that one might sensibly expect computer scientists to have a strong interest in placing their engineering subdiscipline on a more formal footing, but such activity is almost non-existent in the ranks. This has resulted in the standing of computer science professionals being abysmal outside of pure academia, and rightly so because their ability to perform quality engineering has no solid footing in their discipline. CompSci has really missed the boat on this one.

Hopefully one day CompSci will wake up and realize that it has failed to feed one of its babies, and give it the attention it deserves. Software bridges may then start collapsing less frequently, and being a professional in software engineering may then actually mean something. There is no sign of it yet though.

I restricted to present the idealistic perspective.
Fortunately you presented the realistic perspective.
So it can provide a wider and more balanced panorama.

The idealistic position is about what CS & SE should be.
The realistic position is about what CS & SE is actually.

The expectations of society and the scientific and engineering
communities are expressed in the idealistic position.

On the other hand you need to know who you are,
where you are and what are you doing
so the realistic position is necessary as well.
Certainly the discipline in its actual form comprises all M.O.

I hope some day CS & SE gets closer to the rest of the sciences
and engineering but it will take time.
It took centuries for the most mature disciplines to develop
to its current state.

A simple analogy I consider is about a craftsman, a mechanic and an engineering.
A craftsman understand the item he makes.
He understand it enought to make it work.
A mechanic understands an engine so he can diagnose and repair it.
But their understandings are partial and superficial.
A mechanic lacks a complete view of the engine as a system
and its subsystems.
He doesn’t know why each part has the dimension it has.
Nor he has a knowledge about the forces and torques implied.
Nor the mechanics of the fluids and gases involved,
or the combustion process and the thermal dissipation, etc.
On the other hand, the scientific knowledge possessed by an engineer let him understand an engine in a complete and profound way.
A mechanic cannot devise an engine.
An engineer can.
Engineers are happy learning tons of science and math
to make the impossible possible or to improve their creations.

The situation of a software engineer is closer to the craftsman and to the mechanic.
A SE understands the software enough to make it work, like a craftsman.
A SE understands the software enough to debug it like a mechanic.
But this understanding is partial and superficial.
The completeness, breadth and depth of understanding of a system that characterizes mature engineerings are still light years away of SE.
And CS still did not produce the sort of scientific knowledge needed.

The question is what languages have enough depth to build a massive, complex, real-time, distributed and embedded system, complete with any sorts of I/O (and slick user interfaces), and in the process have enough depth to create all of the other language’s to boot. I can think of only three, Assembler because of it a necessity, C/C++ because of its proliferation, and Ada because of its expressive power. Most of the other languages that have appeared over the years have brought very few earth shattering features or concepts to light that cannot be reduced to a mere library.

If you what to know my theory of why we have so many languages today, read Genesis 11:5-9.

“Unfortunately, this argument is the hardest to defend. In fact, I am yet to see the first study that convincingly demonstrates that a programming language, or a certain feature of programming languages, makes software development a more productive process.”

It sounds like a copout. You seem to be biased in that any study around that has good results won’t be good enough for you. Well, let’s make one then. We’ll use 30 people minimum for statistical significance.

Get 30 people that know C and PHP. Tell them to write a web app where the user types in a sentence & the app returns both a list of words in the sentence & the number. Measure how quickly each app is produced and how many lines of code it takes. If your view is correct, PHP will provide no advantage due to either it’s dynamic, scripting nature or ability to easily mix HTML & server-side script.

Experiment 2. Take another dual set of 30. Half will use Java to code an enterprise web app. The other half will use Sun’s DASL language and toolkit. Measure time taken, lines of code, etc. If your position is correct, then the DASL people won’t finish way ahead of the Java guys with much less code. (Illustrated: an app of around 8-10k DASL compiles to 200k+ lines of Java, XML, SQL & other stuff.)

Experiment 3. Have a set of people write an app with certain safety requirements. One group uses C & one group uses SPARK Ada. Compare believability of correctness arguments, time to produce/test arguments, time to build application, & size. Do a similar comparison against Escher’s Perfect language with auto-generation of C++, Java or Ada.

Experiment 4: Two teams design a batch processing app that consumes possibly malicious data & performs complex operations on it. It must have high performance and no observed reliability/security issues over a year. One team uses C++ and one uses Ocaml. Compare the time to produce the app, app size, annual no. of crashes, annual no. of security flaws, and general bug count over the year. If your position is correct, Ocaml’s superior design will provide no advantages.

Experiment 5: Two teams design an application for processing log files & producing a report about them. One team uses Pascal and one uses Perl. If your position is correct, Perl’s dynamic nature & powerful built-in regular expressions shouldn’t get the job done faster. Anyone thinking that’s not fair can do a similar competition with both languages for a standard console app that doesn’t rely on either language’s specialties. Perl developers will still finish first.

Experiment 6: Two teams design a SCM. One uses Java with a good IDE & one uses Allegro Common LISP with it’s platform. Measure time to produce, compile times, lines of code, ease of database integration where needed, and ease of modifying the application. Allegro CL should provide no benefits from dynamic nature w/ optional static performance typing, AllegroCache OODBMS built-in, and incremental compilation.

Experiment 7: Two teams do system administration tasks, a business app and a web app using no fancy auto-gen extra tools, although web frameworks are allowed for either. One team uses C++ and one uses Python. Measure time to completion, lines of code, bugs/crashes over a year, cost of IDE’s and time to train developers to achieve this. I’m betting on Python.

Extreme example: Compare assembler to C/C++ for most apps. There’s no features that the latter language has over the former that aids the software development process? Codasys vs SQL? Prolog vs Mercury? Gypsy vs Coq? Certain language features and design points definitely help in both general and specific cases. It’s beyond obvious. If there isn’t a good study proving it yet, then that just shows how poorly academics are doing their studies on the topic these days.

As a professional mechanic for six years before attending college, your analogies follow the line of “a tool for every job”. Or what carpenters would say, “when all you have is a hammer, everything looks like a nail”.

My job is writing code for deeply embedded products. The kind of stuff where if it works, no one ever knows that it even exists. It is all (with the exception of very few lines of asm) written in C. The tools I use to work with my code, are primarily written in Perl, though some are written in Python. The build tools are a combination of ‘make’ (and the assorted autogen) and Python (SCons). Configuration is through XML. Documentation is through plain vanilla html/css. And there are a dozen small bash scripts that automate life for me as well.

I think that gets to the heart of your comment. If your only tool is PHP, everything looks like a Web Page.

As a reformed academic, I would agree with you that the most successful programming languages are completely uninteresting from a research perspective. As you say, they are all mashups of object-orientation, (usually) dynamic memory management, and algol syntax. We’re still working with a dominant paradigm developed in the 70s.

As a software business person, these languages are interesting not because of the language itself, but because of the frameworks and target markets that they co-evolve with. Ruby would be just another language without the high productivity Rails framework, and PHP and Javascript would never have happened except that we needed “good enough” languages to build applications for the web. Academics tend to dismiss this as “worse is better”, but software business people would rephrase this as “good enough today is better”, and would recognize this as a trivial corollary of the axiom that “time is money.”

So I agree that in order to do credible programming systems research, you would have to accompany it with controlled experiments that showed efficacy in the form of programmer productivity improvements and better runtime performance. Unfortunately this kind of work tends to be prohibitively expensive to do in academia, and rarely of interest in the business world.