Our Polyglot Nightmare

0

The diversity of human languages has been on the decline for decades. With the rise of globalized communications and business, the world has accelerated its shedding of unpopular languages, as speakers increasingly focus their efforts on those that afford them the most economic and social opportunities. The loss of these languages has even spawned efforts like the Endangered Languages Project to attempt to save them from extinction.

Those same fundamental forces have been even stronger in the startup community, where teams regularly work across the world’s time zones. Yet, programming languages have not met the same fate as their human language cousins. Quite the contrary, there has been something of a golden age these past few years for programming languages moving out of theory and into practice, with a profusion of new and newly popular ones that are finding uses across startups and large companies alike.

It’s always exciting to see the construction of new tools for programmers, which can potentially increase productivity and give us the products we want, faster. But that linguistic diversity also has a darker side for engineering teams, which must work with multiple languages just to launch their product, let alone maintain it.

To get a sense of what I mean, consider a typical startup looking to deliver an application for users on web and mobile. For the frontend, the developers will use HTML, CSS, and JavaScript at a minimum, and potentially more languages depending on the exact libraries chosen. For the Android and iPhone apps, the team would have to use the native languages of those two platforms, namely Java and Objective-C (and now, of course, Apple’s new language Swift). And to connect all of this, the team would have to build a backend, which probably involves another language like Python, Ruby, or Go, as well as database query languages.

Much as the rise of “big science” in the 20th century transformed the solitary scientist into a research team, the growing complexity of our development environment is requiring us to build larger engineering teams with more individual specialists. That specialization has important ramifications for the job market, particularly in a market as constrained as Silicon Valley engineers. It also means we are fragmenting our development efforts, potentially retarding the evolution of our libraries and toolkits. Perhaps most ominously though, it limits the contributions of indie developers and small startup teams, who can’t write for certain platforms due to limit of their own talents.

While that’s a lot to be concerned about (and I’ll talk more about this later), diversity in programming languages is not necessarily all negative. Obviously, there are diverse uses for code, whether it be for embedded systems, highly reliable systems like air-traffic control, or simply test environments for new startup ideas. Using the right tool for the job can certainly improve productivity.

But source code serves two purposes. It allows one engineer to explain to another engineer what a program should do, and it also allows an engineer to dictate to a computer how it should operate. The greater the profusion of languages, the tougher the first type of communication is.

Even so, there are three main underlying causes for the widening fragmentation in our programming environments. First, the adoption speed of new languages has accelerated. This is largely caused by the Internet itself, which has allowed developers to more rapidly spread awareness of new programming languages through better tutorials, references, and code examples.

In addition, tools like GitHub have encouraged more engineers to get involved in library development, which can make a new language viable much faster than in the past. Corporate sponsorship for new languages is strong, whether from Google with Dart and Go or Apple with Swift. Finally, engineers seem more willing to pick up new languages than in the past. One wonders whether Paul Graham’s call for startups to choose exotic languages to find better engineers has pushed people to use lesser-known ones.

While faster adoption ensures that powerful tools become widely available, it poses a real challenge to engineers, who must accelerate their education in order to stay current. Mastery of a particular language is now harder, since polyglots have to divide their time across multiple environments.

Just take Apple’s Swift, which is likely to become the de facto standard for iOS development. While knowledge gained from Objective-C won’t entirely be thrown out, every iOS engineer has to learn new idioms to fully take advantage of Swift.

The second main cause is the need to develop on multiple platforms. Developers in the tech industry are now programming for the web, mobile devices, cloud infrastructure, multicore processors, and wearables among other examples. Each of these platforms has unique performance characteristics and requirements, which means that one language may have a vastly different utility compared to another. For instance, languages like Go and Erlang are useful in a server cluster where concurrency is important, but would probably have less value on a wearable, where processing power may be concentrated on a single chip.

Third, this diversity in platforms is matched by the variety of language paradigms vying for the attention of developers. Functional programming languages, which have classically been something of a curiosity of academics and some dedicated enthusiasts, have now become more widely popular through languages like Scala and Haskell, in large part due to greater recognition of their value for handling web applications.

Given these underlying causes, it seems unlikely that the polyglot nightmare is going away anytime soon.

The costs, though, are steep. The challenge is not just that the number of languages is increasing, but that the number of combinations of environments is becoming bewildering. As companies begin to use their own unique set of technologies, finding the right engineer with exactly the needed skills is tough, and engineers have to spend more time generalizing their skills in order to stay on the frontier of the discipline. Sadly, startups rarely want to provide the resources required to train a new hire, instead expecting programmers to be productive from day one.

More broadly within the programming community, our increasing fragmentation also means that we are dividing our efforts on building up existing libraries. Our programming time is limited, particularly on the open-source projects that form the bedrock of any usable language. If we spread our efforts more thinly across projects, we risk losing the productivity advantage of having multiple languages in the first place.

Ultimately, we are witnessing a collective action problem that is, frankly, impossible to fix. While there may be no “solution”, startups can avoid the polyglot issue by simply being deeply cognizant of exactly what languages they are committing resources to. Engineering managers should be incredibly cautious about introducing new languages into existing infrastructure, and also doubly cautious about moving from an existing popular language for one more on the frontier. Sometimes a performance hit may be worth a bit more cohesion on a team.

As we continue to ask how engineering can become more inclusive, one way is to ensure that the knowledge required to perform well in the industry is as minimal as possible. While being a polyglot is certainly a requirement of today’s software developers, we can all do our part to ensure that skill requirements are not increasing out of proportion for what our engineering talent can provide. No engineer should have to be a UN interpreter just to build their dream.