James McGovern is an industry thought leader whose focus is on the human aspects of technology around open source, SOA, software security, enterprise architecture and agile software development.

The opinions expressed herein may or may not represent my own personal opinions and definitively do not represent my employer's view in any way...

Sunday, April 09, 2006

Industry Analysts, Ruby and the Fallacy of Productivity (Part One)

Hype is the plague on the house of software. Most software tool and technique improvements account for about a 5 to 35 percent increase in productivity and quality. But at one time or another, most of those same improvements have been claimed by someone to have "order of magnitude" benefits...

The era of breakthrough techniques, the things that Fred Brooks (Of Mythical Man Month fame) referred to as silver bullets, is long since over. Oh, we may have fourth-generation languages ("programming without programmers") and CASE tools ("the automation of programming) and object orientation ("the best way to build software") and Extreme Programming ("the future of the field") and whatever the breakthrough du jour is. But, in spite of the blather surrounding their announcement and advocacy, those things are simply not that dramatically helpful in our ability to build software. And, to paraphrase Brooks himself, the most rational viewpoint to take on breakthroughs is "not now, not ever." Or perhaps, "unlikely ever again."

Communities promoting their wares are simply guilty of hype and will get it twisted whenever someone asked for quantitative research as they cannot provide facts that support their claim but will only respond with assertions that the wrong questions are being asked or will resort to other tactics to rationalize their viewpoint, but we all know that Rationalization is a trap!

Why do we go through this cycle of hype-and-dashed-hopes again and again? It takes two kinds of people to sustain this cycle—the hypesters themselves and the true believers. The hypesters, as it turns out, almost always are nonobjective folks who have something to gain; product sales, high-priced courses, or selling of expensive industry analyst research. 'Twas always thus. Since the days of the Universal Elixir peddlers, there have always been people eager to make a fast buck on promises unsubstantiable by realities.

The ones who worry me, given that there will always be fast-buck pursuers, are those true believers. Why do those folks believe, again and again and again, the promises of the hypesters? Why are we subjected, again and again and again, to massive expenditure and training in new concepts that cannot possibly deliver what is claimed for them?

Nowadays, the folks who truly get practice discipline such as the enterprise architecture community have successfully ignored the hucksters hawking "productivity" by adhering to a set of practices that eschew them. Of course, the attack has to now morph on the attack of a sound fact-driven approach to decision making...

I wonder if the hypesters in the Ruby community would comment on the fact that learning a new tool or technique actually lowers programmer productivity and product quality initially. The eventual benefit is achieved only after this learning curve is overcome. Therefore, it is worth adopting new tools and techniques, but only

(a) if their value is seen realistically (b) if patience is used in measuring benefits.

Tools are the toys of the software developer. They love to learn about new tools, try them out, even procure them. And then something funny happens. The tools seldom get used. Hence my questioning about who is really using Ruby. Yes, they can point to one-off magazine articles and a handful of dot-coms that have mediocre success but in all reality, what are the masses thinking about? Of course we would never want to talk about this because the argument states that if the masses aren't using it, then they must somehow be idiots.

Remember the about the learning curve? The one that says that trying out a new tool or technique, far from immediately improving productivity, actually diminishes it at the outset? Once the thrill of trying out a new tool has worn off, the poor schedule-driven software developer must build real software to real schedules. And, all too often, the developer reverts to what he or she knows best, the same tools always used. Compilers for the well-known programming language they know and love. Debuggers for that language. Their favorite (probably language-independent) text editors. Linkers and loaders that do your bidding almost without your thinking about it. Last year's (or last decade's) configuration management tools. Isn't that enough tools to fill your toolbox? Never mind coverage analyzers or conditional compilers or standards-conformance checkers or whatever the tool du jour might be. They might be fun to play with, these developers say to themselves, but they're a drag when it comes to being productive.