Does Leadership Matter?

Below is an essay I wrote for a software engineering course taught by Professor Terry Woodings. It’s already dated, in the sense that since I wrote it, Steve Jobs has died.
I’ve mentioned it a few times on forums such as Hacker News and garnered some interest, so for my own convenience I’ve taken the original and adapted it to HTML. I’ve removed some irrelevant footnotes and turned most of the references into hyperlinks.

Introduction

There is a tide in the affairs of men
Which, taken at the flood, leads on to fortune…
Julius Caesar, Act IV, Scene 3.1 [fn1]

From an ancient greek philosopher musing about the perfect Republic, to a Rennaissance diplomat giving advice to Princes, through to the Bard’s character studies, leadership has occupied a central place of honour in great literature throughout history.

Software, being a human endeavour, lends itself to discussions of leadership also. What was true of Julius Caesar might be true of Steve Jobs. What was true of Archimedes might be true of John Carmack. We mine our history for parallels to the present day; we draw from old heroes lessons of heroism.

But does leadership make any difference? Or can these outcomes be explained in other ways?

The central thrust of this essay will be to explore the hypothesis that plain luck is as important a factor in leadership as any other traditionally-listed quality. Mere chance can govern the leader’s environment, the leader’s team mates and the leader’s task. Mere chance can be the determinant of success.

Put another way, I will propose that the qualities we see as leadership are necessary but not sufficient elements of the achievements of famous leaders, both in general and the software industry particularly. That we are sometimes confounded by leaders who followed a ‘recipe’ and still failed will be seen as an example of survivorship bias.

Necessities

Before we can address the question of mere luck, it’s necessary to study the question of what precedes luck. What qualities are required to lead a successful technology company, and in particular a software-dominant company?

Technologists

A common thread that can be discerned throughout the history of software is successful companies being founded and co-founded by technologists. Apple was founded part-founded by Steve Wozniak, Microsoft part-founded by Bill Gates, Google by two PhD students, Yahoo! by two electrical engineering students, SAS by a group of programmers, Facebook by a hobby programmer and so on.

So our first predictor of success is technological aptitude. Software is in particular different enough from conventional lines of business that it will frustrate the efforts of a conventional manager. An MBA student studies factories, retailers and hourly service firms. None of these models maps neatly to software development (see “Make a Cheeseburger, Sell a Cheeseburger” in PeopleWare 2nd Edition by DeMarco and Lister).

In contrast, a technologist will have some intuition for what is possible. This allows them to both dismiss the impractical and to embrace possibilities that non-technologists might not, as yet, have deduced.

Vision

Some of the most famous technology companies in the world have simple but all-encompassing visions. Microsoft under Gates and Allen saw its mission as putting a computer on every desk and in every home in the world. Steve Jobs set Apple’s mission as producing “insanely great” computers. Google sets out to organise all the world’s information.

Embracing a large vision requires a certain amount of immodesty and immunity to ridicule. It is not enough to mouth the words; the visionary sincerely believes that the vision is possible and that they, the visionary, will make it happen. This quality is closely related to boldness.

Boldness

Fortes fortuna adiuvat.

“Fortune favours the bold”. This saying, probably derived from a play by the Roman playwright Terence, neatly illustrates the relationship between luck and boldness.

The handmaiden of vision is the boldness both to entertain a grand vision and to carry it out. Visions like those of Bill Gates and Steve Jobs are hubristic. But without the self-confidence of their leaders, Apple, Google, Microsoft and the like would have been footnotes in the history of modern technology.

Luck

I returned, and saw under the sun,
that the race is not to the swift,
nor the battle to the strong,
neither yet bread to the wise,
nor yet riches to men of understanding,
nor yet favor to men of skill;
but time and chance happens to them all.
Ecclesiastes 9:11

It was not enough for Gates, Jobs, Brin, Zuckerberg et al to be bold, visionary technologists. They also had to be fortunate.

Firstly, timing matters. Gates and Jobs began their work in the formative days of personal computing. The market for personal computers didn’t yet exist; therefore the scope of their ambitions could encompass the entire world. By contrast, companies entering into the crowded web and mobile applications markets today often have very narrow missions. There is still a need for the qualities outlined above, but the richest spoils of the fundamental revolutions in technology have been apportioned to those who had the luck to be on the US West Coast in the 1980s and 1990s.

It is possible to be too early with a technology. We sometimes say of technologies that failed that they were “ahead of their time”. A critical component in luck is to bring a technology to market in the right way at the right time.

When Mark Zuckerberg began working on Facebook, social networking was an established concept. LiveJournal, Friendster and MySpace had already been operating for several years. Furthermore, while Zuckerberg is an unusual individual, there are probably hundreds of individuals with his mix of education, connections, intelligence, experience and boldness. So why did Facebook succeed while pioneers of social networking foundered, faded or failed?

Lucky timing, in part. Facebook, a refinement of the social networking concept, struck at a time when constant engagement with the internet was becoming the norm amongst incoming university students. Before long Facebook had an almost total penetration of the US university system; before long it spread from there to become the most successful company of its kind.

Secondly, boldness to seize the opportunities that arise at random. The most important moment in the history of Microsoft came in 1980 when IBM told Microsoft that it was looking for a BASIC interpreter — and an operating system — for its upcoming PC project. Microsoft seized the opportunity to create the operating system but retained the copyrights. To do so required cunning but also boldness: the boldness to both bite at the IBM opportunity but also to be prepared to undermine them in future.

But boldness by itself is not enough, an unconstrained boldness becomes foolhardiness. Around the bright stars of successful leaders and companies there is far more dark matter composed of the bold and unfortunate. Luck is still the deciding factor.

Survivorship Bias

So how is it that mere chance does not get top billing in any discussion of successful leaders? The most likely answer is survivorship bias. We judge history based on examples and records that have reached us from posterity. But then we are biased what has managed to survive the passage of time.

Take the complaint that “they don’t make things like they used to”. Perhaps the complainant is referring to a particularly sturdy antique chair. How can it be that chairs made in the 1800s have lasted for centuries, but a $30 chair won’t last for 2 years?

The mistake here is to generalise from this one, exceptional chair, to all chairs of that vintage. If chairs from the 1800s were “made like they used to”, then in fact the world would be awash with such chairs.

By only studying successful leaders and trying to infer common qualities and patterns, we run the risk of introducing survivorship bias. Unless we also study abject failures, we cannot determine whether or not any given quality is necessary, sufficient or merely coincidental. We have no control group: the evidentiary standard is too low to draw any scientific conclusion.

It has been said that “History is written by the victors” (traditionally attributed to Winston Churchill). In the case of leadership studies, history has been written about the victors. Very little ink has been spent illuminating the much larger pool of failures. Those failures who do receive attention were at first victors. Nobody writes about Commodore International merely because it failed, but because it thrived first. We seek out Rise and Fall, not Fall and Stay Fallen.

Measuring Greatness

The other problem with determining the properties of a great leader is to identify them in the first place. We must define how we know they’re “great”. This too introduces problems, because any assessment must be based in time.

In studying the great leaders of history, and especially antiquity, this is not a problem. The details of their life have been exhaustively canvassed, the consequences long since unfolded into the tapestry of history. But in software we are mostly studying leaders who are still alive, who can be expected to remain alive for some time yet, who are still shaping the world we live and work in.

So any selection of “greatness” must deal with the now. Today Steve Jobs is admired as a brilliant leader of Apple; but it was not so long ago that Apple was considered to be doomed. Did his qualities as a leader change in that time? Probably not. We took the revenues, profits and market capitalisation of Apple under Jobs as a proxy for his greatness. We studied outputs to choose the leaders and only then did we ask what the qualities of leadership were.

We can compare contemporary leaders with historical leaders, but we must be careful. When looking at Julius Caesar we can integrate his life and the centuries that followed; but for a living leader we can only take a particular slice of time. There will ups and downs and the sum of their efforts and achievements will not be known for some time.

This may seem like a very fine distinction, worthy of angels dancing on a pinhead. But it matters because it means our survivorship bias will whip around almost from year to year. We exalt this year’s winners and excoriate the losers, but who will sing the praises of the steady improver? We won’t know for some time — if ever [fn2].

When luck plays its role and elevates some obscure software company into the spotlight, do they suddenly have better leaders? Is there something about the leaders of that company that made it destined to succeed? Of course not. So why do we measure leadership greatness purely according to outcomes, which are so heavily influenced by luck and timing?

What About Process?

Perhaps companies become elevated because they crossed their Ts and dotted their Is. Where then does leadership end and process begin? Can luck be tamed by process?
Suppose we line up Microsoft, Google and Facebook on a spectrum of adherence to process; do we find some dependable gradient of successfulness? No. Instead we find very different lessons of process.

Many parts of Microsoft, including its Windows division, have followed something like a iterative waterfall process for decades. But when that process led to the expensive delays and poor results of the Vista project, Microsoft adopted a heavily revised process for the more successful Windows 7 process, but the concepts of working to a plan, assigning tasks centrally and so on remained in force.

Then contrast Google: “From a high level, Google’s process probably does look like chaos to someone from a more traditional software development company”. Yet Google relies heavily on automation, scalable tools and code reviews to ensure that the code they release into production is of as high a quality as possible.

Perhaps most radical of all is Facebook, where engineers can launch entire features, live, at any time. From Google they borrow a free-ranging engineering-led culture, but with almost zero central coordination. “[E]ngineers can modify specs mid-process, re-order work projects, and inject new feature ideas anytime”.

Microsoft and Google are diversified companies making billions in profits; Facebook is generally regarded as a near-future cash fountain. Yet their engineering processes have few things in common. More divides Google and Microsoft, for example, than unites them. Yet all three companies have had episodes of substantial luck.

Beyond a certain point process has ceased to be the predictor of success; at best it is a constraint on growth. Leadership here has been about creating the conditions to embrace change and dynamism, but luck has still played the central role in the very existence of these companies.

Conclusion

Cause and effect are easily distinguished, when they occur in succession; but are often confounded, when the operation is continuous and simultaneous.
— Jean-Baptiste Say

Are we then doomed to mere wandering? To casting our die? To muttering deo volente every time we sit down to write code or launch a business venture?

It is the impenetrable complexity of humans, of human systems, and the history of those systems, that drives software engineering. In history there have been two great technological revolutions: the industrial revolution, where for the first time musclepower was replaced by machinery, and now the information revolution, where brainpower is being replaced by machinery. The consequences of the latter revolution have some time yet to run. Just as conventional engineering was born in the crucible of steam and iron, so too will software engineering emerge from software itself.

Variance and fortune arise from the systems of the world. They are often unpredictable, given our level of understanding. Sometimes we can deduce causes in retrospect, but other times we will not. We will ascribe some outcomes to “luck”, simply because we are unable to deduce causes.

This is deeply unsatisfying. Leadership must not merely be about relying on luck, but about creating luck, about preparedness to embrace fortune when it arrives. When in 1995 Microsoft performed a radical about-face on the issue of Internet connectivity, they demonstrated that the wise leader sometimes senses the tide of history and rides it on to fortune. Microsoft foresaw misfortune and manoeuvred to avoid the shallows.

Without boldness, without vision and without technologists at the helm, Microsoft might not have made that change. Today they might have faded into history like DEC or Sun or Wang Laboratories or hundreds of other technology companies whose time has passed. In actual fact, luck, by itself, was not enough.

Microsoft had luck, and Google, and Facebook; and a lot of their luck has been extrinsic. But the rest they made themselves. And that, perhaps, is the most fortunate lesson of all.

Footnotes

fn1: About the quotations. As I grow older, various things I once read and considered trite come back and strike me with new forcefulness. In writing this essay, many such quotes came back to my recollection. I have included them both to illuminate my points and to show off my thin erudition. I sincerely hope I will not be mistaken for one of those lazy students who merely hit a quotations book in order to spice up a dull essay.

fn2: Note for contrast the vast business literature on Toyota’s rigorous self-improvement focus. But until Toyota was successful, who cared what they did?

2 Responses to Does Leadership Matter?

I think one point you are missing, is that most of the leaders you have mentioned, have ‘created’ the future (that we all live in now). It could be their vision, or a happy confluence of things. but the fact that they did dream it up and made it happen – that is the part you are missing. Your perspective takes a tone that paints these folks as just being there at the right place. They were, but they used that better than everyone else did.

They are not unique in trying to create the future; lots of others were trying to do the same. By necessary assortment, some will be considered “winners” and others “losers”. The difficulty is in distinguishing how much of that is from directed effort (leadership) and how much could be expected merely as chance. I think it is human nature to drastically overestimate the contribution of self-direction and to underestimate mere chance.