I'm learning a lot of things in the last little while. Here's a fun set of facts:

1. The more customers you have, the more problems they'll find. Feature request lists grow boundlessly, and you'll never finish them all. Accept imperfection, and aim to solve the most important problems first.

2. Nobody will use your product unless you eliminate 100% of their reasons not to use it. Since solving 90% of problems is not the same as solving 100% of the problems for 90% of people, a partial solution isn't acceptable. Aim for perfection.

(Doesn't this sound like the two sides of the argument in Worse is Better?)

Think of those two points by themselves; you can probably agree with each one. Each one suggests a course of action to respond to the problem. But put together, no solution seems to make sense: nobody will use your product until you finish going through the entire requirements list, but the list of things people ask for will expand endlessly. The only sensible course of action suggested by each one is nullified by the problem suggested by the other.

I realize it's generally social suicide to quote Ayn Rand, lest you be labeled a crackpot, but sometimes she said smart things:

Contradictions do not exist. Whenever you think you are facing a contradiction, check your premises. You will find that one of them is wrong.-- Ayn Rand, Atlas Shrugged

The wrong premise in this case? Maybe it's obvious to you now, but it wasn't obvious to me. To solve the problem, you need to redefine the problem itself. One way is to find an important subset of people, then solve all their problems. In other words, you have to find a way to solve both #1 and #2 at once.

Contradictions do not exist. It's good advice. When you have a contradiction, you can't possibly succeed until you figure out what it is, because the real solution is neither of the obvious ones.

Programmers know that programming efficiency comes largely from getting into The Zone for a few consecutive hours - and those few hours can be 5x-10x as productive as "normal" working hours.

Well, for a change of scenery (and because it was necessary to get my job done), I relocated myself temporarily from Montreal to our Toronto office for the last few weeks. This place is not an environment conducive to getting into The Zone in any normal sense. It's not an R&D department. It's a business environment, where people have daily problems, and those problems all seem important, and they really want to solve those problems right now.

Luckily, my job in the last few weeks has mostly not been programming; it's been doing a bunch of random business supporting tasks for a new product we're working on, where the product largely consists of repackaging our existing technology into a newer, more "customer focussed" package. To do that, I had to learn a bit about marketing, a bit about customer business models, a bit about project management, a bit about salespeople, a bit about financial projection, a bit about technical support, a bit about manufacturing, a bit about "webinars", a rather scary amount about IBM DB2, and lots of other things. Then, in real time, I had to turn around this new knowledge and apply it to my work. And I had to do all that in a very short time.

So I dedicated every waking moment to learning and doing that stuff. When I wasn't working, I was reading books about it. When I wasn't doing that, I was eating while thinking about it. When I wasn't doing that, I was dreaming about it, and sometimes I would wake up with the right answer.

Whether I did a good job or not in the end isn't really the point, but the process itself is really interesting: you can deliberately focus your thought processes on a specific set of complex, confusing, interrelated things, and tune out everything else. When you do, you're doing something very similar to programming in The Zone. The magic in programming happens because when you hold all the concepts in your head at once, you can just see the answer clearly; the right answer is the only thing that fits all the pieces at the same time. It turns out that other kinds of work can be done the same way.

The result is that I know and understand a lot more things than I did a month ago. I understand a few things that even the people teaching me didn't understand, because they weren't in The Zone at the time: they didn't see the whole picture.

The problem is, once you've seen the big picture, you'll never not see the big picture. It's very hard to go back and sit in your pigeonhole.

At work we summarize our general office policy in one rule: "Don't do stupid stuff." This is an amazingly powerful, simple, and all-encompassing rule. Recently some people have asked to have more specific policies written down so that it's easier to determine if they're in compliance. In fact, if you're working for a company, this is the wrong thing for you to want. Why? It's simple.

Because explicit company policies exist so that it's easier for the company to fire you.

There. I said it. End of rationale. Think about it: if you don't follow clear, explicit company policy, it's obvious that you're an uncooperative employee who's not a team player and can't play by the rules. And by the way, ignorance of the law is no excuse.

By extension, you'd think that simply following the company policy, then, will keep you out of trouble and prevent you from getting fired. In many companies, that's absolutely true: which is why you have so much dead weight. Ironically, some unions (the ones who are prevented from going on strike because they're an essential service, or who don't want to go on full strike and thus sacrifice their income) will decide to simply do absolutely the bare minimum amount of work in order to comply with the rules. The result is that useful work grinds essentially to a halt, because most useful work isn't actually covered by company policy, and will never be.

Well, I don't buy it. Don't do stupid stuff. But how do you know if you're doing stupid stuff? Well, here are a couple of steps that help me a lot:

1. If you don't know whether what you're about to do is stupid, it would be very stupid to proceed without first figuring that out.

2. If you think someone else is being stupid, do everything in your power to make them stop. Doing otherwise would make you an Accessory to Stupidity. If the person in question resists all reasonable attempts to educate them, they have a serious problem that will block their career: chronic stupidity. Chronic stupidity, being very vague, is not good enough grounds for firing someone. But lack of productivity is, and chronic stupidity always leads to low productivity; that's what makes it stupid.

Another problem arises when someone in a higher position of authority than you orders you to do something stupid. Now you have two choices: do what you're told, which would be stupid; or don't do what you're told, which could get you fired, which might also be stupid. (For example, if you can do more good by staying with the company and doing this one stupid thing than by leaving entirely, it would seem stupid to disobey.) This is a typical moral dilemma, and other than advising you to do the "less stupid" of the two choices, I won't try to answer that question. But I can avoid the question by asking another one: Why is that person with authority telling you to do something stupid in the first place? There are two possible reasons: either the person is doing something stupid, or the person knows something you don't. If you're not sure which one it is, you need to go back to step 1 (First, figure it out), then make your decision. And if it turns out that person is doing something stupid, you need to move on to step 2 (Tell them).

In engineering, we call this process Root Cause Analysis. You might have to go back several steps to find the basic, underlying cause of the stupidity. Rumour has it that Toyota requires employees to ask at least five levels of "Why?" when trying to find the root cause of any problem; sometimes it lies deeper than you think.

So that's the bargain I make with people who work for me. I don't make strict rules; do the Root Cause Analysis and you'll see why that's doing you a favour. But in return, I expect you to think for yourself and do what makes sense. You'll have to explain yourself in case it looks to me like you're doing something stupid... and I promise to do the same in return.

Incidentally, doing things this way in some other companies could happily get you fired. But you can avoid that pitfall, the whole company will be better off if you completely ignore their policies and avoid stupidity at all costs.

Aside on Governments and Laws

Incidentally, "ignorance of the law is no excuse" is a comment usually applied to governments, and often ironically in view of the probably tens (or hundreds) of thousands of pages of laws that apply to you at this very moment that you don't know about and might very well be violating. The essay above doesn't directly apply to governments, even though there are definite parallels. The similarity is: in my opinion, explicit laws exist mainly so that we can throw people in jail. But actual civilization is based on much more than just laws: the vast majority of us would do perfectly sensible things even if there were no laws against it. The parallel to "Don't do stupid stuff," for societies is, "Do unto others as you would have them do unto you."

Where the analogy breaks apart is: in a company, we really can kick you out just because your end result - your productivity - is obviously too low. We don't kick you out just because you did something stupid (and we can't), but because your stupidity led to bad consequences. And being kicked out isn't really that serious; there are lots of other companies where you can try again. In society, however, we can't kick you out at all. We can throw you in jail or kill you, I guess, but that's far too serious for most violations. So we have to have a very, very clear system that won't punish you unless it's totally obvious that you did something totally wrong. It's not very efficient, and we depend much more on the rules we didn't write down, but it's a working compromise.

The bargain you make with your lawmakers is: Ignorance is no excuse, but in return, the laws must all only be specific applications of the most basic rules of society, like "do unto others."

Aside on Piracy

And that's why in Canada you're allowed to copy music for your friends. "Do unto others" is the basic rule, so our government can't morally make specific laws that obviously violate it. (Not to say they won't try, but so far, so good.)

Many people do one or two of these things. The real magic happens if you can align all three of them.

(Disclaimer: my neural net spat this out a couple of days ago. It sounds like something I would have heard somewhere else. If I'm plagiarizing it from someone, my apologies in advance, but congratulations, I guess I think you're smart.)

The answer is, from just the context of the article, you don't know. The problem is that the author suffers badly from what's called the selection bias: wanting to believe something, and then examining only the evidence that supports your belief.

I learned about the term "selection bias" by reading Harvard Business Review, a really excellent magazine, even if you think (like I do) that most businesspeople are stupid. Ironically, I can use HBR as a counterexample to Graham's point about "blog knowledge" being more interesting than stuff published by traditional companies. Meaning no offense to Paul Graham, nearly every article in every issue of HBR is better researched and more instructive than any of Graham's essays that I've read. Why? I don't know, but if you follow my link, you're going to have to pay to read those articles, and Mr. Graham gives away his knowledge for free.

Has anyone ever told you about selection bias before? If not, then like me, you're going to make lots of stupid decisions. Browsing the Internet didn't get me that information. Paying HBR some money did.

So, what specifically in this article suffers from selection bias? Nearly every statistic and every fact in the entire article. Not to say that he's wrong... but the proofs are all invalid.

For example: Open Source produces better software, eg. Linux and Firefox. The evidence? People - maybe more than 50%, depending who you ask - are "switching to them" from their proprietary counterparts. That's pretty convincing, right? Not really. First of all, they haven't switched yet. The proprietary software still has the vast majority of the market. Secondly, Linux is obviously inferior to Windows for certain types of uses, ie. anything a small business might want. (There's no way a small business could figure out how to set up a Linux system to do, say, what Microsoft Exchange does out of the box.) And Firefox? Okay, maybe it's better, although I find it horrendously slow, and by the way, Microsoft hasn't released a new version of IE in something like three or four years. (Longer, if you consider the fact that nobody could really tell the difference between IE 4 and IE 6 just by looking.) Firefox has had a lot of time to work on becoming just barely comparable to IE. But, even though Microsoft just politely let them catch up for a few years (something I claim was their actual, smart business strategy!), Firefox is just now becoming barely better than IE. But yes, some people are switching to Firefox. Congratulations.

You can prove anything you want with statistics. Apache is the best web server, yes. Linux is the best OS platform, maybe. Firefox is the best web browser at the moment, quite possibly. But three examples doesn't prove anything; what's the best graphics package, word processor, music library, media player, game platform, game, or accounting package? The answer to almost all those questions isn't in Open Source. Now, by skipping some examples I didn't like, I just used selection bias on purpose to prove my point. Did you notice? And do you see why, after this whole rant, we still don't know if Paul Graham is right or not?

I think the worst job in the world would be to be a full-time art critic. Cursed with the ability to understand what makes something great, but not the ability to make such great things yourself, you're forced to look at other people's inferior work and try to explain what's wrong with it. How depressing.

On the other hand, if you have no idea what you're doing - as in, for example, my case with most kinds of art - it's much easier to appreciate. "This person can do something I could never do." There's something admirable in that. It makes you feel a bit safer about the world, knowing that even if you can't solve all the problems, at least there's probably someone around who can fill in the blanks.

This is where I could put in a bit about innocence lost and the fact that happiness enabled by ignorance is always unsustainable, but that would kind of ruin my point, so I'll leave it out.

Yesterday I and my friends were looking for the site of the Canada Day fireworks (see if you can find them by browsing the web site!). We were initially having some rather bad luck, but were getting reassured by meeting with occasional people who would come to us and ask, "Is this where you can go to find the fireworks?" Being asked the question so many times, of course, implied that there were many people who at least had some idea that this was the right place.

But since we didn't know where we were going either, it eventually became more clear: we asked a few other people for directions, and they gave the same answer as us: "I hope so!" But people were walking in some direction or another. At one point we actually caught ourselves following a group of people who, five minutes earlier, had asked us whether this was the right place. Some consideration led us to this important life observation:

If nobody around you knows where they're going, the people who know where they're going are somewhere else.

Upon realizing this, we departed quickly in search of people who knew where they were going. Happily, we found the site in our second-guess location of where the fireworks ought to be, right before the show actually started, so it all worked out for the best, and with a free philosophy lesson to boot.

(For the morbidly curious: you should know that the Quai Jacques-Cartier is apparently not the same as the Pont Jacques-Cartier. Perversely, the fireworks last night were at the Quai, but the fireworks tonight are at the Pont.)