Saturday, October 07, 2006

Egomania Itself

Q. Is it your testimony that it's perfectly okay for GNC to put products on its shelves, to sell pills to people if those pills have no benefit?A. We make the assumption that they make the decision based on some perceived benefit. — GNC board chairman Jerry D. Horn, in sworn deposition

Whew! What a crazy ride the last 2 weeks have been: I was Joel'd, Bray'd, Slashdotted, featured in Agile Mafia Monthly, and completely buried in an avalanche of email from Alert Readers.

The highlight of it all was probably the invitation I received to speak at an Agile conference in Vancouver, B.C. They told me, and this is a direct quote, "we promise not to cut your balls." They then went on to reassure me that just to be sure, they'd get me a security guard. Yikes!

No doubt the Agilawyers have been scrutinizing my Tyler Durden clause, seeing if they can find a loophole and take me up on it early. Ha! I won't be taken so easily. "A fool and his balls are soon parted," as the old saying goes. More or less. So I declined politely, filed for a legal name change, got on the list for facial reconstructive surgery, and moved to Oklahoma. Can't be too careful when your you-know-whats are at stake.

The second runner-up was definitely this email, which I've reproduced in full:

From: Agilists Carnival

If you've received this email, you've been referenced in theOctober 5th Carnival of the Agilists. Thank you for yourinsight and commentary on how to make Agile softwaredevelopment as good as it can be.

If you like, please consider linking to the carnival fromyour blog, to help spread the word. And, as always, if youwould like to help out with the Carnival, either as arotating host, or as a one-time guest host, please feel freeto contact me at: <email withheld> and let me know.

Thanks again! J.

Hee hee. Looks like someone didn't get the memo.

Anyway, because my last blog entry was soooo read and skimmed and linked and contested and grokked and misunderstood and praised and denounced and everything, I figured I'd break tradition, buckle down, and actually write a follow-up.

Incidentally, I can't even begin to tell you how happy it makes me to break all the stupid-ass grammatical rules my Agile Grammar Teachers forced on me in grade school. They represent a brittle, method{olog}ical personality type that's still alive and thriving today, busy certifying scrum masters around the world, painting their cats, all that good stuff. [We'll get to the cats in a little while, in case you were wondering.]

OK, enough preambling already. On to the main show! I promise you many mean jokes in tonight's entertainment, an insight or two, and if you're a good audience, I've got a bona-fide magic trick for you at the end. Hang in there!

There's More Than One Good Kind

Before we get into the interesting stuff, I thought I'd clear up one major misconception that kept coming up.

Some people apparently thought that by using Google's development practices as a counterexample, I was implying that Google's approach is the only alternative to (a) Agile, (b) Waterfall, or (c) Nyarlathotep.

Not true! I was merely using Google's process to illustrate that alternatives do exist and they do work. I offered this evidence largely to combat the Agile claim — and I'm paraphrasing loosely here — that "if you don't forward this Agile Manifesto to 5 friends within 47 seconds, you will die instantly and then be thrown off a high building into a pile of manure."

Seriously, though: you don't need to look at Google to discuss life beyond Agile. If you just look around, you'll find plenty of "success stories" from non-Agile teams. Looking for "success stories" is, of course, stupid; it's exactly the kind of selective reinforcement that can lead to (e.g.) chronic gambling disease. If you want real supporting evidence, you should construct the scientific kind, using experiments. Yes, I'm afraid that means looking at the failures too. Ouch! So bad for marketing. Whenever you hear Agile people asking around for "success stories", remind them politely that only looking at the positives is pseudoscience.

If you walk around and do a survey for yourself, you'll find two dimensions producing four categories: Agile vs. non-Agile, failure vs. success. You'll be able to find projects that fit into each category. Don't focus on the "success stories" — look at the whole picture, and correlate your findings with those of friends from other companies. Most of you will find that if you did an honest assessment, Agile's not very widespread, and it's not as success-prone as they'd have you believe. Not by a long shot.

As one concrete example, one commenter observed that Blizzard doesn't ship on a schedule; they ship "when the app is done". Well, raise your hand if you haven't heard of World of Warcraft. Hands? Anyone? Didn't think so. WoW broke into the crowded and locked-in gaming space (people like to stay on their current MMORPG), and stomped the living crap out of huge competitors like Sony and Microsoft. Overnight. Boom. And Blizzard is in a business that's as about different from Google's as you can get — especially their shrink-wrap games, which have also historically been groundbreaking, award-winning, runaway bestsellers.

But rather than enumerating all the non-Agile teams and companies as special cases, let's get straight to the rule: Most great software developers around the world don't use Agile. They just work hard, they stay lightweight, and they ship great stuff. Most developers still have no idea what Agile even is. Think of that!

Agile is a niche, a market minority, almost an aberration, really. It just happens to have a lot of marketing, because it opened up a hitherto entirely untapped new market for snake oil in the tech sector. Consultants are making money hand over fist by extending their contracts with credulous clients who are told they don't have the process "quite right" yet, but they're almost there!

Agile wouldn't be a big deal, except the Agile camp is really loud. Annoyingly so. Loud enough to start interfering with regular developers' work. Which is why I had to speak up. Agile was the Mystery Topic that gave me Blogger's Block for nearly 2 months. At some point, though, I just couldn't take it anymore. There were critics here and there, sure, but none of them were as loud as the Agile folks.

So I yelled as loud as I could: loud enough to make it onto Slashdot, even! Which is exactly what I wanted. I only had one high-level takeaway, one marketing message I wanted to get out to developers everywhere: It's OK to say No to Agile. That's it. Nothing more complicated than that. But the Church of Agile was getting so powerful that it was becoming increasingly acceptable to criticize people in the workplace for not being Agile. More on that in a bit.

You've probably noticed by now that the Agile folks are utterly lacking in humor. After my last blog, the whole Agile community started maneuvering and positioning and strategizing — and I'm talking about the smarter ones. The others just screamed "I lost thirty-five pounds on that diet!", demonstrating that they might possibly have missed my point about experiments vs. statistically meaningless anecdotes. Can't blame 'em; it was a lot of information to process: a whole book squeezed into a chapter-sized blog. But across the board, there hasn't been a whole lot of self-deprecating humor on the Agile side.

Well, any time you find a community whose overall Humor Level is rated at "Homeland Security", it's a pretty good indicator that you don't ever want to have to deal with them. Ever.

In any case, the whole discussion about whether Google's approach is viable for tech companies in other domains is a red herring. Most companies don't use Agile Methodologies, or if they do, it's only a handful of teams, maybe 10% or fewer, I'd guess. At least it's true at the companies I know lots of people from - Sun, Microsoft, Yahoo, Amazon, Google, Blizzard, and other places like them: industry leaders who write kick-ass software. They do it almost entirely without Agile. It's not just Google. It's everyone.

Agile folks are so damn loud, I mean I swear they ought to get some sort of disturbing-the-peace ticket, that they'd have you believe that even if they're not a majority, they're some sort of moral majority.

I think you know better by now.

Static Typing and the Need for Reassuring Metadata

A few people caught a rather subtle point about static typing I made towards the end of my good/bad agile rant. I didn't dwell on it, but all the follow-up cries of "well what should we use, then?" have got me wondering whether it's actually more central than I suspected at the time.

I'm going to share some apparently unrelated short anecdotes. They mesh, though. Bear with me.

I had a high school English teacher who, on the last day of the semester, much to everyone's surprise, claimed she should tell just by looking at each of us whether we're a slob or a neat-freak. She also claimed (and I think I agree with this) that the dividing line between slob and neat-freak, the sure-fire indicator, is whether you make your bed each morning. Then she pointed at each each of us in turn and announced "slob" or "neat-freak". Everyone agreed she had us pegged. (And most of us were slobs. Making beds is for chumps.)

As I've done for a great many other programming languages, I've bashed on Perl's technical weaknesses at length in the past. To my continued amazement, the Perl folks are the only ones who never get upset. They just say "Haha, yeah, boy, you're right, it sure is ugly. Heh. Yeah, so, um, anyway, I'm going to get back to work now..." It's awesome. I've gained so much respect for them. It's almost enough to make me go back to programming in Perl.

Well, almost.

The Perl folks have this Perl Haiku competition each year. It's a nifty idea, and it's pretty amazing that you can write useful seventeen-syllable programs.

I tried it with Java once, and produced a valid Java haiku:

ArrayList<int> myListOfInt = new ArrayList<int>();

which, spoken aloud, reads:

ArrayList of Intmy list of int, equals newArrayList of Int

Maybe it's just me, but I find it pretty farging depressing that a simple declaration like that can be haiku-sized in Java.

The noted Far Side cartoonist and computer scientist Dr. Gary Larson published the definitive paper on Java-style static type systems in October 1987. I've taken the questionable liberty of reproducing his paper here, in full, sans permission, in the hope that it'll fall under "fair use" if enough of you buy a Far Side book. If you don't, well, just remember what happened to Miranda Pinsley!

It's truly one of the most insightful Computer Science papers ever published. Yet what Dr. Larson may not have realized at the time was that he was also predicting Agile Methodologies!

If there's one thing I've learned over the years about type systems, it's that you have your slob type systems and your neat-freak type systems, and it comes down to personal preference. The neat freaks (Java, C#, C++, Pascal) know damn well that the slobs (Perl, Python, Ruby, JavaScript) are just as productive. Maybe even more so. Nobody really knows for sure. One thing is clear, though: everyone's getting stuff done, regardless of their language choice. The whole debate isn't actually about productivity at all, even though most people still think it is. It's about whether you can stand to live in a house where the bed isn't made.

Did I mention making beds is for chumps? Well, that's just my take on it. My wife makes our bed almost every morning. I find it annoying and useless, especially since we have to tear the thing apart again every night. But she has to have the place spotless or she gets really uncomfortable. So, you know, I have to pick up my socks and all that stuff.

There will always be slobs, and there will always be neat freaks. There will always be programmers who have to paint "THE CAT" on their cat and "THE DOG" on their dog, metaphorically speaking, or they'll be as stressed and uncomfortable as my wife is when I've left a bunch of dishes in the sink and clothes on the floor. And there will always be programmers who need a methodology: even if you can convince them that their current one isn't helping, they can't just drop it. They need to switch to something else.

And that's perfectly OK with me, as long as they understand that it's not some proven silver bullet — it's just a style thing.

Unfortunately, Agile has been propagating like a virus via a slightly buggy survival mechanism built into all our brains, every single one of us. You have to take desperate measures to combat a viral epidemic, so I'm going to pull out the really big guns, and tell you about my dad's chili.

Making Chili

Many die-hard Agilytes have angrily retorted that Agile has been working for them for years, that they've been successful with it, and so on. They seem to think I'm claiming that Agile "doesn't work". A few even thought I was claiming that Agile projects are unsuccessful 90% of the time!

Well, make no mistake: Agile works (or at least it can work) just fine. The problem is much more subtle than that; most of you appear to have understood it no problem, but many Agile folks missed it entirely. So allow me to try being un-subtle, and see if that gets the point across to them any better.

When I was a teenager, my dad and my brother Mike decided to make homemade chili. I'd never seen it made before, and I watched with keen interest as they added beef, beans, some veggies and spices, and other ingredients. Dad would taste it, add some more ingredients, wait a bit, taste it again. My dad has some pretty good recipes. So you can imagine my puzzlement when he opened the cupboard, pulled out 2 cans of Hormel chili, opened them and dumped them in.

I waited a respectful moment or two before asking him why he was adding canned chili to his chili. They both said it tasted terrible, but, as my dad now-famously observed: "You can start with dog shit, and if you add enough chili, you get chili."

Similarly, if you start with an Agile Methodology, and you add enough hard work, you get a bunch of work done. Go figure.

But that's a tautology; you can substitute anything you like for "Agile Methodology" and it's still true. It's probably not difficult to find people who believe that Feng Shui has brought them success in their projects for years. Or throwing pennies in fountains. Heck, there are probably some people who practice witchcraft to help their projects out, and a great many of those projects — probably the majority — wind up being successful.

If you do a rain dance for enough days in a row, it will eventually work. Guaranteed.

So I'm not saying Agile doesn't work. It does work! But it's plain, unadulterated superstition.

And that, dear audience members, brings us to our last big topic today. Let's start prepping for that magic trick!

The Break-Dancing Chicken

Oh yeah. First I have to tell you about the chicken. We've done the cats, we've done the chili, we're on to the chicken. Almost ready for the final act, which is really gonna piss a lot of people off, I can tell you that right now.

In 1986, my college Psychology 101 professor told us a little story about her Abnormal Psych class from the previous year. The students each got a lab rat, and they had to train their rats to run through mazes by rewarding the rats for exhibiting properly ratty maze behavior. But there was one girl who was terrified of rats, so they gave her a chicken, and she trained it to play the piano by pecking notes with its beak. Each time the chicken played the right notes, she'd give it a pellet (or whatever the hell chickens eat). Wrong notes, no pellet. She'd teach it one new note at a time, and eventually it would know the whole song.

At one point, after she'd coaxed the chicken to learn maybe half its song, the chicken made a violent side-to-side motion just before playing exactly the right sequence of notes, including the new note she was trying to teach it. It had played the right notes, so she had to give it a pellet. The chicken had definitely figured out that doing what she wanted would get it yummy pellets. After that last pellet, it decided that it had been rewarded for both the note and the motion, and from then on, it made crazy twisting motions continuously as it played. She had to keep rewarding it when it played the right notes, and she had no good way of informing it that the twisting was unnecessary (not without starting over, and there wasn't time). So the chicken went on happily believing that thrashing violently was helping its project succeed.

They eventually went on tour to the local colleges with it, billing it as "The Break-Dancing, Piano-Playing Chicken".

Our prof told us wide-eyed freshmen that this phenomenon is called superstition, and it refers to the exact same kind of superstition you think of when someone mentions black cats, walking under ladders, breaking mirrors, opening umbrellas indoors, and so on. 'Superstition' is any belief not based on scientific experiment or pure deductive reasoning. (More or less.)

Well, geez, that seems pretty broad. We don't bother to go verify most of the stuff we know (or think we know) — does that mean we're all really superstitious?

It turns out we are. See, our brains are powerful pattern-matching machines, and in order to survive and to learn about the world we have to be able to build up millions of inferences about our experiences, and then match against that "pattern database" in real time. In order to process all that information fast enough, our minds have a tendency to assume that if two events are coincidental, they're also correlated.

That last sentence was probably the most important in this whole rant, so I'll restate it, in case you were dozing off at my boring lecture. Don't do it! We have a magic trick coming soon. You won't want to miss it!

Restated: when you see two events happening that appear to have a cause-and-effect relationship, then your brain naturally and automatically looks for reinforcement. Also known as "success stories". The chicken, completely by accident, got the right notes at roughly the same time it twisted. Then it received a pellet, and from then on it believed increasingly that the twisting was one of the things that "caused" pellets to appear. The belief was only reinforced from then on.

That's it. That's how superstition happens. And wouldn't you know it, most of the time it's right. That's why it works so well as a pattern-formation strategy. You touch the stove, you burn your finger. Do it again, sure enough, you burn it again. Cause and effect. But just because you've drawn the correct inference doesn't mean it's not superstition! You never really know if it's a "fact" until science has promoted it. Getting things promoted from superstition to accepted fact is the fundamental endeavor of all science.

Ironically, the chicken story is an Urban Legend. I have no idea whether my prof was telling the truth when she said it was "her" Abnormal Psych student who trained the chicken. Maybe she said it was another prof, and I just remembered it as being her class. I suppose I could go dig up the details (with a LOT of effort; this was 20 years ago) and possibly find out whether it's true or not. Until then, it's an Urban Legend.

That doesn't actually mean it's not true. "Urban Legend" doesn't refer to the message content; it refers to the transmission protocol. As it happens, many urban legends are true, if you believe Snopes.com at any rate. What makes them urban legends is that they're passed from person to person without keeping any of the original details correct about who, where, when. So even if they're true, the way you heard about them makes them superstition. Until they've been verified (if necessary, by repeatable experiment), that's all they are.

So superstition isn't necessarily a bad thing! We couldn't possibly go and verify every single fact we've heard that we think is probably true. We'd never make any progress (as individuals or as a civilization). We have to take most things we know for granted. Most of what we know at any given time is superstition. It's normal.

And of course many of our superstitions are transmitted to us from other people, through virus-like propagation. One propagation type is the urban legend. Another one, sadly, is marketing. One way or another, ideas make their way to us, and our tendency is to believe anything that seems plausible.

As I was thinking last week about what I'd write about in this essay, I Googled for "technological superstition" and came across this lovely little ACM article by Jef Raskin from 2003. In it, he talks a little about the psychological foundations of superstition, and goes on to poke fun at a popular and widespread superstition, namely that expensive audio cables sound better than cheap ones. It's definitely worth a read, and it has a little jewel at the end that I'm sure will make you gasp with appreciation for the author's Nostradamus-like predictive powers.

It's all fun and games until someone loses a testicle

Superstition is harmless, right?

Well, sort of. Mostly. Usually. In fact, the point of the previous section was the superstition is not only inevitable (for all of us), it's actually a critical brain function. Most of what we believe is superstitious, and if we really want to, we can go track down any particular belief and do some reference-checking on the sources (e.g. is the source Wikipedia or the National Enquirer? Both are sometimes right and sometimes wrong, but the confidence levels differ dramatically) or even repeat the experiments ourselves.

Superstition is strongest when people desperately want to influence or control events around them, but it's too hard for them, so they turn to the magic of wishful thinking: Feng Shui, pennies in fountains, shooting stars, Agile Methodologies.

Nothing wrong with that.

The fundamental cultural problem with superstition arises when people don't know how to differentiate between reliable and unreliable data-verification sources, so they treat non-science as science. If they don't recognize or admit that they're being superstitious, then they'll feel no qualms at all about trying to propagate their beliefs to YOU.

In fact, if they can do it successfully, if everyone believes something, then it's no longer superstition. (This is why philosophers wonder whether even scientifically accepted facts are still just superstition. Quantum mechanics has revived that line of inquiry in the last century.) That's why they want you to believe what they believe: it reduces the nagging doubt that they might be wrong. It's why superstitious communities spring up and clump together and get all insular — they self-reinforce within the group.

This, folks, is what's been gradually going wrong with Agile over the past decade. It's a complicated problem. And it has NOTHING TO DO with whether it "works" for you or not.

The problem is that if it kept spreading, it was eventually going to be a majority opinion, at which point the non-believers would start losing their jobs because they weren't playing along with the Establishment. Cowboys, indeed. Now you know why Agile folks call them Cowboys. They want to brand us all as rogues, as non-team-players. Scary, eh?

I don't want to have to wear an armband just because I don't share their techno-religious beliefs. I'm already wearing enough of them as it is. The Church of C++, for instance, is a physical and moral majority, and you're a lesser programmer if you don't use it. Get used to it. You live in an unenlightened age.

As far as I'm concerned, Agile Methodologies are fine for you to use, but if you try to push them on a coworker, it should be treated as an HR violation, just as if you were discriminating against them because they don't believe in Feng Shui.

If it's a manager pushing Agile on a subordinate, it should be treated with extra severity. Reprimand, goes on the employee record, mandatory training, and 3 strikes you're out.

You have to fight fire with fire. I want it to be their career problem before it becomes your career problem.

I joke about it a lot, and I'll continue to do so, but this is actually pretty serious stuff.

But let's go ahead and close out on a lighter note. I promised you a pony, right? No, wait, it was a magic trick. That's right. Sigh. All right, then. You asked for it, you'll get it!

Anagrams and Hidden Meanings

Here's one laaaaaaaaast superstition for you: anagrams! Ooooh, they're one of the most popular of all. You rearrange the letters of a word or phrase to see if you can find hidden meanings. That's all there is to it!

Unless you're truly superstitious — and I mean the Old World kind, where you still believe in crazy stuff like leprechauns and numerology and a steroid-free Olympics — then you know that anagrams don't really have hidden meanings in them; it's just random coincidence.

I mean, let's face it: if you look hard enough, you can always find some phrase describing your intended target that also has a really unfortunate anagram. It's just blind (usually bad) luck. I think Spiro Agnew knows this better than anyone in history.

And yet... sometimes it's soooo hard to believe it's really random. When I was in my mid-twenties, I ran my full legal name (Stephen Francis Yegge) through an anagram solver to see what came up. Ouch! I mean, here I was, struggling with my weight, and all the meaningful anagrams of my name seemed to be about being fat. The worst one was "piggery, hence fatness". Gosh. Was I cursed? It sure seemed that way.

Meanwhile, my little brother Dave was all pissed off. "Why are all mine about being sick? This is lame!" I remember it like it was yesterday. "David Francis Yegge" (my parents were so creative) came back with anagrams like "gas acid fever dying" and "dying grief cave sad", and Dave was NOT happy about it. Hidden meanings my ass, he announced, and he stopped playing with the program. I don't think he ever used one again for the rest of his life.

Are "hidden messages" like these a coincidence? Yeah. Yes. They are. From any rational, sane perspective, they're just sheer bad luck. Problem is, our brains sometimes just don't want to be rational.

I'll close with one last example, and it's a dirty trick — one that would be utterly beneath a more principled blogger than myself. Because after all my ranting about superstition, I'm about to use superstition for my own nefarious purposes.

Let's say, hypothetically speaking of course, you were to rearrange the letters of — oh, how about Agile Manifesto, just for pure curiosity's sake. An innocent exercise, nothing more. ("Hey, it can't hurt! It works for me! I've been rearranging letters since 1990!" Dorks.)

Well, chances are pretty good that you'll find lots of meaningless and ho-hum anagrams, and possibly one or two moderately interesting ones.

But if you're hoping for a really clear message, especially with bigger words in it — you know, something that would be unmistakable as a hidden message — you know you're likely to be disappointed.

So let's try it. Can't hurt.

How many 2-word anagrams of "Agile Manifesto" do you think there are?

Guess what? There's exactly one: Egomania Itself

Yup. I thought the exact same thing you just did. :-)

Sure, you can argue about probability and statistics and the fundamental nature of randomness, and you can cry foul play! no fair! poor sportsmanship! low blow! until you're blue in the face, and you'd probably be right.

But in the end, I mean, come on, you have to admit, it's a pretty spooky coincidence...

(cue haunted mansion bat music)

...or IS it?

(cue happy end-of-show music)

Tune in next time! I think I've pretty much finished up with Agile for now; I've stuffed garlic in its mouth and pounded a wooden stake through its heart and it's making melodramatic retreating gestures, shaking its fist at me and promising revenge in the inevitable sequel.

So now I can go back to writing about boring stuff that won't make me infamous. Like, you know, Google.

101 Comments:

I've just been given a special "non XP" position (read : cowboy along the XP team, think of Jean Reno in Nikita) for not being a team player enough (read : "I don't give a shit 35 hours a week, my weeks are between 20 and 100 hours depending on the mood"). Thanks you for fighting for my freedom.

I though I'be fired for not beeing XP-capable. By pushing things like : "people over process" you make management teams get back on the earth.

The Agile methodology, like other extreme technologies that tend to induce religious ferver in adherents (e.g., Lisp, Smalltalk, and for good reason), will provide the long-term benefit of some of its best ideas (unit testing is helpful in many cases) trickling into mainstream development processes.

There is nobody - nobody - who has as high of a Polarizing-Yet-Insightful-Opinions * Testicular-Fortitude-To-Post-Those-Opinions product as you, Mr. Yegge. You are one of the few bloggers who I read every day.

That breakdancing Chicken? It's a Pigeon, as described by B.F. Skinner.Interestingly, hearing the story in an Ethology class inspired me to write my first program in about ten years, to demonstrate the selection effect of random reinforcement to myself.

I'm a tech lead on a project at Google. We very recently started using Scrum to see whether we could get better at producing more of what our customers wanted faster. I was the one who sold management on the idea; it didn't come from above, so don't blame them. So far this is working pretty well.

I understand that this is a personal blog, and Steve's giving his opinion. Still, it bothers me because it gives what seems like a pretty inaccurate view of Google and what little I know about agile development (sorry, I don't think it merits a capital 'A'; it seems faintly ludicrous to me; maybe that's the sort of thing that bothers him too).

As far as Google: there's been a lot of talk here about the "Google way", which sounds a heck of a lot like "Google methodology" to me. If there is one, I don't know what it is, and I've been working here almost three years. Different groups use different approaches -- some are design top-heavy, some are totally XP (though they are the minority), and some just lump new functionality in until they decide it's done. Groups do what works for them, and nobody much seems to mind.

My point is: "Google" does not mean "doesn't do agile." It doesn't mean anything remotely like that. We have an "intergrouplet" (a semi-formal organization of engineers supporting an aspect of engineering life, like documentation or the build system) devoted to agile, and upper management (who are not stupid people) supports the spread of agile ideas. So yes, we do agile at Google, big A and little a.

I spent a lot of time talking to people in that intergrouplet before we started trying Scrum. The attitude I got was, "Well, here's what the books say, here's what we've tried, and here's what seems to work pretty well for us." Which is what I wanted to know. I'm not interested getting a certification; I'm interested in making my group happier and more productive, and if there's an idea that will help me do that, I'll try it. Many of the agile ideas seem to fall in that category. We're trying a few now, and if they work out we'll try more later. If not, we'll drop them. Maybe that means we're not really doing official Scrum; but if it works, who cares?

What bothered me in these posts was the idea that anybody who used agile was either a gullible mark, or an unscrupulous consultant, or both. I'm not a consultant, and I don't think I'm that gullible. When I first saw Steve's post on "Good Agile, Bad Agile," I thought, great, here's somebody who's opinions I respect, and I'm going to learn something. What I got seemed like pretty wild, unsubstantiated ranting about why agile is bad, without reference to any specifics, or any discussion of particular agile ideas he didn't like. The one specific I do remember him disliking was pair programming, but the argument seemed to be, "Pair programming sounds stupid to me; therefore, it is stupid." I'd be very happy to hear why it's not a good idea, or anecdotal evidence about how it failed, but that's not the kind of thing that's going to convince me of anything other than his strong personal dislike of pair programming, the same way that some people don't like science fiction or folk rock.

(I really wish, by the way, that Steve had described what experience he had with agile development; he said he'd tried it, but not what actually happened. Did he have to do daily standup meetings? Did his group do pair programming? Did they have regular meetings with the customers? Why didn't it work? That's a post I very badly want to read.)

I was depressed and frustrated by these posts, because they're getting a lot of play, and the lesson people I talk to seem to be getting isn't, "It's okay to say no to Agile," but "Agile is stupid, so don't waste your time learning about it, and if you use any agile methods, you're stupid." I bet that Steve is a very good engineer, and I'm ready to be convinced, but I need more than that to do it.

It occurs to me that maybe the point wasn't that agile development ideas are stupid, but that people have a tendency to turn ideas into religions, and while some agile ideas might be (and, I think, are) healthy for software engineering, the Church of Agile Development isn't. That's an idea I completely agree with.

Patrick: The well over 95% of teams at Google -- and everywhere else, mind you -- that do not use Agile needed a voice as well, precisely because of the Church problem you bring up in your last paragraph.

I would encourage you to be sensitive to the fact that it's hard to back out of Scrum if it's not working significantly better than not using it. (And you won't really be able to tell; it'll just be a "feeling".)

You should run weekly double-blind polls with your team, peers, and management. If anyone goes limp, taps out, says stop, even if they're faking it, the fight should be over, and you should put the process back on the shelf.

This article was very hard to read. You really need an editor or two. The rambling is out of control. What exactly was the point of this article? Agile something or other is bad? Your dad made chili? There is a computer program that makes anagrams? Very, very hard to follow...

Steve, I have (worked|working) in two diagonally opposite organizations.The first one was a services sector IT company and its as far as you can go from Agile. Truth is, yes stuffs get done there...but i spend entire month doing changes pertaining to a pkzip version change.A thousand documents to prepare, log the defects here, there everywhere. I was frankly pissed and i quit. Now..i work in a organization which is sort of middle path. I would even go one step ahead and may call it Agile. Stuff gets done really fast here. I personally feel more productive and i code more here ( * no substitute to hard work, you see *). But there is a downside, some people here...just code away stuff, without keeping scalability in mind. But stuff normally gets sorted out in code reviews. So, whats the point. I guess, you have gone too far in your criticism against Agile. You haven't really worked in a non-agile environment, you were fortunate enough to work in product development companies like Amazon or Google. But just come out of the shell and there is a IT services sector out there and developers there being bogged down under tons of documentation, meetings, lunch with project manager and some stupid team building exercises. Ever heard of 6 Sigma ratings or CMM level 5 certifications, nah you haven't dealt with them. My previous employer was a CMM level 5 certified IT company and I was not allowed to use Emacs there.

Why? because their code indentation rules sucked. Agile, is a breath of fresh air for them. I am not fully in support of Agile, but you have to agree there are good things out there and its not just pure coincidence or superstition. And you don't really have any substantial arguments or as you said scientific arguments against Agile either. I bet, someday you will look back and call "Steve Yegge Drunken Blog Rants" and blame Heisenberg's uncertainty.

Steve, thanks for responding to my comment. Maybe the conventional attitude toward Agile is just very different from my own experiences. Both Google and the last company where I worked are very light on process, and engineers have a lot of authority, so they would have to be convinced of its value for it to be adopted. At a company where engineers don't have authority, or where management is blindly willing to follow the latest process fad, I can see the potential for disaster. Maybe the potential is worse with big-a Agile because there's a kind of touchy-feely, let's-hold-hands-and-sing-Kumbaya quality to some of the presentations that clouds the discussion of what should be specific, practical, measurable development practices.

Speaking of which, I strongly agree with your point about evaluation. We are concerned about how to tell whether Scrum is working. We're talking about internal customer surveys, engineer feedback, comparing quarterly objective results, etc. to try to measure this, but we don't know whether those are going to work. I don't know whether we'll be able to find an objective measure of success. I haven't been able to find much convincing evidence in the literature for or against agile, just a lot of very small experiments and anecdotal data. That's not a criticism of agile -- it's incredibly hard to run a systematic experiment to measure things like developer productivity -- but it means we're trying this because it "feels" like a good idea. We've given ourselves a quarter to experiment. I think (and hope!) we're wise enough to drop it if it doesn't "feel" right after that.

I'm a big fan of agile techniques in general but I've always had a problem with the unjustified advocacy and shameless marketing. Of course it's just the latest in a long line of snake oil solutions. Maybe one day we'll stop trying to define the perfect process and just get on with it.

disclaimer: I've never done XP programming, and I wouldn't know Agile methodologies if they poked me in the nose. The first time I heard someone go on about XP (at JavaOne a few years ago), my thought was, "how interesting: they're trying to find ways for programmers to be cogs so they can get control of the process."in 2001, I interviewed at a hardware company to be their VP of software development. They'd implemented the 802.11a/b/g stack in hardware (yes, entirely in hardware, so they told me) and were very perplexed about why software development was always so hard to schedule -- their hardware development was very deterministic, and they could gauge the complexity and be very accurate in their scheduling, but software... well, it was squishy. I had no good answers for them.software development is a fundamentally creative act, and debugging is often an intuitive one (where intuition likely is subconscious reasoning). I think much of software can be reduced to the production of cogs, but the great stuff, the really innovative stuff, will never be spit out of a machine.

I've always been a fan of statically typed languages, though I've been able to use variant-typed languages like JavaScript without feeling the need to jump in the shower, but the recent push towards type inference in one of my languages of choice has left me feeling scared that I won't know what the compiler is actually using. I crave the organization and trust that I feel coming from the practice of assigning an interpretation to each symbol at each level of the code.

By your theory of neat-freaks, this should make me an early-riser who can't focus on anything until the bed is made and the counter cleared, but this is not the case (and it is not the case for my wife either! :-). What, then, am I? Am I some sort of freakish hybrid?

To people claiming that Steve "needs an editor" and that there's "too much rambling", remember that these are not essays he is writing. These are blog entries. Blog entries are meant to be vitriolic, meandering, fun, and feel more like sitting around a campfire with a great storyteller than listening to a PhD candidate defend his thesis.

Heck, Steve even claims outright that they're "[drunken] blog rants". It turns out that they're quite sober, and contain many quite well crafted jabs. Story and metaphor drive points home much deeper than an RFC.

Just add me to the list of people who have unmade beds and use a statically-typed language. I've just spent too much time playing guess-the-type with interpreters where it's basically impossible to have "37" remain a string and {{"37"}} remain a list of lists (that happens to have one element).

I don't use statically-typed languages because of how I was potty trained, but out of the same desire for effort minimization that drives your anti-static preference. It's just that for me, the minimal annoyance is in having to have a preface to every function labelling variables and having to manually cast when necessary, and for you minimal annoyance is evidently in accepting the speed hit from frequent type-checking and having to fix the occasional mis-cast when necessary.

Also, S., thank you for not editing to the least common attention span. If they don't want to read the whole thing, they learned how to skim an article in junior high.

I find your comments pretty interesting, but I'd like to present a counterpoint.

I worked on a team (at the same company as you) where we were essentially "forced" to use Scrum. I don't think a single engineer on our team found it useful, and I personally found it annoying, but we didn't really have a recourse for evaluation: I gave feedback that I didn't like it, but I'm not sure that everyone else was comfortable doing the same, and there weren't any of these wonderful double blind weekly evals. I also think its results were mis-used. The data that was derived from it (how much our team could accomplish in a week) was tainted and it shouldn't have been turned around and used to make decisions.

I think this shows that even our venerable company is not above top-down pressures to adopt processes that should remain the the personal choice of individuals on a team. I appreciate Steve's posts so far because they address the issue of this being adopted as a process for many, without evidence.

I personally don't ever want to use Scrum again. I'm not sure how this jives with people who find Scrum useful -- am I just supposed to stay off of their teams? Are we going to be divided into agile vs. design upfront vs. whatever teams in the future? What happens when someone on my team wants to use index cards, but I find them ridiculous and contrived?

I agree that each team should be free to pursue whatever strategy it finds useful, but individuals on those teams should never be made to use a process without evidence of its worth. And just because its Google doesn't mean that we don't have to be very careful about these issues -- you make a lot of assumptions about our engineering culture that are not true everywhere.

I'd be very interested in talking with you about your experiences with Scrum. I think there are things about it that are going to be valuable for my team, but our goal is to make our development process better, not to market Scrum. If there are pitfalls here I'm just not seeing, or not taking seriously enough, it would be great to understand what they are.

I'm not sure how to answer your question regarding groups that disagree about process issues. I don't see a fundamental difference between Scrum and any other approach here: what about engineers who dislike the waterfall model (which must be most of us)? What happens when someone in your group won't start coding without a big up-front design document and you just want to use test-driven development? Some approaches to software development won't play well with others, and I guess I don't think it's always reasonable for each individual in a team to pick a process that suits him or her.

Clearly there are some aspects of process where most people can agree, or we think it is acceptable for a higher authority to mandate certain behavior. Source code control and code reviews are a couple of examples that nobody I know of argues are bad ideas, and even if they're occasionally irritating, most of us follow them. On the other side are things (or so I've heard) like telecom software reviews, where they'll do multiple rounds with large committees to read over small changes to make sure they aren't going to mess up the phone system -- almost anyone not in that world would regard that as overkill. The problem, which feels to me like the Emacs vs. vi debate, is the large, fuzzy world of things like 3x5 cards and daily standup meetings and pair programming, where people have intensely strong opinions; but process, unlike editors, is something that the group does, and isn't an individual thing, so we can't always just agree to disagree.

My immediate reaction to your question was to think that yes, teams should collectively pick their processes, but I realize that that's not very realistic in most cases; either the team can't agree, or management has a way they want things done, or (as in my case) there's somebody evangelizing for an approach that nobody's strongly objecting to. The alternative I see is a sort of no-process process, where everyone does what they personally prefer, which can be amicable but has some big disadvantages too. To some extent, that's the world we're coming from, but it still remains to be seen whether more process helps us out or not.

paulo -- thanks for the reference. I wonder if my 1986 psych prof was passing along an urban legend, or if she recounted the facts from the paper accurately and my re-telling of it changed over the next 20 years. I'll have to go review the tapes after I'm dead. In the meantime, I'll go check out the paper. :)

I worked in a company that was "sold" agile by a string of consultants. It ended with redundancies. I've seen so many methodolgies come and go in exactly the same manner and it's good to see this one going. I won't miss it.

As an Orthodox Jew, I think that I can bring a different perspective to this religious war. I hold tight to all those Commandments in Torah that most people ignore. The essential thing is this: I'm not looking to take that cheeseburger out of your mouth, or prevent you from going to the movies on Friday night. Holding strong religious convictions is fine (i.e., Agile); just limit your proselytizing to those who seek you.

[You never really know if it's a "fact" until science has promoted it.]

This was incredibly lovely, and in fact sums up these last few posts perfectly. If, for example, Steve bumped his head on the door, he would not know whether he has a concussion until "science" (wow... just take a moment and admire the reification here) declares that yes, a new fact has been discovered, and Steve has bumped his head. (For more points, "science" should do repeated experiments to come to that conclusion.)

For additional fun, realize that repeated experiments, where event B (called "effect") follows in time after event A (called "cause"), is exactly how "science" supposedly works to decide what is "fact" and what is "mere" superstition. Apparently, the only difference between a regular Joe seeing something happen several times, and a scientist doing the same is that, well, the latter is a scientist.

Yegge, Someone is always going to bash you for some side comment you make that you never expected anyone to question, so here I go:

I understand that the Enquirer, despite being a gossip magazine, has been trying to improve its reputation by extensively fact-checking all of its articals. Wikipedia can be said to have extensive fact checking for those articals with high interest and multiple submitters, but surely there is still a lot of unchecked crap.

So I demand you take it back!!! Enquirer rools Wikipedia drulez!!!!!!!

And in defense of C++, some of us are still doing client applications with large data sets and find that C++ has the best combination of performance, collection of libraries, and platform support and don't feel like "well, you should still use ruby wherever that _doesn't_ matter" makes any sense.(Though I totaly understand your cat painting comment. Really what gets me is doing consructor/copy constructor/assignment operator/destructor before I can write code that actually does stuff -- and that so many argue about what the right way to write those fillers are so that it is thread/exception/DRY safe!)

Stevey, if you didn't read Umberto Eco's "Foucault's Pendulum" you should. Echo - who is a first professor of semiotics with University of Bologna - shows how our mind correlates disconnected facts, regardless that the facts are related or not in reality. In his book he gets the reader to believe in a story which he demolishes completely in the end.

He shows that a statement is true or false not based on fact, but by how many people believe the statement is true or false. Given enough believers, then anything BECOMES true.

Same as Eco, or the Agile Manifesto people, you do the same. If you can build enough belief into your readers, than suddenly you become a man of truth. And you are a pretty good writer. "You can start with dog shit, and if you add enough chili, you get chili." LOL. Your chili is pretty good ... :)

Of course, being scientific and objective about things doesn't help either, because of two reasons. First of all because it seems science is extracting facts by randomly connecting dots (see George Chaitin's work), same way our brain does. No surprise there since our thinking process is what invented science. And secondly, objectivenes is no good because it is too easy to twist and turn around. As an extreme, one can even justify genocide based on objective facts. Objectiveness can be that bad.

With that being said, you're article is strictly your loud opinion on agile and it represents the truth as much as the agile manifesto does.

Finally, if you're one of the good guys and you're honestly trying to make a difference, maybe you try to encourage / remind people to think for themselves?

Honestly I don't know how to take all of Steve's comments. You can read them one way and they are funny, another way and they are not funny.

Many of the arguments suffer from well known fallacies. http://www.nizkor.org/features/fallacies/

It is hard to tell if Steve is saying that there is Agile, Waterfall, some other devilish method, and Google's way OR if he is saying there is Good Agile and it is Google's way OR something else.

I do believe that there are people selling Agile, Spiral, CMM, and many other methods, including Google's way, as being the best.

From what I have gathered the Google way consists of:a) Hiring people who can't be dupedb) Not telling when a product may release.c) Hard work is the solution to all problems.

I am pretty well known in the Agile communities. They know that I don't believe in any one process. I do find many good practices in the Agile camp. I haven't found anything new in the Agile world, but I have found new ways to talk about techniques and practices which I consider good.

I am not as elloquent as Steve and I often have to find my voice through someone else. I found that Ricardo Semler had experiences that I could understand and had shared them in a book. Here is my summary of that book:http://home.att.net/~geoffrey.slinker/maverick/Maverick.html

Now let me explain a bit about myself. I have always believed that books that tell you how to be successful are a joke. The book itself is how the author is being successful and the author depends on fools to buy his book.

After much resistence to a friend's suggestions to read Semler's book, he finally loaned me his copy so that I wouldn't have to spend money on the book and therefore I wouldn't feel like I had fallen into the trap.

Another text that speaks volumes to me is "The Law". Here is a link to that as well:http://home.att.net/~geoffrey.slinker/maverick/TheLaw.html

This should give readers some insight into my personality.

I took the lessons from the "Maverick" book and I came up with the Maverick Model for software development. http://home.att.net/~geoffrey.slinker/maverick/MaverickDevelopment.html

Shortly after I had done this friends wanted me to apply the Maverick Model to a specific situation, thus creating a method. I was hesitant knowing that once you state things in a method there would be many that would take it out of context and then find fault.

Sorry, I have been rambling. Maverick is not sold, there are no boks to buy, no consultants to hire. There is a discussion group which is available to all. Since I do not get any money for my efforts does it make Maverick better than Waterfall (which has many many books and papers) or Agile or the Google way?

There is a type of person that I call and Agilista. Agilistas are nothing more than opportunists that have jumped on the Agile bandwagon and they shout "come down to the river and be healed". These Agilistas don't understand development, they don't know why Beck, Cockburn, Jeffries, Fowler, or whom ever suggest specific practices. These Agilistas don't even know what the real problems are!

I feel that there have been a lot of non-developers that have found their position weakened by new methodologies (in this case Agile) and they found themselves loosing clout. Therefore they have relied on their "snake oil" salemanship and jumped on the wagon and started shouting their sales pitch. These types of people ruin everything and they don't even understand what they are ruining.

Also, I have seen those that have not experimented with new applications of old ideas (such as Agile)and have superficially declared them stupid or snake oil, or some other form of the "appeal to ridicule" approach.

When I first heard of XP I did some quick study of it and I started a paper. The title was "XP eXposed". I wanted the paper to be sound so I was doing my homework. As I experimented with XP I soon found that it wasn't as stupid as I first thought. Eventually I found that there were many good things to learn (for the novice) and to be reminded of (for the seasoned). I never finished the paper. Why? Because I found it better to focus on the good than to focus on the negative.

I understand that some people feel the need to debunk the myths and expose the frauds. That is good, but not good enough.

What is needed are solutions to real problems. I have always found it hard to listen to conservative talk radio, eventhough I am very conservative. Why? Because it doesn't take too much to point out what is wrong. It takes much more to point out how to fix what is wrong.

Ridiculing early risers, strongly typed language users, bed makers, etc., may be entertaining but fails to get us closer to solutions.

Also arguing that if any one soul finds fault with some practice (in this example a daily Scrum) then the practice should be shelved immediately then we can clearly shelf Waterfall, Spiral, Google's way, C++, Ruby, Python, Smalltalk, Fords, Chevy's, Hondas, Democratic Republics, Socialism, Communism, Scooby Doo, H.R. Puff-N-Stuff, the Wiggles...

So, to keep to my mantra of offer solutions:1) Experiment. (http://home.att.net/~geoffrey.slinker/maverick/Scouting.html)2) Learn the difference between Leadership and Management.3) Work can not be replaced. Talking about work is not work. Planning to do work is not work. Meetings about work is not work.

Sincerly Geoff.

Here is a snippet about leadership that I believe to be useful:

The leader, for example, has a passion for equality. We think of great generals from David and Alexander on down, sharing their beans or maza with their men, calling them by their first names, marching along with them in the heat, sleeping on the ground and being first over the wall. A famous ode by a long-suffering Greek soldier named Archilochus, reminds us that the men in the ranks are not fooled for an instant by the executive type who thinks he is a leader.

For the manager, on the other hand, the idea of equality is repugnant and indeed counterproductive. Where promotion, perks, privilege and power are the name of the game, awe and reverence for rank is everything and becomes the inspiration and motivation of all good men. Where would management be without the inflexible paper processing, dress standards, attention to proper social, political and religious affiliation, vigilant watch over habits and attitudes, etc., that gratify the stockholders and satisfy security? ... Managers do not promote individuals whose competence might threaten their own position, and so as the power of management spreads ever wider, the quality deteriorates, if that is possible. In short, while management shuns equality, if feeds on mediocrity... For the qualities of leadership are the same in all fields, the leader being simply the one who sets the highest example; and to do that and open the way to greater light and knowledge, the leader must break the mold. " A ship in port is safe," says Captain Hopper, speaking of management, 'but that is not what ships were built for," she adds, calling for leadership... True leaders are inspiring because they are inspired, caught up in a higher purpose, devoid of personal ambition, idealistic and incorruptible... So vast is the discrepancy between management and leadership that only a blind man would get them backwards... "

Steve, I gotta say this is pretty unfair to agilistas--I don't know one of them that can write half as entertainingly as you.

A lot of this stuff is common sense, but the Far Side cartoon really cracked me up. It's so true about static typing. I mean, it's gotta be pretty well proven by now that static typing is not necessary for scalability or maintainability, but yet some people just can't stand not having it.

When it comes to methodologies I always go back to "No Silver Bullet". The fact is, there is a certain capacity to be a good programmer based on the interest and focus to solve difficult problems. I think it's characterized by the confidence and tenaciousness to debug any issue, not necessarily any particular knowledge. The people who don't have this ability area always looking for a way to make programming easier and more foolproof. Of course methodologies, new languages, and tools can help, but sooner or later you will hit a bug which requires a real programmer to solve. The developers of these methodologies are effective salesmen, because they are honest. These methodologies are the result of skilled developers refining their own process.

The problem is mediocre programmers won't get anything out of the methodology if they don't understand why. Otherwise it's just blind obedience without reason. The best you can hope for from a methodology is inspiration for how to solve issues you are already aware of.

Well, I'm off the net for 2 weeks, and some interesting stuff sure happens.

First, I am a strong proponent of XP. I have seen it work, and seen it work well, in several projects at several companies.

What I sense from your blog is that the XP "zealots" you see make it out that there is only one way to do XP - by the book. However, I see (and believe) some strong leaders respond to comments like "If I do this, am I doing XP?" with comments like "XP isn't about following a strict methodology - it's about delivering value to the customer"

To that end, I believe that XP is a tailorable process. Pair programming doesn't work for everyone, and may not be needed all the time. Does that mean you aren't doing XP? (And the answer is: it depends on who you are talking to).

Your comments highlight one of the biggest fears I've always had about XP - that once managers and consultants got ahold of it, it would be basterdized into being a prescribed methodology, instead of a collection of practices that can help improve the development in your shops.

I, instead, view XP as a cookbook. I look at the practices and the current situation, and see which ones fit for the current problem the organization is experiencing (or percieving). For example, if the relationship between the developers and the customers is adversarial, then some of the customer practices like planning games and the such can help mend that relationship.

But, it might not.

And good teams know that, in XP and agile, when something doesn't work, you do a retrospective of why it doesn't - and then ditch it if you can't fix it.

It's a shame that what you've seen for XP has given you the impression it has. No matter how completely amusing your blog posts are.

Because, in my world of XP, one of the most important things for the teams to remember is that it is /their/ process, not anyone else's, and it is /their/ responsibility for ensuring that they understand the principles behind the practices so they can tweak them to fit their needs.

First time commenter; followed the link from Jeremy Zawodny's linkblog. Just wanted to let you know that your page (main page) screws up my browser... not quite a full browser freeze, but a freeze-of-the-current-window-and-all-tabs-in-that-window (Firefox on a PPC Mac). Anyway, adblocking "http://rcm.amazon.com/*steveysblog*" gets rid of the problem. 27 iframes in a 216kB page seems to do the trick of killing my browser. Individual post pages work fine, though.

Python and Lisp are languages I use most often. They are veryflexible, but their lack of the ability to do static typingsometimes makes bugs more difficult to find. In other words, if your code looks unusual, it's either a mistake or asign of genius. In most cases, the former.

I prefer to treat types as compiler-understandable comments. Clearly,not every variable declaration needs a comment (as C/C++/C#/Java wouldlike), but sometimes comments are useful. Type inferrence, orgenerally "comment inferrence", is the way forward. Most importantly:it should be up to the programmer what and when to comment, not to thelanguage designer.

I'm curious what it means about Google's product management and development processes that they just paid 1.65 Billion dollars for a competitor to one of their products? I'd also like to know this: you describe a pretty warm and fuzzy environment of positive reinforcement. Will there be any negative consequences for the Google Video team getting their asses kicked to such an extent? Or will they just melt away onto other teams? My final question is whether you think that any kind of date-oriented product management could have saved Google Video. Those of us who work in the parallel universe where our companies cannot afford to buy our competitors are frequently told that the reason dates matter is because there is an advantage to being a significant force in the market first. Your opinion?

There is probably too far down to ever get read, but I'll post it anyways... it's a short anecdote from my personal career.

In 2001, I was working for a Small California Software Startup. We had 10 or 11 engineer working on an enterprise application, and had been for about a year. Management was forcing us towards a ship date that was too aggresive (by a few months), in order to book revenue (which of course would ultimately leave the customer very unhappy with the unfinished product, and they wouldn't be repeat customers).

So, one day, they fired the VP of Engineering, the VP of Marketing, and the Director of QA, and introduced us all to none other than Mr. Ken Schwaber (author of "Agile Project Manager with Scrum"). And, www.controlchaos.com.

Interestingly, the old VP of Engineering was keeping track of all the tasks on a big paper calender with stickies that she would move around as features got put in or moved out. Mr. Schwaber convinced us that this system wasn't any good, and having a once-a-week meeting that took about an hour to update this calender wasn't going to work. Then we started having DAILY meetings where we answered the three questions.

I bemusedly noted after about the third day that the answers to the questions that I gave didn't seem to matter. I could lie about what I did yesterday, I could like about what I was going to do today, and I could make up all sorts of reasons about what was blocking progress. No one seemed to be recording anything, and there seems to be no actual destination, just a continual tacking of the boat, back and forth, back and forth.

I quit and changed jobs about 2 weeks into this new process.

The paper calendar with stickies seemed quite good to me, and I'm sure we doing Scrum wrong.

I don't think the product ever shipped in whatever form it was in, and I know it didn't make any money.

Unfortunately, I think management truly beleived Scrum was going to allow them to slow time itself.

And by the way Mr. Schwaber speaks, you'd be hard pressed not to beleive he thought himself able to slow time.

For additional fun, realize that repeated experiments, where event B (called "effect") follows in time after event A (called "cause"), is exactly how "science" supposedly works to decide what is "fact" and what is "mere" superstition. Apparently, the only difference between a regular Joe seeing something happen several times, and a scientist doing the same is that, well, the latter is a scientist.

No, that's not how science works. You really should read up on things before you post such nonsense.

What science actually does, in addition to noticing correlations between events, is attempt to find a way to show where those correlations break down. A scientist comes up with a theory, then tries to falsify it (prove it wrong by running other tests).

And I think that's part of what Steve is saying isn't happening with development methodologies. Lots of people are doing the annoying marketing/religion thing - they're using anecdotes about how their system worked for them and completely ignoring any of the problems or attempting to find out where it goes wrong.

He doesn't say how to test such things, but he at least claims that he's seen Agile development not work. (I agree with Patrick that some specifics would be nice.) That's a step in the direction of doing some actual work to figure out what does work, but it's not the final step by a long shot.

You should read about Russell’s Paradox. When you ascribe to a Manifesto, of any sort, you, by definition, have put individuals and interactions in the back seat. Thus, any process that defines itself as above process is not adherent to the process to begin with. Try saying that 10 times fast.

Working software over comprehensive documentation

What good is “working software” if no one knows how to use it?

Customer collaboration over contract negotiation

Yeah, that really works. What programmers need, more in life, is to start dealing with the customers more and to not worry about getting paid. It’s one of those nice sounding phrases, where you get the picture of the programmer and the customer holding hands and walking though the flower fields sing kumbaya. Be weary of things that sound so happy.Responding to change over following a plan

Like many folks in these threads, I've been managing software projects for a while - enough to have managed with waterfall methods, agile methods, back-of-a-napkin methods, and run-fast-the-exec-is-coming methods.

In my experience, the equation has been quite simple: if

-you have a strong engineering team-your problem is reasonably well understood-you can reasonably manage your managers

(note that I said "reasonably" and not "perfectly" in both of the latter two items)

then less process always wins. The right things get done, priorities are basically ok, people are basically productive. Anything that's in the way is, well, in the way. That goes for Microsoft Project, random web tools, and index cards with a system.

This is not a revolutionary point, though it's worth remembering when you have the privilege of meeting those top requirements.

Beyond the bluster, predicted paranoia, and straw man attacks inside Steve's posts here, that seems to be the main point. It is perhaps a valuable one.

Yeah, that really works. What programmers need, more in life, is to start dealing with the customers more and to not worry about getting paid.

Actually, you are right. Delivering what the customer specified initially instead of what he wanted leads to worrying about your income eventually. No employee has a problem with immediately changing the work when the boss says so (instead of sticking to the old order). External suppliers have, but the problem here is simply the contract sum which one or the other side thinks needing renegotiation.

And what do you want to say with Russell? That it is logically impossible to state generally that you ought to make up your own mind instead of following rules? Good thing men are not slaves to logic, then.

My point about Russell was that the definitions in the Agile Manifesto stated above were misleading. If there is a manifesto about how to act and that manifesto claims to talk about the need of individuals and the need to not stick to a “plan”, then what does one do who follows such a manifesto if the thoughts of one individual are in conflict with those beliefs? In most cases, people who follow such beliefs are following a “plan”, in the classic sense, so what if not following the “plan” is in conflict with the plan itself, ie. The Agile Methodology.

So my point about Russell was that the underlying definitions that were brought about were a little full of it. You can be an individual, as long as you subscribe to our way of thinking. Plans suck, but if you don’t stick to our plan, then you are a Cowboy. The ideals and the reality do not reconcile.

Some dis-Feng Shui-inal ramblings...(I prefer bad puns to anagrams). How about methodologies are tools just like programming languages, IDEs, editors, etc? Most only do well on certain types of projects. Agile works well on highly seperable, shallow problems that amenable to casual user verification. The church of agile wants to apply it to all problems and it simply isn't capable of solving complex, highly interdependent problems.

It looks like that Google folks have the longest comments/articles I'd ever encountered. Either:

1. They have a woderfull model that makes 10 people doing work of 100 by multiplying their skills.

2. Google had(!ve) a great techology and marketing strategy to suck a lot of cash from investors/advertisement and now throwing all this cash freely into the heads (pockets) of overexcited programmeers that are not capable on using reasonably readable English (not that I whant to cut anyone's part of the body, but offering this opportunity in public is really a bad taste!).

Don't take me wrong, I admire Google of being first reasonable challenge to MS - but I hope neither Page nor Bryn will through this god-like speech and spend that amount of thime typing it (nad, NO, I don't expect them to offer parts of their bodies to be cut in case they need to do things they considered wrong).

Holy Smokes! You're posts are complete garbage! You wasted just under 5000 words to state the following:

-Agile is crap-Google doesn't use Agile therefore Agile is crap-Perl programmers are goofy therefore Agile is crap-My dad puts shit in his chili therefore Agile is crap-My professor told me a silly story therefore Agile is crap-Superstition isn't science therefore Agile is crap-I made an anagram, therefore Agile is crap

You did not make a single point in your steaming-pile of a post, you didn't prove anything, and you portray your self worse than any slimy snake oil sales man.

If you really want to disprove Agile, as opposed to writing a massive flame-bait article, you should do some actual research and present it in your post.

I’m not advocating Agile, I could care less about it. I’m just pissed off because I wasted a portion of my life reading your poor excuse for a post. It seemed like it was going to be an interesting topic so I stuck with it.

Now I’m even more pissed off because I feel the need to post a reply (thus stoking your blatantly massive ego) about how pissed off I am.

Poor Mike. If the post is such a steaming pile, why on earth would you allow it to ruin your day? You know that's just your choice right? You can choose to think of Steve's post and allow it to piss you off or you can choose to move onto something else that makes you happy. That's kind of what the post is trying to push I believe.

I've just been given a special "non XP" position (read : cowboy along the XP team, think of Jean Reno in Nikita) for not being a team player enough (read : "I don't give a shit 35 hours a week, my weeks are between 20 and 100 hours depending on the mood"). Thanks you for fighting for my freedom.

I though I'be fired for not beeing XP-capable. By pushing things like : "people over process" you make management teams get back on the earth."

Huh? I thought one of Agile's principles was "people over process": "Individuals and interactions over processes and tools".

I'm reading about Agile and it doesn't look like the religion he makes it out to be, but a handful of good practices. But I'm just learning it from a book and not from one of these Agile-istas he's ranting about.

Pat : I've spent 4-5 months beeing culpabilized, people over my back trying to fold me to enter into the quiet stiff process. I though I should resing, but for a reason I can't figure, they felt more important to keep me collaborating than going out. All this because "I'm not a team player" wich is obviously true, as it is true for at least 50% of the world population. But resources I economised by communicating less were spent learning my stuff : computer science.

Moreover, I think some practices we have (tight control over people, "design sessions" where the boss leads ...) are not from any XP books, but rather in Robert Sutton's one, in the "bad management practices" sections.

I do believe that we have gone beyond tolerable limits of religious fervour and into the realm of the doomed:

What's all this about us non-agile (OK, I'm arthritic) developers missing the ... "long-term benefit of some of its best ideas (unit testing is helpful in many cases)"?

Gosh. I'm so glad that I have this stack of cases where unit testing is of no benefit at all. In fact, I believe that any maniac who insists on unit testing, for any reason at all, is a sure-fire witch and should be burnt at the stake.

... which should be a lesson to all Agile programmers. Except that XP didn't invent unit testing, and to their credit, don't focus on it as a core philosophy. (There are around eleven others, depending upon who's counting, but they all have to be present at the same time. This means, for example, that having "unit testing" without having a "customer" does not make you Agile.)

This is worse than snake-oil. This is turning into the Childrens' Crusade.

Nice posts steve. I look at superstition in a whole differnt way now haha. But if everything we think, that is not proven by science is superstition, then where does that leave religion? Just a thought.

Good post; you've raised awareness and caused people to feel and think.

Re: static styping, have you read Gabriel Garcia Marquez's 1000 Years of Solitude? There's a great scene in the village of Macondo where everyone starts to forget what anything is/does, and so people start putting little tags on *everything*: "This is a cow. It produces milk. This is how to milk the cow..."

Thanks for the great dialogue here, Steve. Some interesting stuff all around.

One of my takeaways has been to avoid letting any product development process become religion. The dogma that agile, or xp, or waterfall, or joe's devprocess, is the only way to do things -- and following it with blind faith -- is a fallacy that companies I've worked with seem to fail to understand.

Yahoo had an interesting course, which I experienced first-hand. No real process when the company first started, which worked great. Once the bottom fell out, we went to some pretty religius waterfalling approaches. Once we slowed down to the speed of a brown slug, with Goog eclipsing us, we went to Agilee whole-hog. Every step of the way, it was beat into the teams that you must develop products this way or they will not launch -- even when we didn't know the process fully, and had to hire outside Agile Priests or McKinsey Bishops to help the lowly product teams learn from their wicked ways. Maybe that's more of a sign of desperation than anything, but it doesn't seem dissimilar from other friends I talked with at places like eBay, Netflix and a host of other smaller online companies.

Since then, the start-ups I've worked with have tried to swing somewhere in the middle of the pendulum. I like how you've characterized the Google environment, as it seems to take some of the best elements from a variety of philosophies.

I hope it's a good lesson to other folks that they should adapt and adopt to what's best for them. That doesn't necessarily mean Google or Agile or Whatever is bad -- just to remember to shrink to fit.

But there ARE good ideas in Agile. At least reading about Scrum/XP might give people some ideas they may want to try out in their own team.

You talk about Agile like its all a waste of time and doesn't work at all - but at the same time what you described about how Google does things itself includes some of the same ideas in Agile like being "lightweight on processes" and "adapting to needs of the team".

Its like for example Religion A has many teachings, one of the teachings says meditation is good for your mind. Person A is a follower of Religion A so he does meditation. Person B says Religion A is trash, doesn't work, doesn't believe in following it, but at the same time Person B likes to do meditation which he learned from somewhere else - he doesn't even know Religion A teaches about meditation because he never really practiced Religion A.

Now Person C has never tried or heard about meditation before. He has just heard about Religion A and is a little curious. He reads Person B's blog that Religion A is trash, is a waste of time - so Person C stops persuing learning about Religion A and goes back to Religion Waterfall. Too bad for him that he didn't get to learn about Religion A because if he did he would learn about meditation and it would have been a great part of making his life better even though he wouldn't like some of the other ideas in Religion A so wouldn't have followed all of Religion A. He would also have told many of his co-workers about how meditation is good. Instead he tells his co-workers that Religion A is a waste of time and so they all go back to Religion Waterfall.

So basically, just trashing Scrum/XP like you did isn't doing any good but probably is causing more harm. Anyone can write a post trashing Agile (but admittedly probably no one could have wrote it as crazily as you did).

And even though following an Agile methodology blindly isn't a good idea, but there are good ideas in there that can expand thinking about project management just by reading about it.

Forcing a religion on someone else is bad I agree, but if people are smart enough they would learn to choose the good teachings that apply to them from the religions.

And sometimes, for people for some reason need to follow a specific religion by the letter, it cou;d be that Religion A may be better than Religion Waterfall. Science can't prove it (or hasn't proven it yet) that followers of Religion A on average are happier with their lives than Religion Waterfall. But some people who have tried both can give an idea.

For example, if you know more people personally that converted to Religion A from Religion Waterfall than from Religion Waterfall to Religion A .. then maybe you could draw a conclusion. :D

But the point still stands that both Religion A and Religion Waterfall may have good teachings, so dismissing any of them before actually trying to understand them might not be smart. And focusing on the bad points of the Religion and then dismissing the whole Religion as useless doesn't do much good.

For the people who don't want to waste time with religions anyways .. well no problem :)

I'm just saying that I agree that following religions blindly is bad but sometimes religions are useful because they have teachings that *smart* people can choose to adapt to their own life.

Hope you haven't got bloggers block again, no posts for over a month. I love it when people actually think on what they do instead of simply following the rules. Remember rules are not meant to be followed blindly. They're meant to be followed after you decide they are actually good. And even then you need to think about ways to improve them without making them overly complex.

Perhaps just post holiday recipes until you are unblocked. "Deep fried yams with plum sauce is always a favorite at the family table", etc.

Then you can do something really foolish like declare that only complete morons use imperative programming (though admittedly that's still not as inflammatory as your poking the Agile guys with a stick).

Or have you been relocated to a concrete pile supporting the Loop in the agile version of Chicago? (Which I believe is on Interstate 44, right by Lake Arcadia. Last time I was in Oklahoma, there was a wonderful German restaurant which would probably cook your gonads in a really nice, satisfying cream source and let you have them as a last meal before passing you on to the, ahem, contractors.)

Not that I'm criticizing agile methodologies, or indeed Oklahoma, of course.

What was that crap about "agile with a big 'A'"? Have I missed something? Is there another movement on the way here? I mean, I did get rejected for my last contract job for "knowing the technical side of the job inside out, but we don't feel he'll be able to get up to speed on the agile process."

Hee hee.

Teeny question here: if it takes too long for a neophyte/member of another religion/albino dwarf addicted to clove toothpaste to get up to speed, does this make the process agile?

'They just work hard, they stay lightweight, and they ship great stuff'.

Sounds like agile to me...

Index cards and suchlike are an implementation detail. They're handy because you can rip them up if you get the requirements wrong and move them around if the priorities change. Most ThoughtWorks projects also have an electronic copy, because that's just common sense. A good implementation of agile looks remarkably like common sense. Do more of what adds value, and less of what doesn't. Reflect, measure and repeat. Job done.

Interesting article but I agree with Patrick that there is no proof or anecdotes of what and why in Agile doesn't work but yes religious zeal in implementing what you think is agile can be bad because there is a lot of stuff subject to interpretation and saying your interpretation is the right interpretation is always problematic.

I am amazed how no one has mentioned Life Cycle of a Silver Bullet, which is an excellent article explaining what agile is going through and how Google Method is best kept unpublished.

You seem to not control your bloger's block syndrome. What a pity - I'm waiting for a new article :)However I totally disagree with your point about agile - looks like you just don't want to know anything real about it. this holy war thing is fun however.

And I'm glad there are places on this Earth where there's no calendar pressure - thank you for showing the light :)

By the way, in new C# (3.0 for the moment), there is quite clever (imho) approach to the problems of static type systems. It emulates a huge deal of DL's by the means of type inference, automatic collections, anonymous methods/classes etc, but in the context of a given method only. It seems that in this context DL-like approach is really useful (read: time-saving, both in programming and reading programs), while on the class level static approach does better.

and thishttp://www.shmula.com/159/scrum-at-yahooThe New York Times published an article comparing Yahoo! and Google’s products and their development times. It was an interesting read. In that article, the Ash Patel, Chief Product Officer at Yahoo!, mentioned Scrum as a method used to reduce development time:

What the Times doesn’t say is that Yahoo! is now 18 months into its adoption of Scrum, and has upwards of 500 people (and steadily growing) using Scrum in the US, Europe, and India. Scrum is being used successfully for projects ranging from new product development Yahoo! Podcasts, which won a webby 6 months after launch, was built start-to-finish in distributed Scrum between the US and India) to heavy-duty infrastructure work on Yahoo! Mail (which serves north of a hundred million users each month). Most (but not all) of the teams using Scrum at Yahoo! are doing it by the book, with active support from inside and outside coaches (both of which in my opinion are necessary for best results).

Meanwhile, Yahoo says it is now trying to emulate Google’s faster method of creating products. Like most big companies, it used to develop software by first creating a comprehensive design that defined how features would be written and tested. Instead, it is now trying what is known as a scrum method, where it will plan, build and test parts of a product every 30 days.