Posted
by
timothy
on Tuesday February 18, 2003 @10:30AM
from the don't-break-your-nose dept.

bADlOGIN writes "Anyone and everyone on Slashdot probably knows that business-driven software development efforts all too often end up as a mess. After a number of years of observation, research, and fine tuning, Ken Schwaber and Mike Beedle have released a book that makes a subtle but vital revelation about the nature of software projects and how to better run them. Learning what Scrum is and how to practice it is not all that profound. However, sitting back and realizing why Scrum works and how it addresses the fundamental flaws of the last 20 years of software engineering is. This book could be viewed as the "why" component to all of Extreme Programming's "how."

Agile Software Development with Scrum

author

Ken Schwaber and Mike Beedle

pages

158

publisher

Prentice Hall

rating

9/10

reviewer

bADlOGIN

ISBN

0130676349

summary

This book could be viewed as the Why component to all of Extreme Programming's Hows. It explains managing software development as an empirical process.

What it's all about:

Books that claim to hold the keys to developing software the right way are most often: a) a dime a dozen, b) self-serving vendor drivel, or c) all of the above. While this book is fairly new on the shelf (Copyright October 2001), it has a level of research, professionalism, and effort towards being tool- and language-agnostic that may place it in a fourth category of being: d) none of the above. Agile Software Development with Scrum is a complete picture of the Scrum process from theory to practice in real life examples. The name Scrum was chosen because the process is similar in nature to rugby play where success is built upon being quick, adaptive, and self-organizing. The target audience for the book is "executives, software managers, project leaders, and programmers." While the authors make no assumptions directly, being familiar with Extreme Programming, "classic" waterfall methodology, and having hands-on experience with the chaos of software development is indeed helpful.

The primary theme of the book is simple, but the impact is profound: software development is an empirical rather than a defined process. That's a nice big sweeping claim to make: fortunately, the authors spends a lot of time making sure that you as the reader understand what they mean by the statement and that they're serious about it. Comparisons to other empirical processes are illustrated with examples of changing, complex problems. The authors seek out and provide unique insights from process dynamics experts on the nature of empirical versus defined processes, and cite profound supporting work regarding the limitations of algorithms in complex systems with external inputs (e.g. Wegner's Lemma).

Along with a good dose of theory, there is a generous helping of practice and how-to. Agile Software Development with Scrum covers the basic practices and concepts needed to manage software development in an empirical fashion. The authors do a good job of addressing the classic "but what about..." questions on the spot in a conversational manner and include anecdotes throughout to make specific points and include full process examples towards the end.

What's good about the book?

Scrum is the missing "why" to Extreme Programming's "how." By it's nature, Extreme Programming incorporates all of Scrum's spirit, and vice versa. This book has a foundation of ideas and an explanation of what it takes to seriously improve the state of the practice of software engineering. The order is reasonable, and the depth of information should give any determined individual the ammo they need to make a change in how software is developed in their current job or their next.

What could have been better?

There are only three things worth mentioning for improvement, all of which could be easily done. First, there were occasional typographical and formatting errors -- places where indentation, capitalization, or bullets were missing broke the flow. Second, the graphics in more than one section were blocky, low resolution circa 1987. And last, the $30.95 list price was a bit steep for 158 pages. It should be noted that the typographical and graphics issues were the only thing that prevented me from rating this 10 out of 10.

Summary

In my opinion, this book has been needed for a long time. The issues and failures of defined processes such as the "classic" waterfall methodology can't be set aside until there is an approach that can justify itself both in theory and in practice to replace it. Extreme Programming has gained much attention, but tends to depend too much on the fact that "it works because it works." Scrum gives you a way to fix your development efforts without as much culture shock or risk. It's worth considering implementing before your competition does.

...While this book is fairly new on the shelf (Copyright October 2001)...

Since when is a year and a half "fairly new"? This makes me wonder if both this book in particular and extreme programming in general are really the Nirvana they appear to be? Is lack of promotion for or lack of substance with extreme programming the reason the Big Guns like IBM and Sun haven't promoted Extreme Programming Consulting services, or have they simply not found a way to co-opt the market yet?

Not to be rude, but perhaps if you feel that way you should spend more time setting expectations. Developers always complain about management (I know I've dealt with some crappy managers), but I think that it's the fault of the industry as a whole for not setting expectations right.

Half the time the problem is the vendors telling management that their product will slash costs by 200% and be implemented in a week for a 10th the price of the competition.

If you're a consultant I'm sure you've seen the ever popular salesman screw job where your sales person doesn't have to guts to tell your client what their development is really going to cost them, so it ends up being done by the you (the developer). That always leads to some fun discussions.

It's not a secret, why business-driven software development so often turns to be a such a mess. The demanding part (management, customers) is just plain stupid

I'd argue that it is our professional responsibility to make management and customers happy, we should drive the process and we should feel responsible for success. Having said that, I'll be the first to say that there are customers out there that you do not want to have (I'm sure we all have war stories about these!).

I just want to make the point that business requirements change, clients (or employers) do not know exactly what they want, and all of that is a part of life. We do not get clearly-defined programming assignments like we got math problems in school. That is why methodologies that embrace change (like SCRUM) are so exciting.

This really isn't a fair comparison. GM's engineers make releases every year at the same time. They tend to make only incremental changes to their previous designs. Because of this, they can only react to changing customer\marketting requests at these intervals. Also, there is little room for manevour after release so things pretty much have to be right before wide release. Patches are expensive, difficult and their need may have killed people. Therefore, serious testing has to have happened before a true release.

Wait a sec....

Incremental and regular releases, insulation from changing requirements, lots of testing... kinda sounds like what the XP guys want us to do with software. We just have to have much tighter iteration periods.

Seriously though. We are talking about differant engineering tasks. Some methodologies might transfer with only a little modification, but most won't.

Every few weeks someone posts a book here about some new "methodology" that will save the world of Software Engineering. Please write this down and stick this on your forehead. Then have your pair programmer read it to you once in a while:

1. Programmers are not engineers2. Programming is a human-centric activity3. There are no "silver bullets"4. Agile methods are useful only to the extent that they remind us of everything Fred Brooks already said.

We could scrum, but that just wouldn't have the same effect as:
Software (yes it is!)
Creation of some bugs
Realization of the fact we have bugs
Objectification of the bugs into modular and reusable parts
Timing of when to unleash bugs
User requirements: They had better require bugs!
Many Iterations to turn bugs into Features!! (MS only needs one iteration to do this)

Or we can just call it SCROTUM. For example, insted of saying: let's SCRUM, you could say: let's scratch out that SCROTUM problem.

After reviewing the first 73 comments on this book review, there are some important background pieces of information missing.

SCRUM was invented by technical people for technical people by a Smalltalk team in 1993. The inventors were well steeped in Brooks Mythical Man Month. The challenge was to get mere mortals to function as well as Brooks Surgical Team, which IBM had shown was the highest productivity approach to software development. The solution came from the Japanese auto industry where they first applied the SCRUM term to new product development. Critics should read:

Takeuchi, Hirotaka and Nonaka, Ikujiro. 1986. The new new product development game. Harvard Business Review 64:1:137-146 (Jan/Feb), reprint no. 86116.

SCRUM is designed to eliminate all project management meetings and replace them with one technical meeting a day about 15 minutes long. It should be led by a technical leader, i.e. someone who does real coding. All decisions are made in the SCRUM.

SCRUM is designed to get management out of the way and to assign them tasks they are responsible to fix, i.e. get stuff out of the way that is blocking team progress. If they can't do that, the SCRUM should replace the managers.

SCRUM is designed to completely eliminate project leaders. Project planning and reporting including complete charts and graphs that will satisfy senior management is designed to be done in less than 10 minutes a day. Development input for project status should be restricted to less than 60 seconds a day. GANTT charts are verboten.

If a SCRUM doesn't do the above, it is not a good SCRUM. Management must give up some control in return for getting a better product faster. Anyone who has seen SCRUM really work will never go back.

Looking back over the past four years I can honestly say XP is the best thing that happened to software development at our company. And this is coming from a grizzled veteran of 22 years.

For last 15 years I've been writing software for company that makes medical instruments. It's a tough market, because the FDA is all over us, and for good reason: if the software doesn't work as stated, people can die.

Up until four years ago we used the waterfall model of development exclusively, then we made the jump to XP. I was a little leery of XP at first, but there is no denying the result -- XP produced the best tested, most rock-solid version of our product ever, with all the features Marketing wanted, and on a schedule Marketing was comfortable with. Smiles *all* around. This, folks, is not nothing.

I don't think XP is for every programmer, but for the right mindset it's a blast. In our particular environment, our team of 8 voted to tear down all the cubicles and instead sit around one large table in a common "war" room, surrounded by whiteboards on all four sides. The table itself was packed with up-to-date PCs networked to common repository. The goal was that there should be no barriers to communications, and it was your responsibility to do everything you could to increase the flow of information. To that end, team member were encouraged to ask each other anything, at any time, simply by shouting a questions and answers across the table. It was similar to a loud, chatty newsroom: lots of conversation, plenty of funny, sometimes sarcastic humor, and some occasional swearing. A word of warning: if you don't have a sense of humor, the respect of your peers, or an ability to work in a noisy environment, you wouldn't last a week on our team; but we screen our new hires pretty carefully for this.

Privately my biggest apprehension about XP was pair programming. I always considered myself a bit of a free thinker who works best independently, and I was certain I wouldn't like it. But I can't argue with the results. Although I'm a fairly adept Java programmer, it's humbling to see how much better the code is with another pair of eyes on it as its being created. On top of that, I got to work with some pretty top-notch folks; seeing others pick apart a problem definitely increased my analytical skills.

At the end of the day, each programming pair checked their work into the repository using their initials as part of the version number; for us, this constituted a code review. No code ownership was allowed, which meant that 1) everyone got a chance to work on the things they thought was cool at least some of the time, and 2) everyone had a shot at improving the code. New unit tests were always written before writing code, and added to the growing suite of unit tests. Before checking in a code module the submitter had to run all unit tests, which by the end of the project took about 10 minutes (over 15000 assertions!). Every night an automated test engine would kick off 6 hours of functional test scripts to exercise the latest build... Every morning started with a ten minute "stand up" (a meeting where no one is allowed to sit down - believe me, this *guarantees* brevity) to review the previous day's progress, the evening's functional test results, any problems/cool things encountered, and plans/pairings for the new day. Overall product progress tracked by everyone on a big wiki web, which was also used to communicate with Marketing and other groups.

What impressed me the most about XP was how satisfied our customer was. I'm speaking of our Marketing department. For once, Marketing finally felt like they were in control of the schedule. This was accomplished not by allowing them to dictate end dates (never works anyway), but by giving them, as the customer, full and exclusive control over the feature list. Our software team worked in four week development cycles called iterations. At the end of each iteration we'd get together with Marketing, show off what we have, and then have them tell us the features to implement for the next iteration. We could not change the feature list, nor argue with it - that was their call; however, they could not change our estimates for how long it would take to complete each feature. The schedule was totally in the customer's hands, at all times, as it should be.

I'm not saying XP is for everyone -- in fact, if you really have profound doubts about the process then don't even bother applying for a position on an XP team because you will *not* be happy, and you'll just bring everyone down). I am also not saying that XP will alway produce excellent products that are on schedule; but what I am saying in the right hands, XP can do ezactly that, and has the pleasant side effects of re-energizing your team and re-establishing everyone else's faith in your team's ability to deliver results.

IT monkeys who havent done any Computer Science is heavily in need of methodologies to hide behind processes, so they can cover up their incompetancies by following a process. I've also found this to be true for managers as well that have gotten out of their engineering roots into managing people. In the process of looking for advancement, these managers have lost their true passion of solving technical problems, and have come to be those they despice; paper pushers, managers that have to kiss ass for their next advancemet, etc. Its these kinds of people that seek methodologies to help them complete a project, and sadly they have forgotten what really matters; the engineers that solve the problems. So, they come up with these methodologies when they hear some one at GM or Ford used a methodology and completed a project. Whe the project finally succeds, because of the hard work by the team of engineers, the insane manager writes up a post on slashdot about how good the methodology is, instead of giving credit to his team that really made the difference. The real truth is that companies like GM and Ford have poured so much money into their projects that they really cannot fail any more.

By contrast, real Software Engineers, would like to think for themselves and excell in their projects because they have a passion for solving real problems, and find themselves hindered by monkey managers who push processes over intelligence. If this is not the truth, open source projects like Linux would not succeed. they dont dictate a process over people, infact they reward people over a process; self fullfilement fo doing something right.