The smart folks at JetBrains just made our lives a lot easier. TeamCity 4.0 has an extra verification stage that ensures any changes actually work BEFORE allowing those changes to be commited to the repository.

Most continuous integration servers today (CC.NET) rely on teams to commit their changes to version control first, and then kicking off an automated build. And as long as the team members do an update and build locally before checking in, things are usually fine.

Being the humans we are however, builds still periodically break.

TeamCity 4.0 apparently fixes that.

Now, instead of monitoring the repository for updates and then doing a build, TeamCity does then builds first, and then commits the changes to version control only if the build succeeds.

This way, the build is always up and running, and if there is a problem, it never makes it into the repository.

JetBrains refers to this as a delayed commit and I think it is a fine refinement on the continuous integration process.

For more information on this and other features (like distributed builds), check out the TeamCity 4.0 web site.

I just watched an excellent video of Pixar’s hiring practices with Randy Nelson (Dean of Pixar University).

Two core principles they use when interviewing come from the world of comedy improv:

1. Accept every offer

2. Make your partner look good

Accept every offer means if you’re doing a skit as part of an interview, and your partner says: “Boy it sure is raining in outside.” You don’t look up at the ceiling and say “No it isn’t.” You reply, “Well that’s why they gave us these umbrellas!”. You accept what your partner has offered, and do you best to make them look good.

Depth

Pixar likes people who are very good at certain things. The problem is that a lot of the things they would like people to be good at have never been done before and require a lot of innovation.

NASA had the same problem when they were looking to find people to walk on the moon. No one had done it before. And they realized that looking at resumes purely for depth wasn’t going to get them their candidate.

So instead, they look for mastery. People who have truly mastered things in their lives demonstrate a mindset, and level of commitment that is proof they can do it again in other areas. It’s not a promise. It’s not an intention. They’ve done it before and then can do it again.

Breadth

The other thing Pixar looks for is breadth. Pixar likes to see breadth in their people because a broad range of interests means that people will have a much better chance of communicating with people they need to collaborate with.

A programmer who’s interested in art is going be able to communicate his or her ideas more clearly when collaborating with an illustrator.
Watch the video

If you’re interested in learning more about one of the world’s truly innovative companies, watch this video (it’s only 10 min).

Life as a software consultant is pretty good. You come into a company, build an application or two, and once the project is over you climb back up onto your horse and ride off into the sunset.

* You aren’t asked to hang around.
* You don’t have to worry about any pesky bugs or support issues.
* You get to work on new greenfield projects.
* You often work with the companies best and brightest.

It’s great fun and life is pretty good.

It wasn’t until I stopped being a consultant however, and starting building and maintaining my own applications, that I realized I was missing out on a big opportunity for learning.

What you miss

When you build something, and then don’t stick around to maintain it, you are only watching half the movie. You don’t get to see how it ends.

* You miss out seeing where you got the design right – and where you got it wrong.

* You miss out on seeing which areas you tested well – and those not so well.

* You don’t get to see which features your customers love – and which ones they hate.

You basically miss out on all the great feedback that would tell you where you kicked butt, and where you screwed up. All of which would of course help you on your next project.

And that’s a shame.

So for those of you building apps and thinking about moving on – slow down.

See how the movie ends.

Your future customers will benefit from your broadened experience, and you will be stronger for it.

Last week, I showed you how to host your own Bug Hunting Party for developers. Well this week we hosted one for business, and while the feedback was completely different, both were valuable tools for improving the quality of our app.

Let me explain why having both outside developers and business trying to break your app is the best of both worlds.

Developers, developers

A lot of people think developers make poor testers. I think that’s rubbish. Some of the best testers I know are ace devs and I would pay cold hard cash to have these guys come and try to break my app. And that’s exactly what we did last week with stellar results.

You see outside developers (those not familiar with the business of your application) have great technical insight into the technology behind how your app is built. They will know what types of exploits you’ll face, what kind of validations you will need to do, and better yet be able to offer suggestions on how to handle any problems they find.

It’s like they are the security guards at the bank with the inside track.

They also have the added luxury of not really knowing how your app works from the business side. That’s an advantage because they won’t necessary start trudging down the happy path. They will take detours. They will try and do things you never thought of. You can watch them, see where they struggle, and listen for things that are confusing or don’t make sense.

Taking care of business

Business users on the other hand brings an entirely different perspective to testing. They don’t care about technology per se. They care about whether the bloody things works!

As a developer, you can stare at your application for months on end, and not see the forest through the trees. Sit someone from business down however and they can tell you in about 30 secs the 10 things that are wrong with your front page. That’s why we always have business test.

And data. Boy do they know data. When you load up you application with production data, business can tell in an instant if things are working correctly. They have a feel and touch built up that only comes with experience.

You need em both

Of course the ultimate scenario for bug hunting is when you get both parties (developers and business) trying to break your app. Most teams focus on the business (which is the right thing to do) but don’t forget that outside developers can make fine testers too.

So if you’ve done a Bug Hunt with one side of the family tree but not the other, consider inviting both. Developers and business together each bring something different to testing applications, and together form a great one two punch for improving the quality of your application.

Not all agile projects have the benefit of a full time customer, or dedicated QA resource. Yet without their feedback, we still expect our teams to produce bug free, easy to use software. The Bug Hunting party is a fun easy way to get more eyeballs on your application, and give you the feedback you crave. With three easy steps, I am going to show you how to turn a slow, boring Friday afternoon, into one of the greatest feedback sessions your project will ever get.

Not enough feedback

XP and Scrum make it very clear. The success of your project is directly proportional to the input you receive from your customer. Yet many projects can’t/won’t/don’t give development teams full time dedicated resources. The result is that despite the teams’ best efforts, and all our agile software engineering practices, we still somehow manage to find bugs, usability issues and other improvements that somehow evade us during development.

The Bug Hunt

To give your team the extra feedback boost you need to find these low hanging bugs and pieces of usability fruit, consider hosting your own Bug Hunting Party.

Bug hunts are fun, team building exercises, designed to find many of the little hidden bugs, and design usability gems lingering and trapped in your software.
You may not see them. You may even doubt they are there. But when you put your software in front of people, and ask them to use it, you’d be surprised at what they find.

My partner, Arthur Tam, and I just hosted our second bug hunting party this Friday. The results were amazing and there is no reason why you can’t host your own.

Here’s how we did ours.

Step 1. Invite the hunters

Invite as many people as you can. The more the merrier. For us, we pinged all the other developers in our department and dared them to try and bust up our system. The dare is an important part of the hunt. You want people to really try and bust your app, and offer lots of suggestions for improvement. So grease the wheels a bit and throw down a challenge.

Step 2: Host the hunt

Arthur and I like hosting bug hunts on Friday afternoons. It’s down time for most people and they usually don’t seem to have anything better to do anyway.

Snacks are an essential part of any bug hunt. Pop, candy, chips, nuts, a vegetable platter, what ever floats your boat. People are giving up their time to improve your application, so the least you can do is make them feel appreciated by getting some snacks.

After demo’ing the application, and explaining the business, we go over the prizes. That’s right prizes. We want a bit of a competitive atmosphere and make the hunters know their efforts will be handsomely rewarded with giant chocolate bars.

Here are some of our favorite things to reward for:

1st bug found

Most bugs found

Deadliest bug found (most severe, the show stopper!)

Best UI improvement suggestion

With the formalities over, we then grab our notebooks, and start the timer, and let the hunt begin!

What’s the first thing users do?
Where do they start?
Where do they they struggle?
Do they understand your error messages?
Is your validation right?
Did you forget to strip off empty spaces in all your text boxes?

One of the most important things to remember when conducting your bug hunt is to actively listen and not debate. When a user finds a bug, or offers a suggestion, don’t debate. Just write it down and listen. You’re not there to debate your design. You’re there to improve it, and these people are kindly offering you suggestions. It doesn’t mean you have to act on everything. Just thank them for the feedback and move on.

Step 3: Tally the kills

After 30 – 45 minutes invite everyone back and go over what was found.

Were there any areas of your application lacking?
What was good?
What was bad?
What was the number one improvement each hunter would like to see made to the application?

Give out the rewards and thank people for their time.
And congratulate yourself and your hunters on all the bugs you discovered!

Conclusion

Don’t despair if you don’t feel you are getting enough feedback on your application. Create your own. You probably have friends, colleagues, and team mates all around who would love nothing more than to tell you why your kungfu is weak and how your application can be made a whole lot better.

So host your own bug hunt!

If your parties are at all like ours, you will discover a treasure trove of bugs, receive some amazingly insightful feedback, and your application will be a whole lot better for it.

All this, of course, delights customers.

A special thanks to all our big game hunters, and my partner Arthur Tam for use of his camera:

Occasionally I am asked by IT managers to prove, with numbers, that agile engineering techniques like Test Driven Development (TDD) work.

Unfortunately that’s not possible. Academics have been trying unsuccessfully for years, and while I respect their efforts, I would be skeptical of any results proving or disproving that agile methods like TDD work.

The reason I would be skeptical, is because it’s not possible to apply the scientific method to things like TDD.

I remember being taught in grade 7 science, that when doing a scientific experiment you:

Create a hypothesis

TDD is better than non-TDD

You pick a manipulating variable

to TDD or not TDD

You conduct the experiment

Measure the result against your hypothesis.

This type of experiment can’t be done on software projects because the variables can’t be kept constant on both sides.

How many projects have you been on where you have:

the same team

with the same requirements

where you build something the same way

and you don’t leverage any learning’s or mistakes made from your previous experience

Dealing with questions like proving TDD works can be frustrating. One one hand you believe it works, yet on the other you can’t prove it (at least the way some managers would like you to).

So stop trying.

Accept that you will not be able to answer the question the way they would like it answered.

Instead, try looking at the question a different way.

Why they are asking the question in the first place?

Is the customer unhappy?
Did the last push to production not go smoothly?
Are projects costing more and taking longer to deliver?
Is there a concern teams are spending too much time testing?
How are we defining success?

You can’t discount the managers question. It’s a good one, and they very well be onto something in asking it.

Just understand that you can’t always answer every question the way the questioner would like. Especially when it comes to quantifying knowledge work like software development.

The presentations and speakers were excellent. The conference material was well organized. And the most stress many of us felt were trying to decide which sessions to attend (a good problem to have at any conference).

One session, Agile Playground put on by Michael McCullough and Don McGreal, focused on using games to teach participants about agile. This session was excellent because it provided trainers and coaches with lots of good ideas on how to help people new to agile, experience and feel some of the practices and principles that are sometimes hard to put words too.

Not only were the speakers professional and well prepared, they were also generous enough to document many of their games on index cards participants could take away.

If you are into agile games, and are looking for ways to teach agile to colleagues, be sure to check out Michael and Don’s website.

In general, when you try to apply the TPS, the first ting you have to do is to even out or level the production. Leveling the production schedule may require some front-loading of shipments or postponing of shipments and you may have to ask some customers to wait for a short period of time. Once the production level is more or less the same or constant for a month, you will be able to apply pull systems and balance the assembly line. But if production levels – the output – varies from day to day, there is no sense in trying to apply those other systems, because you simply cannot establish standardized work under such circumstances.

So says Fujio Cho, President, Toyota Motor Corporation.

Jeffrey Liker’s book, The Toyota Way, Jeffrey explains how it’s not possible to run a lean operation using Dell’s build to order model. One day you will have many customer orders, forcing your staff to work beyond capacity. The next there will be much fewer orders (and your people will be idle).

Toyota found that to work lean, they needed to even out production.

They do this by focusing on the three M’s of waste:

1. Muda – Non-value added work

2. Muri – Overburdening people or equipment

3. Mura – Unevenness

When most people start with lean, they tend to focus on Muda because it is the easiest to identify and eliminate. Unfortunately, fixing muda by itself can cause greater stress in the organization as now spikes in customer demand forces people to work harder. People become overburdened, equipment breaks down, and people abandon the lean initiative.

As Taiichi Ohno describes it:

The slower but consistent tortoise cause less waste and is much more desirable than the speed hare that races ahead and then stops occasionally to doze. The Toyota Production System can be realized only when all the workers become tortoises (Ohno, 1988).