Happy Holidays everyone. I hope the new year brings everyone more work and less stress.

Now that I am back from my vacation I will resume my posts on the Defect Tracking Patterns language. We've come a bit over halfway, and I've gotten some good feedback. Thanks to everyone who has commented!

Fixed is not Closed

A simple but often overlooked distinction in tracking bugs in software. After the code to fix a bug is checked in, it still isn't really done until it is verified. In the context of the development team receiving feedback from the customer, programmers who work on bugs need feedback to know if the changes fixed the bug as reported. Those changes need to be verified by a third-party and the work on the bug brought to resolution.

Therefore
A fixed bug is not a closed bug. Once the programmer has made code changes and checked in the results, that programmer can mark the bug fixed, but cannot close it. The customer who entered the bug is responsible for verifying and closing the bug if it really is fixed. The resolution of a bug should be different from the status. "Will not fix" or "Unable to reproduce" are among the other resolutions besides fixed that can close a bug. The dispatcher should be especially alert to resolutions other than "fixed", there may be additional information that can change this.

The customer who submitted the bug should close it, or, if there is an dispatcher role, that can be where fixed and closing are coordinated.

I'm on vacation until after the first of the next year, woohoo! I'm leaving today to visit my family in Houston, so I'll probably not post again until after Christmas. This one will have to hold you until then.

Identifying and fixing bugs involves several steps to coordinate as the bug changes states and assignees. Any team member with a stake in a bug needs to know what stage the bug is in at any given time. The customer will look to see when it has been fixed, the dispatcher will need to be able to determine when the bug needs re-assigning, and programmers will need to know when a fix is tested and accepted.

Therefore

Define and follow a life-cycle for bugs.

A bug can only be assigned to one person at a time. That bug may flow back to the reporter with a misdiagnosis or fix that doesn't work. The assignee may assign the bug to another programmer more familiar with the fault or for review. It may go back and forth from reporter to programmer a few times before it is fixed, retested, and resolved.

A quick note to bjf, I also recently bought a Nikon FM3A and I love it. I chose it because of it's classic design and reliability, and I'm glad to see not everything is going digital.

The next pattern in Defect Tracking Patterns is the one that has generated the most commentary so far, often because of confusion over the terms.

Differentiate Priority and Severity

The effect of a bug on the software does not automatically correlate with the priority for fixing it. A severe bug that crashes the software only once in a blue moon for 1% of the users is lower priority than a mishandled error condition resulting in the need to re-enter a portion of the input for every user every time.

Therefore:

Define a range for Severity as "how bad does it hurt?" according to project requirements. One example of a range in use is

Define Priority according to business value and project timelines. High priority bugs should be first to be fixed.

Importance and Destructiveness are alternate words which may be clearer to some.

An example of the difference and interaction is in bugs that are purely cosmetic problems, misspellings in dialogs, redraw issues, etc. These can be priority 1 fairly often, because they are frequently very annoying to users, and fixing them is generally easy and doesn't destabilize things.

Track priority and severity separately, then triage appropriately. It helps to have input from others on the team on priority. The importance of a bug is a project decision, different from the bug's perception by the Customer. In some cases it makes sense to track Urgency, the customer's point of view, separately.

The Developer, when determining what to work on next, would pick the bugs with the highest priority, regardless of severity. The Dispatcher is responsible for setting these appropriately, especially in the case where the Customer marks everything Very Severe and Highest Priority.

Extreme Programmers may see a resemblance to Story Cards and the Planning Game.

The next pattern in the Defect Tracking Pattern Language deals with the context of a development team that may be geographically dispersed or otherwise not able to sit together in an XP way, but has a sense of collective or at least weak code ownership. The product being worked on is large an complex enough that a defect tracking tool is need and used.

Untracked bugs can lead to wasted or duplicate effort. The wrong programmer attempts to fix it or multiple programmers all work on different solutions at the same time without coordination. Or a programmer works on an already-fixed bug not knowing that the fix is in. The bug isn't really a bug, it's a new piece of functionality, leading to scope creep. Low priority, easy, or "fun" bugs get fixed before the higher priority bugs.

Therefore:

Define and use a bug lifecycle. Assign bugs to one person at a time, and give that individual responsibility for resolving it or assigning it elsewhere.

As the customer, watch your bugs and respond to queries for additional information or candidate fixes. If a bug you reported is no longer an issue, close it. If you are using a bug tracking system that allows it, you may watch bugs you do not own but have a stake in. As the developer, update your bugs when you work on them. If you get stuck, assign them to another developer. If you need additional information or want comments from the submitter, assign it back to them with appropriate status.

Identify someone to play the Dispatcher role, and modify priority and assignments as necessary. The Dispatcher role can streamline the process of moving a bug through its lifecycle.

However

Adhering tightly to this pattern requires discipline in using the bug tracking system. The Use a Tracking Tool and Document Work patterns are intimately tied to this pattern's effectiveness.

Next on the list of Defect Tracking Patterns is the bug tracking tool and usage as a kind of history-keeping system. At the moment it's one of the weaker patterns -- comments welcome.

Document Work

A bug report has a life and grows with changes. Bug state is not static, and tricky defects can take several iterations of work to find and fix properly. In the context of this ongoing work, knowing what changed and why, and what worked or didn't, help maintain a sense of history to the code.

Therefore

Document all work on a problem in the bug tracking tool. Use the description and comments field to make note of important issues and facts. Attach files with error output, screenshots, or even snippets of code patches to document what was done and why. Records of what was worked on and resolved represent a knowledge base.

The related idea of micro-decision awareness asks only that when making a decision, however small, you are aware of the fact. Recording that decision in the bug tracking software preserves that decision.

Sorry for the silence, Wednesday I went home early and I've been home since. I picked up a wicked cold and haven't been able to think straight.

Thanks robocoder for taking a look at the whole of DefectTrackingPatterns, I welcome your comments. Until I get my head clear, here's a short list of my brain dump of things that a good bug tracking system ought to have.

Users can split items (e.g. when one item actually describes two separate issues) or combine them (e.g. when several contributors submit identical items)

Users can associate items in groups and dependencies.

Old discussions or issues archived, indexed, and searched. When and how does the system and/or its users identify submissions or discussions that are re-raising old topics or issues?

Issues, and changes in the state of issues, are connected to the use of other tools, such as checking source code in or out

Users are able to perform intelligent searches on issues, i.e. something more sophisticated than searching for keywords in the bodies of issues

URL-style link to defect in other documents, for example http://bugzilla.mozilla.org/show_bug.cgi?id=100000

Allow defects to have dependencies on other defects, use to create "tracking defects" and "release requirements" defects that refer to other defects that must be closed before the parent can be closed.

Allow users who are neither the submitter or assigned developer or admin to put themselves on a list of users to CC email on changes (watch list)

Clearly differentiate between defects and RFEs

Able to identify and push issues that are really support questions to the appropriate persons/tools

Use with any web browser on any O/S

Allow anyone on the email-on-change list to adjust which events they wish to get email about; then the system can by default be configured to email on all non-trivial events and users can turn the noise level down

Another installment on Defect Tracking Patterns, this one discusses how an automated bug tracking tool can help programmers and other development team members keep tabs on what their assigned tasks are.

Defect Tracking As Automated TODO List

Programmers and testers need a way to know what tasks they are responsible for and what priority those tasks have.

Therefore:

The defect tracking system should have sort and filter features that allow maintaining an automated TODO list. Each item in the bug tracking system is assigned to one person and has a priority associated with it. The list of bugs becomes a personal, automatically updated and prioritized, TODO list.

As a developer, filter on Assigned To to see your responsibilities. Sort by Priority to track the top problems that need to be worked on first.

The Dispatcher's role in assigning and prioritizing bug reports helps to keep things flowing during the development process.

Post-it notes, programmer's memories, even email, are all unreliable repositories for bug reports and feature requests. All of them are also difficult to coordinate on a team of more than 2 people working together for more than a few days.

Therefore:

Use an automated tracking tool. The tool doesn't have to be sophisticated -- it can be a spreadsheet in simple cases -- but it must be shared.

Bug tracking software has features to help manage work. Users can create filters and sorts to assist workflow. Filter on the field Detected By with your id to show the bugs you've filed. Filter on Assigned To to see your responsibilities. Sort by Severity to track the top problems.

IBM is going to acquire Rational Software for about US $2.1 Billion. Stories are all over your favorite web sites, like this Washington Post article. How will this affect the Java/OO/Web software development world? Discuss amongst yourselves.

The common context for all these defect tracking patterns is that the bug database and the defect tracking system are internal, and used by the development team members, which may include someone in a customer-facing role who takes reports via the telephone or email and enters them into the system. A general-purpose customer support problem reporting system is outside the scope of these patterns. Some aspects of defect tracking within the development process are either too low-level or irrelevant to the general user population.

The patterns frequently reference four roles that development team members may play. These named roles have specific definitions within the patterns.

Customer

The Customer role may be played by a real end user customer, but more likely it will be a QA Tester, or another team member. The individual entering the bug report is the system is playing the Customer Role.

Programmer

The Programmer is the role taken by the person who is assigned to write code or otherwise modify system to fix the defect. A Programmer would at a minimum unit test the fix, and run any unit test suites, and may, depending on the team, run regression and integration tests.

Tester

The role of Tester is to verify the fix and validate it against the bug report. The same person may take both the Programmer and Tester roles, but the goals are different. For the Tester, the implemented fix is not considered done until it provides a satisfactory resolution to the bug report.

Dispatcher

The Dispatcher has the role of setting priorities and assigning bugs to be worked on, then following up on work. The Dispatcher also coordinates the life cycle events of a bug report between various other roles.

Keeping these roles in mind while reading the patterns is an important aid in understanding them.