December 20th, 2011

The goal of the program is to take a deep look at how people are currently engaging with Bugzilla. We want to see what we can learn about the tool’s good, bad, and frustrating qualities. You can’t look at Bugzilla without understanding how people work. The research is also surfacing information on the life cycle of bugs and how people interact with them along the way. The primary goals are as follows:

Identify the information we want to gather.

Interview stakeholders across the organization and community.

Surface statistics that are relevant to its use.

Identify best practices, useful hacks, tools, and key problem areas.

Provide an overview of how Bugzilla is being used.

Complete the research by the end of January 2012.

How are we going about doing this?

In an effort to get to the root causes of issues and find the hidden best practices, we are gathering data in two ways. First, we are conducting 1 hour interviews that probe into various topics focusing on how people interact with bugs throughout their life cycle.

The second method is statistical. We are diving into the considerable dataset we have stored in Bugzilla’s historical log. We hope to surface trends that help point out problem areas where we can improve as well as shed light on what’s working well so we can highlight those successes.

What have we learned so far?

We have just completed 10 interviews and posted the raw transcripts last week. We held off posting this material until we felt we had the absolute minimum representative coverage. The transcripts are raw feedback, they have been verified by the contributors, and they have all volunteered, in true Mozillian fashion, to put their names on them. Thank you to all the brave contributors that went first in the hopes of improving things for all.

We have also been working with Metrics to gather data on Bugzilla usage. Much of this data is still behind layers of security as the tools that help visualize trends are still prototypes. Once the tools are ready, the security group will need to review the system as it includes statistical information on security bugs. We will need to work with them to double check that we aren’t leaking out sensitive material before reducing the security to LDAP or preferably making it publicly available.

How can people contribute?

The best way to help is to review the data, give us feedback on a new set of interview candidates, and anything new you think we should be exploring. We intend to do another 10 interviews as the next phase. We will be including more community members in this round as they are currently under represented. The current list of questions we are asking and who is on our radar as candidates can be found at the following Etherpad. Feel free to add suggestions here:

Once we start to surface the statistical side, I will write up another post.

If you have any questions you would like to see added to the interviews, have any feedback on the project or results that you would like to share, you can find me on IRC in the #pm channel, email me at mbest@mozilla.com or comment below.