How hard is it to write a bug tracking system that doesn’t suck? Quite hard it seems …

I spent ten minutes trying to work out how to enter a bug in a Flyspray-based BTS. And I gave up. I gave up entering a bug! Isn’t that, like, the most important single task that BTSes should enable me to do?

Trac is a pile of crap too. The enter a bug button is hidden away somewhere (it’s called “New ticket”), and the main interface seems to be arranged around producing arbitrarily chosen “reports”, none of which I have ever found to be in the slightest bit useful.

And .. Bugzilla. Oh Bugzilla. When I get my own starship, I hope it’ll be simpler to use (and faster) than Bugzilla.

I would encourage everyone to try out (proprietary) FogBUGZ for an example of a usable BTS. It’s proprietary software, but you can learn from it. You can learn that (a) most of your users won’t be BTS experts, and (b) you should encourage as much feedback from all your users as possible. That means doing usability tests and concentrating mainly on making it as easy to use as possible. It’s the most important thing for BTS authors to do. Clue: if your users cannot work out how to enter a bug, then you should go back to the drawing board.

27 responses to “OSS bug tracking systems all SUCK”

The default set of reports in Trac is not expected to be useful, but just to serve as examples of what reports you can set up, the site admin is expected to set up the reports (IMHO a classic case of “dear user, please write the program for me” soft coding, but whatever). Sadly, most projects never bother customizing their reports (which should serve as a hint to the Trac project that they need to focus on sane defaults more than customizability). For a basic set of Trac reports that makes sense for most projects, see e.g. https://www.calcforge.org/trac/calcforgelp/report (try clicking on “SQL Query” for each report to see how I set them up).

I think that most OSS BTS around think about power-users and forget about simple reporters.

I must had some other BTS:
– debbugs: fully mail oriented, quite hard to use for beginners
– roundup: probably the easiest, but quite close to trac

I also worked with Serena Teamtrack (proprietary), and it is at least as complicated as bugzilla.

Most of the time people writing BTS try to make it as flexible as possible because bugs workflow is not (yet) well defined. It depends on how you handle bugs and many other factors: do you have someone that dispatch bugs for you, are you working alone or not…

I think, if someone is willing to create a BTS, then he must begin by avoiding flexibility and makes a single workflow mandatory. He can then create something easy to use.

Isn’t Launchpad open source now? I’ve no idea how easy it is to set up, but it does have a much cleaner interface than some of the older projects that refuse to reform.

The silly thing is, all these projects just need one extra person: a user-experience/front-end web designer-come-developer. There are literally thousands of people they could induct (or even pay) to get a much better interface.

The problem is that I don’t have a choice over the BTS that <some random project> uses. So while I could install this for my projects, it’s all the other projects I report bugs to that I really care about.

Not OSS either, but I had the impression the one on google code [1] was also a step in the right direction. Especially the use of labels which avoid huge forms with useless fields but still allows you to define your own bug organization structure. But I didn’t end using it, so I don’t know how it turns out in practice.

Mantis, failing to look up (without being logged in) an issue I did enter myself says everything I think.

Oh and you didn’t mention gforge. But this is beyond hope, the whole system is bloated by a bureaucracy that 99.999% projects don’t need.

Debian’s BTS is easily the worst of the lot, since it provides no direct interaction whatsoever, forcing you to check the man page for the correct email command format for even the simplest changes to a bug. I recently had to send three or four emails to successfully retitle a bug, because Gmail insists on wrapping long lines and the bug daemon couldn’t deal with it.

Launchpad was GPL’s last year and it’s easily the best OSS bug tracker I’ve seen (though I’ve yet to see it anywhere but launchpad.net). Obviously it doesn’t have the polish of a commercial tool…

I strongly agree about Debian’s BTS. Other OS’es BTSes vary from good (Launchpad) to okay (Bugzilla), but Debian stands alone as being nearly unusable through a web browser. Having to dig through documentation to figure out the format for email just to add a tag to a ticket is insanity…

The state of bug trackers quite a few years ago was even worse, which is why I wrote Anthill (http://freshmeat.net/projects/anthill) quite a few years ago (started it 10 years ago I think). I moved on to other things and Anthill is pretty much unusable now due to it being written for PHP3 IIRC, but it was pretty simplistic and easy to use back then.

I don’t think Bugzilla is terrible, really, once you get used to it. It does have significant issues when used for a distribution, though, as it was designed to be used for a single software project. It and Launchpad are the two I cringe least upon seeing. Trac tends to be a nightmare, I agree.

I think something that was a combination of the best ideas in BZ and Launchpad – and, of course, 10x faster – would be great. Go write it! 🙂

Well it’s very slow, but the significant problem is the one you allude to just there: I don’t think Bugzilla is terrible, really, once you get used to it. You mustn’t discourage feedback about your project by having an unusable BTS. It must be simple and effortless to submit a bug.

Many of those bugs submitted will be spam/crap, but that is our problem as package maintainers, not the problem of the volunteers testing and giving feedback.

It’s due to the size of the component list. That’s not a problem with bugzilla itself (bugzilla itself is quite snappy, actually). I’m not an admin on there, so can’t even guess, but next time you’re in there, look at the product list. It’s huge. Then look at the component list for a given product (let’s say Fedora or RHEL). They also are huge. And I mean _really_ huge. Probably on the order of thousands of components per product (not all products, of course, but at least for RHEL and Fedora).

So you’re looking at, overall, a few thousand components, maybe close to a hundred products (completely off the top of my head)… yeah, I’m not surprised that the Red Hat bugzilla is slow. In fact, I’m impressed that it isn’t _slower_, all things considered. =)

I wrote an article about this same frustration at “Bug Trackers: Do They Really All Suck?” (O’ReillyNet) http://lwn.net/Articles/163500
They are are applications that are used by many kinds of people – developers, QA, project managers, the public, and this pulls them in many directions.

I find that the tool I consult on (Atlassian’s JIRA) is flexible enough to make most people happy (80%-90%). It’s still good value for money ($10 to a few thousand for most companies). It’s not OSS but you get the source code at all license levels.

About the author

I am Richard W.M. Jones, a computer programmer. I have strong opinions on how we write software, about Reason and the scientific method. Consequently I am an atheist [To nutcases: Please stop emailing me about this, I'm not interested in your views on it] By day I work for Red Hat on all things to do with virtualization. I am a "citizen of the world".

My motto is "often wrong". I don't mind being wrong (I'm often wrong), and I don't mind changing my mind.

This blog is not affiliated or endorsed by Red Hat and all views are entirely my own.