Bug management/status

This page is obsolete. It is kept for historical interest only. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date.

Mark Hershberger reached out to other open-source communities (like Mozilla) to look for best practices in bug management and workflow; he started to experiment with a new "unprioritized" value for the "priority" field. He has also been organizing weekly bug triage sessions at different times to allow for participation from different timezones.

In November, Mark Hershberger and Sumana Harihareswara led themed bug triage sessions focusing on non-MySQL databases, MediaWiki 1.18 bugs, and UploadWizard. A session was also dedicated to reviewing patches in bugzilla; volunteer Rusty Burchfield wrote a tool to check if the patches could be applied to trunk, and only 50 were not obsolete due to bitrot. Mark watched for bugs and comments on local village pumps following the deployment of MediaWiki 1.18 to Wikimedia sites. He also continued to prioritize bugs and find developers to address those of highest priority.

Mark Hershberger followed up on MediaWiki 1.18 bugs, and wrote a FAQ listing issues and offering solutions until 1.18.1 is released. Mark also continued to go through "highest priority" bugs, dealt with bugzilla vandalism, reviewed patches submitted in bugzilla, and held bug triages on MediaWiki 1.18 and Fundraising engineering.

Mark Hershberger has been using the 1.19 deployment cycle to work with on-wiki editors to find and fix bugs as the deployment cycle goes on. Through the connections that he makes, he hopes to use these relationships during future deployments to make them smoother.

The Wikimedia Foundation is nearing the end of its hiring process for a new Bug Wrangler, who will lead triage activities and train volunteers to triage as well. In the interim, volunteers such as Krenair and Thehelpfulone have stepped in to partially fill the gap. Volunteer Matanya Moses is planning to lead an online bug triage meeting, focusing on unreviewed patches, on September 5th.

Andre Klapper, Sumana Harihareswara, and Daniel Zahn discussed the necessary prerequisites to upgrading bugzilla.wikimedia.org, which is currently at version 4.0.6, to 4.2.3. We have 4 possible approaches:

Do nothing right now because it's not urgent, and check again in 3 months. We believe we've hotfixed all the crucial problems between 4.0.6 and now; if that's the case, we will likely choose this option.

If there's an urgent need to upgrade NOW, grab the tarball, unpack it live, and pray. Andre is checking whether our current version as installed urgently needs upgrading, in which case Ops (most likely Daniel) will do this.

Find or create a Debian package suitable for installing on Precise, write the necessary Puppet four-liner, and deploy that way. Andre has some packaging experience and we can get help elsewhere in our community & WMF if we go this route.

Puppetize all of Bugzilla and deploy that way. We probably do not have enough spare time among Puppet domain experts at WMF and in our community to do this soon, so it's the least likely right now, even though ideally it's the best and most sustainable path.

Andre is investigating and will be leading the upgrade effort, no matter what option we choose.

Update 30 minutes later: We've live-upgraded to 4.0.8, which is old-stable (has all the current security fixes), so now Andre is going to investigate how much functionality we want (and how urgently) that would require a major upgrade.

New Bug Wrangler Andre Klapper had many discussions with different stakeholders to get a better impression of how work is done, how people interact with the bug tracker, what the expectations are and what policies might be needed. He investigated the product/component organization within bugzilla, started triaging incoming and older reports, and did maintenance work (creation and partial cleanup of products and components). bugzilla.wikimedia.org was upgraded to 4.0.8 with the help of Daniel Zahn, and investigations started to determine how urgent an upgrade to 4.2 was with regard to functionality improvements. Plans for the next month include improving documentation on bug management and bug triaging, and describing interactions between the bug wrangler and the different teams.

In the past week, aside from the usual triaging, Andre updated some bug triage and Bugzilla documentation, published his Greasemonkey scripts in a Git repository, went through obsolete extensions and updated their BZ descriptions.

This week, Andre will follow up on the process for collaborating with product managers and teams (including expectations and workflows with bugs), and on "undeploying" (removing) deployed extensions from WMF sites.

Andre continued working on standardizing Bugzilla components, documenting the bug workflow, and triaging bugs. He especially investigated thumbnailing and search issues. Andre also started contacting WMF engineering teams to consult with them on how they use Bugzilla.

Andre is also considering an ECT yearly goal of reducing the rate of growth of new bugs or changing the velocity of our new Bugzilla issues growth.

Andre mostly spent the week with Bugzilla gruntwork (adding extensions, people to bugmail, triaging) and continuing tasks from the previous week, such as improving the bug management documentation and keeping an eye on the discussion on standardizing the meaning of "highest priority" in Bugzilla. He also investigated upgrading Bugzilla to version 4.2 by some basic testing.

Andre Klapper improved and cleaned up updated large parts of the bug management and Bugzilla documentation. This includes the beginnings of a triage guide. He also published his Greasemonkey scripts in a Git repository and went through obsolete extensions and updated their Bugzilla descriptions. Andre started analyzing how Wikimedia engineering teams use Bugzilla and their related workflows. He also investigated a potential upgrade of Bugzilla to version 4.2 by doing some basic testing. Furthermore, a wikitech-l discussion on standardizing the meaning of "highest priority" in Bugzilla resulted in creating a new "Immediate" priority status.

In the last week smaller regex fixes got deployed in Bugzilla to fix automatic linking to Gerrit changesets (bug 40344, bug 41321). Discussions took place on wikitech-l mailing list about a "patch in gerrit" bug status, with the ops team about the situation of RT, and with the Wikidata team about automatic notifications (comments) from Gerrit into Bugzilla about patch status changes. Also, the amount of bug reports that are resolved as RESOLVED LATER was decreased by about 100 tickets, in order to get rid of that resolution in the long run, and also a number of unprioritized bug reports received a priority setting.

Daniel Zahn and Andre Klapper upgraded Bugzilla to the latest stable version (4.2.4) which provides higher flexibility for displaying interface elements, improved custom search, better JSON-RPC support and a solid base for future improvements being considered. Andre continued to improve the bug management documentation. Many bug reports that were previously closed as RESOLVED LATER were retriaged and RESOLVED LATER was disabled for future use, and a large number of previously unprioritized bug reports received a priority setting. Furthermore, Andre looked after reports about CSS issues after the MediaWiki 1.21wmf5 deployment and followed up by triaging, creating requested Bugzilla components, etc. Several smaller regex fixes were deployed in Bugzilla to fix automatic linking to Gerrit changesets. A "patch in gerrit" bug status was discussed on wikitech-l with the conclusion to wait for automatic notifications (comments) from Gerrit into Bugzilla about patch status changes first (which is being worked on by the Wikidata team).

Handling some smaller Data Center move aftermath issues; also clarified "patch-need-review", "patch-reviewed" keyword meaning in Bugzilla to explicitly EXCLUDE patches in Gerrit, so search results are less noisy. "patch-need-review" was historically called "need-review", not implying existence of a patch, but also "this extension needs a review before we could deploy it" and other similar interpretations. Also checked if extensions were deployed in the meantime and made requests block bug 31235 whenever appropriate.

As part of the usual work, triaging of incoming reports, investigations on various bug reports (especially datacenter migration aftermath, and thumbnail cache purging issues) and threads on various forums (Village Pumps etc.); continued reducing of the backlog of unprioritized tickets; and Bugzilla maintenance (e.g. adding a Bugzilla component for "Wikimedia Planet") were done.
Andre went through some "extension deployment" requests to check if still applicable while clarifying the meanings of the "patch-need-review", "patch-reviewed" keyword meaning in Bugzilla.
Furthermore, it was discussed how to improve interaction on Bugzilla tickets that need handling by the ops team (which mostly prefers to use the RT bugtracker instead).

This month, a first bugday was held, targeting bug reports which had not seen any changes for more than one year, resulting in about 30 tickets being updated. In addition, some cleanup work (decreasing the number of unprioritized bug reports and going through open reports in "ASSIGNED" status for more than a year) took place. Andre Klapper worked on small Bugzilla code changes and published initial information on Bugzilla usage per development team. Community members were invited to join the MediaWiki Group Bug Squad. Furthermore, some problems due to data center migration were investigated, and it was discussed how to improve interaction on Bugzilla tickets that need handling by the Operations team (who mostly prefers to use the RT bugtracker instead).

Andre continued the number of unprioritized bug reports and reached out to several development teams to better understand their use of "Priority" in the Bugzilla issue tracker and who should set it. Furthermore some triaging of ArticleFeedback reports has taken place. Valerie summarized the latest bugday and published an initial version of a Bug Life Cycle flowchart describing the life of a bug report by its status changes over time.

As part of the Weekly QA Goals, a Git/Gerrit Bug Triage day took place. About 25 open reports were retested and/or synchronized with their status in the upstream bugtracker. The Bug day format will be developed further to make it more attractive to new contributors.
Valerie helped in testing the Commons Upload app for Android and the mobile browser as part of Mobile QA testing this week.
In the context of the OTRS support discussion on the wikimedia-l mailing list, Andre retriaged the open OTRS tickets in Bugzilla, in order to get a better and up-to-date overview of OTRS' bugs.
Furthermore, discussions how to mark bug reports which are fixed in the development branch as backport candidates for MediaWiki's stable release branch took place, potentially resulting in the addition of a dropdown menu ("flag") in Wikimedia Bugzilla.

As part of the Weekly QA Goals, a Git/Gerrit Bug Triage day took place. About 25 open reports were retested and/or synchronized with their status in the upstream bugtracker. The Bug day format will be developed further to make it more attractive to new contributors.
Valerie published an initial version of a Bug Life Cycle flowchart describing the life of a bug report by its status changes over time, continued investigating feedback channels and workflows of other bigger free software projects, and also helped testing the Commons Upload app for Android and the mobile browser as part of Mobile QA testing.
A table on Bugzilla use by development teams was made available.
Furthermore, reachout to several development teams continued to better understand the different bug management needs, and discussions took place about a workflow how to mark fixed tickets as backport candidates in the issue tracker, potentially resulting in the addition of a dropdown menu ("flag") in Bugzilla.

This week, the number of unprioritized MediaWiki enhancement requests in Bugzilla was decreased further, and a patch to enhance the Weekly Bugzilla Report on wikitech-l mailing list by including a list of urgent issues was written. Furthermore, Valerie published a blog post explaining how to create a good first bug report.

As part of the QA Weekly Goals, a bugday took place which resulted in about 90 reports Skin and page rendering bug reports being looked at and commented on.

Andre investigated the globalwatchers setting in Bugzilla and the situation of the wikibugs-l@ account which is used for notifications in other places, such as feeds and IRC. In consequence, wikibugs-l@ was removed from the default CC list of many components for security reasons.

To reduce the amount of bugmail for the QA team, the Bugzilla component "Testing Infrastructure" under "Wikimedia" was renamed to "Continuous Integration" and a new component called "Quality Assurance" was added for Browser testing (Selenium) issues.

Over the weekend, Andre investigated thumbnail cache purging issues brought up in the Village Pumps. He also went through open bug reports with "bugsmash" keyword, retested some of them, and removed the keyword from all of them so the keyword could be used again with its original meaning.

An IRC office hour about Bugzilla and bug management took place (log). Thanks to Ori, a Wikimedia Labs instance to test Bugzilla software changes is now available. Apart from that, the last week mostly consisted of gruntwork: With regard to Bugzilla taxonomy, a number of Bugzilla components (for MW extensions) and a Bugzilla product "Tool Labs tools" were created. Andre tried to debug connectivity issues, reviewed some older ephemeral files on the Bugzilla server that are not in the Bugzilla repository, prioritized a number of previously unprioritized tickets, and had some initial discussions with Wikimedia's Release Manager about improving the bug life cycle workflow (making it clearer when a fix has been deployed).

Andre experimented with Bugzilla's currently useless "guided" bug reporting form, fixed two issues (1, 2) with the "Weekly Bugzilla Report" email sent to wikitech-l, and reorganized the Security-related components in Bugzilla after a meeting with WMF's security engineer.
Apart from spending the last week at the Open Source Bridge conference and giving a talk on bug management together with Mozilla's bugmaster, Andre discussed with Wikimedia's Gerrit maintainers next steps to automatically set a "patch in gerrit" status in Bugzilla and met the Language Engineering team to discuss the UniversalLanguageSelector deployment. He also provided some input and feedback on a potential future blogpost describing the bug fixing process in Wikimedia.
Furthermore, as part of a weekly series on "Bugzilla tips", Andre blogged how to change columns in Bugzilla search results.

In the last two weeks, Andre ran a Bug management IRC office hour (full log here), had a meeting with Bugzilla upstream developers to discuss future plans, created a patch to make Bugzilla's guided bug entry form usable for Wikimedia Bugzilla (bug 36762), kept an eye on SUL2 issues, published a Bugzilla admin policy announcement, and introduced a PATCH_IN_GERRIT Bugzilla status as previously discussed here (with related followup tasks still to perform).

Apart from usual gruntwork (triaging, commenting, reproducing, nagging, following VillagePumps especially discussing the SecureLogin deployment), Andre mostly continued working on Bugzilla code improvements. He fixed a number of Bugzilla issues and prepared patches for some more. He also visited Wikimedia Germany to have a Q&A session on Bugzilla use.

Apart from the usual triaging, reproducing and prioritizing of problems and tickets, Andre spent a week in San Francisco and had numerous meetings and discussions about bug management with Wikimedia developers. In Bugzilla's taxonomy, MediaWiki version 1.21.2 was added to Bugzilla and the numerous 1.17.x versions were merged into a single "1.17.x" version. Tickets under the "Tools" product in Bugzilla were retriaged, and a bug report was filed to think about renaming "Tools" to avoid confusion with Tool Labs. As a side effect, MediaWiki-Vagrant is now a product on its own in Bugzilla. Andre also investigated the possibility of restricting access to Bugzilla's Priority field as this was brought up by several developers. In Bugzilla's code, the icons on the Bugzilla frontpage are now centered and some Bugzilla code was resynced with upstream. Reasons for an issue with Bugzilla showing cached CSS files were also investigated.

Bugzilla received an option to display metadata changes of a bug report (which are available via the "History" link in a Bugzilla ticket) inline, between the comments. It is now possible to distinguish the products 'MediaWiki' and 'MediaWiki extensions' in Bugzilla's search results. Furthermore, work on creating a guided bug entry form for newcomers continued. Andre pinged on numerous open tickets in PATCH_TO_REVIEW status where the corresponding patch has been merged, went through open tickets with target milestone set to 1.22.0 and pinged/updated their statuses (in reply to tarball release plans), reset tickets from ASSIGNED to NEW status when the assignee was set to the default nobody@ assignee, and started investigating potential issues with our custom CSS theme for Bugzilla.

Bugzilla now offers a new guided bug entry form which will make creating good bug reports easier for newcomers. Bugzilla now also displays metadata changes of a bug report inline for all logged-in users, so they can see in the comments who changed a value of a field (without clicking on "History"). Daniel Zahn upgraded Wikimedia Bugzilla to latest version 4.2.7. Legoktm mass-imported about 400 Pywikibot tickets from Sourceforge to Bugzilla. On a related note, Amir ran a PyWikibot Bug Triage resulting in nearly 100 tickets receiving updates. Furthermore, Andre Klapper investigated Wikimedia Bugzilla's customizations in CSS and code in order to clean up and sync with the upstream code base, to simplify current maintenance and also make a potential future upgrade of Wikimedia Bugzilla from version 4.2 to 4.4 easier.

Andre spent most of the last week on Google Code-In preparations and organizing (improving documentation and importing tasks, supporting mentors and students via IRC, Google Melange and mail). He worked on an initial draft for a Bugzilla etiquette. Code-related, the Greasemonkey triagescripts received some updates (e.g. stock answers to nag assignees), two more Bugzilla custom CSS cleanup patches (1, 2) were tested and got merged, and the "WeeklyReport" Bugzilla extension code was sync'ed with upstream.

Quim Gil and Andre Klapper continued to run and coordinate Google Code-In for Wikimedia. Andre's draft for a Bugzilla etiquette received lively feedback and discussion. On the technical side, Daniel Zahn prepared the migration of bugzilla.wikimedia.org to WMF's new data center by turning the existing rudimentary Bugzilla puppet code into a puppet module and automatically generating documentation on doc.wikimedia.org. As part of this preparation, Daniel and Andre also eliminated nearly all Perl CPAN modules (in Bugzilla's /lib subfolder) on the new server by using default distribution packages instead. Furthermore, Andre worked on a preliminary patch to display some common queries on the Bugzilla front page.

The last two weeks had less activity due to vacation. Apart from gruntwork (Prioritizing and triaging incoming tickets; running Google Code-In; checking feedback channels like Village Pumps): Following and commenting on the lively discussion in mw:Talk:Bug_management/Bugzilla_etiquette; working on the next step (getting feedback from stakeholders) of mw:Project_management_tools/Review and creating a preliminary patch to display some common queries on the Bugzilla frontpage.

The Bugzilla 4.4 instance on the new server has been tested - next steps are to switch the database and DNS after all involved people have agreed on a timeframe. Andre made the Greasemonkey scripts also work with Bugzilla 4.4. Further code preparations (which are not merged yet) were porting custom code so "Saved Reports" feature can be used in Bugzilla 4.4 and some cleanup of the guided bug entry's code. Andre also added a "Situation specific information" section to the Triage guide documentation about purging and profiling.

Daniel Zahn and Andre Klapper upgraded Wikimedia Bugzilla to the latest version 4.4.4. Valhallasw replaced the brittle wikibugs IRC notification bot by pywikibugs (announcement). A bugday took place updating about 50 MediaWiki General/Unknown tickets. Bugzilla's "Tools" product was renamed to "Utilities" to decrease confusion with tools on Tool Labs. Numerous old forgotten "Backport_WMF?" flags on bug reports, older PATCH_TO_REVIEW tickets with all patches merged, and a lot of older WikiEditor tickets were cleaned up. In general, work mostly concentrated on handling the Phabricator RfC.