What Certifiably (Brendan Donegan's Ubuntu Blog) talks about

The inaugural online UDS (or vUDS as it’s becoming known) is underway. This brings with it a number of new challenges in terms of running a good session. Having sat in on a few sessions yesterday and been the session lead for several sessions at physical UDS’s going back nearly two years now, I thought I’d jot down a few tips on how to run a good session.

Prepare

Regardless of whether the session is physical or virtual, it’s always important to prepare. The purpose of a UDS session is to get feedback on some proposed plan of work (even if it is extremely nebulous at the time of the session). Past experience shows that sessions are always more productive where most of the plan is already fleshed out prior to the session and the session basically functions as a review/comments meeting. This depends on your particular case though, since the thing you are planning may not be possible to flesh out in a lot of detail without feedback. I personally find this is rarely the case though.

Be punctual

UDS runs on a tight schedule, at least in the physical version, although I don’t see any good reason why this should change for vUDS. Punctuality is therefore important not as good manners but from a practical point of view. You need to time to compose yourself, find notes and make sure everything is set up. For a physical UDS this would have been to check microphones are working and projectors are projecting. For a vUDS, in my brief experience, this means making sure everyone who needs to be is invited into the hangout and that the etherpad is up and the video feed is working on the session page.

Delegate

As the session lead it is your responsibility to run a good session, however it will be impossible for you to perform all the tasks required to achieve this on your own. Someone needs to be making notes on the Etherpad and someone needs to be monitoring IRC. You should also be looking out for questions yourself but since you may be concentrating on conveying information and answering other questions, you do need help with this.

Avoid going off track

Time is limited in a UDS session and you may have a lot of points to get through. Be wary of getting distracted from the point of the session and discussing things that may not be entirely relevant. Don’t be afraid to cut people short – if the question is important to them then you can follow up offline later.

Manage threads of communication

This one is quite vUDS specific, but especially now that audiovisual participation is limited, it is important that all of the conversation take place in one spot. Particularly for people who are catching up with the video streams later on. Don’t allow a parallel conversation to develop on IRC if possible. If someone asks a question in IRC, repeat it to the video audience and answer it in the hangout, not on IRC. If someone is talking a lot in IRC and even answering questions, do invite them into the hangout so that what they’re saying can be recorded. It may not be possible to avoid this entirely, but as session lead you need to do your best to mitigate it.

Follow up

Not so much a tip for running a good session, but for getting the best from a good session. Remember to read the notes from the session and rewatch the video so that you can use the feedback to adapt your plan and find places to follow up.

That’s all there is to say, I really hope this first virtual UDS goes very well and that sessions are productive for everyone involved.

Part of my duties as a Hardware Certification Engineer is to develop and maintain our test and bug reports. Previously these have been Python scripts containing lots of inline HTML (yuck :/), which I’m ashamed to admit it because I didn’t know any better when I initially wrote them. I’m currently in the middle of refactoring (hopefully not ref**ktoring) the scripts to take advantage of a new API that’s available for querying data from the Certification website and have decided to use Jinja (precisely Jinja2) as the template engine.

So I have a list of systems which may be in different locations and I want to sort them by location. Initially I did this in the Python code which gathered the data, using a simple .sort(key = lambda obj: obj.datacentre). Then the template code used the sorted list and all was well. Later on when reading the Jinja template documentation I discovered a way to sort a list in the template itself. I considered this to be a cleaner, more transparent way to modify the list (somebody looking at the template asking ‘why does it come out in this order’ has the answer right in front of them). So I gave it a try by changing
for hw in {%reported_hardware %}

to
{% for hw in reported_hardware|sort(attribute='datacentre') %}

and was unfortunately confronted with a traceback from the template engine:
Traceback (most recent call last):
...
AttributeError: Hardware instance has no attribute '__getitem__'

This looks like it’s saying that the ‘Hardware’ object (which I’ll introduce in a second) isn’t the type that the template engine was expecting.

Initially I defined the Hardware object like this:
class Hardware:

def __init__(self..., datacentre):
...
self.datacentre = datacentre

My first attempt to fix this involved making datacentre a property:
@property

I find that sometimes the Network Manager applet in Ubuntu can be a little temperamental (apologies to the maintainer, cyphermox, if he’s reading this – but such is the nature of software). Sometimes it won’t show available routers and if that’s the case then I’ve established a little workaround that I’m telling you about mainly because it involves a script I wrote that lives in a somewhat obscure place in Ubuntu.

Step one in the workaround is needed if you don’t know which networks are available in advance. If you’re sitting in your home then you’ll probably not need this step since most people know their router SSID. If you don’t then you can scan using:

nmcli dev wifi list

This is really reliable and always works if your WiFi hardware is working.

The second step is to use the SSID to create the connection using the script I wrote:

If the router doesn’t use any security (which nmcli dev wifi list will tell you) then you don’t need –security or –key. If the router doesn’t use WPA2 (maybe it uses WEP), then you’re out of luck – and deservedly so. Change the routers security settings if you can!

Obviously being an engineer working on Ubuntu, part of my day-to-day life is working with bugs in Launchpad. Filing them, triaging them and fixing them (etc, etc). For a long time I have felt like the tools provided by Ubuntu to assist with bug filing don’t meet my needs as an engineer working on an upstream project. The well known ubuntu-bug is too heavyweight, for the following reasons:

It spends a lot of time gathering logs and other info which in many cases is not useful (for example when filing very simple bugs.

It wants you to go through the browser to fill in many details, mainly unnecessary.

It complains about ‘non-existant’ or ‘unofficial’ packages, which is often just obstructive.

I’m not complaining that ubuntu-bug is bad or useless by any means, it does many useful things and is ideal for the use case of a user filing bugs against packages. I don’t need the extra stuff and just want to file simple bugs against whatever project I may desire.

Wanting for such a script led me to ask on #ubuntu-devel (FreeNode IRC) if such a tool existed. The consensus from two very senior Ubuntu developers who I would trust to know was that nothing was there but it would be simple to write. I knew that, which made it all the more surprising it wasn’t yet. So a few hours later appears:

lp-file-bugusage: A script for filing new bugs in Launchpad. [-h] [--project PROJECT] [--package PACKAGE] title

Most excitingly, it was suggested by Bryce Harrington, maintainer of the ‘python-launchpadlib-toolkit’ package that the script should be added to the package so I proposed a merge request which he accepted. After fixing a couple of bugs, I have got around to getting a build of python-launchpadlib-toolkit into my PPA. You may install the latest version of it for Precise using:

An editor will be opened (either ‘nano’ the default editor in Ubuntu, or whatever you have set in $EDITOR) and you write the description there, save and close it and then you’ll be given the link to the new bug, which you may use to view it.

I’m only just back from my holidays and it’s the time of year to be making some resolutions. We all know that these things usually go nowhere, so I don’t usually bother – but this year I’m in a resolution sort of mood. Maybe it’s because there’s another baby on the way that I’m feeling responsible right now!

My first resolution is to learn my family by marriages language, that being Urdu. Particularly because my mother-in-law doesn’t speak English (a mother in law who can’t talk to you – sounds like some peoples dream!) and learning a language will broaden my horizons, since I’ve never really learnt a (human) language properly. I think this one will be difficult, but I’ll be happy as long as I make some improvement by the year end.

The second one is in the technical field. There’s a very interesting technology for message passing called Rabbit and though not directly related to my job, it’s something that brings me back to my university days when I was very interested in all sorts of distributed systems (I did projects on distributed file systems and message passing interfaces). Of course no-one has any significant amount of control over their career direction at the early stages, so I never got a chance to indulge myself in these areas further, having been thrust into the realm of QA as a graduate. So I will take some time to get up-to-speed on using this technology and hopefully get a chance to implement something in it (time willing)

Back to the soft skills again, a great passion of mine is cooking. I didn’t start doing any cooking for myself until I finished University (most of my food preperation before then could have been better described as ‘assembly’ rather than cooking) and I’d say by now I’ve reached the level of ‘competent home cook’. I want to take that to the next level. Not yet exactly sure what that will mean :) This ones going to be easy I think, because basically it’s just getting better at something I consider my hobby.

That’s it for concrete goals for now. I might pick a language or two to learn as well (my languages are becoming stale actually as I’m becoming increasingly focused on Python and Bash exclusively) just to keep myself fresh and on the soft (heh) side again there’s a general goal about fitness, but I won’t bore anyone with that.

If I can rememember to then I’ll post again next year on where I got – wish me luck!

Today I needed to use Bazaar to merge the changes from just one file in the project that I’m working on to an older release. Trying to search Google for how to do this, the information wasn’t very clear. I eventually figured it out so I thought I’d share the technique here. Sometimes it’s called ‘cherrypicking’ (thus the title).

With the release of Oneiric Ocelot just around the corner and the archives firmly in freeze mode, my main focus has turned to preparing topics for UDS P which is taking place in Orlando at the end of this month. As you might know by reading my blog one of my main roles within the team is co-ordinating testing of SRU kernels by Hardware Certification. The next cycle of development is going to take on a strong flavor of SRU testing. Personally I’ll be hosting two sessions at UDS,

The first one is titled ‘Improving automated certification testing of Kernel SRUs‘ and is based around increasing the overall scope and coverage of the test suite used when testing the SRU kernels. Recently I took the time to document the test suite we use during SRU testing, and if you read through it you can see that it’s really quite basic and hasn’t been especially good at picking up regressions so far. I’m quite excited at the prospect of doing this and my definition of success here will be a test suite that starts detecting real problems early. Linked at the bottom of the blueprint are some notes that were brainstormed together on the #ubuntu-kernel channel on FreeNode last week which will form a foundation for the discussion. If you’re interested in ensuring that kernel updates don’t break your system and will be at UDS P then feel free to subscribe to the blueprint (of course you’re free to send your feedback via this blog as well).

The second topic is titled ‘Image creation and publishing for kernel SRU testing‘ and has a less broadly interesting premise but will be important for us nonetheless. At the moment we use quite a complicated lab infrastructure to install all the necessary pieces for SRU testing over the network and it prevents us from easily allowing external parties to perform the same testing themselves. If we can easily automated the creation of images which include everything required for testing then we can get rid of this barrier. If the subject matter interests you then, again, either subscribe or leave feedback here.

Every well seasoned tester knows the advantages and disadvantages that manual/semi-manual and automatic tests have when compared to each other. A manual test is easy to create, just a few simple words and you have your test. Automatic tests allow you to (almost) fire and forget about them with your only concern being the PASS/FAIL at the end. A semi-manual test is a funny hybrid of the two, usually only used in a situation where a fully automated test is almost physically impossible (e.g. verifying screenshots and tests involving peripherals). Manual tests are not good in situations where the same test must be run many times across a large number of configurations. This is exactly what we have in hardware certification, where we must run tests across ~100 systems on a very regular basis. To this end we’ve been taking the opportunity this development cycle to update some of our older tests to be more automated.

One of the tests that I updated was one which would cycle through available resolutions on the system (using the xrandr tool) and request the tester to verify that they all looked okay with no graphical corruption. This is the sort of test that is fine when someone is running the tests on a one-off basis, it’s not so good when one tester needs to supervise 50+ systems during a certification run. One of the main problems is that it causes too much context switching, with the tester constantly needing to keep an eye on all the systems to see if they’ve reached this test yet. Obviously, it being a graphical test, it’s difficult to do fully automated verification so a compromise needed to be reached. The solution I came up with was to integrate screen capture into the test and then upload these screens in a tgz file as an attachment with the test submission. Everything going well, the tester can sit down at their own computer and go through the screens and confirm they’re okay. In fact the person verifying the screens doesn’t even need to be in the lab! The task can be distributed amongst any number of people, anywhere in the world.

Another test that looked like a prime candidate for automation was one for testing the functioning of the wireless card before and after suspending the system. Previously the test case was:

This was all specified to be done manually. I am currently updating this test to use nmcli to make sure a connection can be made, then disconnect and reconnect just as would happen if the tester did the steps manually using nm-applet. The one thing I haven’t got down pat yet is connecting to a wireless network where a connection didn’t exist before. This step may be optional as it could be expected that the tester will do this manually at some point during the setup of the tests and we can trust a connection to be available already. This will mean this test has gone from manual to fully automated and hopefully should shave potentially some significant number of minutes off the whole test run!

Saving time on our existing tests will allow us to introduce new tests where appropriate, so we’re able to provide even more thorough certification testing.

I attended a company rally in Dublin last week so didn’t get around to blogging, but having received some good news just yesterday I though I’d make a posting about it.

After the Ubuntu Developer Summit in Budapest one of the objectives I set for myself this year was to become more active in the Ubuntu community. Being a QA person, one of the most obvious routes was through the well structured bug handling initiatives that the community has in place (thus my posting about the Bug Squad and bug days). To this end I’ve devoted what spare time I have to these initiatives. Apart from Bug Days I also take part in the 5-a-day program which encourages participants to update five bugs a day (by commenting, changing status or updating titles – anything that could be construed as a valuable change to a bug). This is going pretty well and I’m on a 3-week streak of fulfilling my 5-a-day quota (according to the 5-a-day report).

As a result of all this activity I’d gathered enough experience (and evidence of that experience) to file an application for the ubuntu-bug-control team in Launchpad, who have permission to change bug Importance and set the status to ‘Triaged’. This involved testifying to having read some documentation and then providing some example bugs to demonstrate that I understood the bug triage process. It took a week and a half to get enough positive responses (two) to have my application accepted, but as of yesterday I am officially a member of ubuntu-bug-control!

I’ll do my best to use these powers effectively and diligently and hopefully make a big difference to the effectiveness of bug triage in Ubuntu.

Something I recently (embarrassingly) discovered is that bash supports the concept of aliases, which are like shorthand for commonly used commands. Ubuntu comes with a few as default already in your .bashrc, e.g. ‘ll’ for ‘ls -alF’ (long listing). You’re free of course to add your own in .bashrc, so here I present some of the ones I use:

As discussed at last months Ubuntu Developer Summit in the session ‘ARM and other architectures certification program‘, there’s a plan to start certifying ARM hardware, or at least start investigating how we’ll do it. To this end I’ve received on loan a TI OMAP4 Pandaboard from Canonical’s ARM QA team. I’ve actually had it here in the office for quite a few weeks now but for some reason or another I haven’t got around to blogging about it yet!

So, without further adieu – here are a couple of shots of my setup:

I like it because it’s really compact and smacks of geekiness, with all the exposed circuits, yet is really quite easy to use in a lot of ways. The monitor is plugged in via the HDMI port on the right hand side (because of an issue with my monitor I can only get 640×480 out of it, so everything is very squeezed on the screen) and the wireless desktop receiver which handles my mouse and keyboard plugs right in to one of the two full sized USB 2.0 ports. The whole thing is powered by my laptop (even when it’s suspended) via USB-AC 5v connector, also on the right-hand side.

It’s running Natty/Unity 2D installed on the 8GB SDHC card on the left of the board. This means that the whole setup cost (if I had have payed for rather than borrowed it) just under $200. The white labeled chip on the top left hand side of the board is the WiFi/Bluetooth chip and that works *perfectly* out of the box – often picking up a better signal than the laptop sitting right next to it. I also have the option of plugging in my USB headset in the the same USB hub as the wireless receiver (it’s a tight squeeze but it just about fits) and that too works perfectly.

Cons are that I don’t have a USB HDD so Ubuntu is running on flash memory (notoriously bad performance) and that if I decide to power down my laptop but forget the Pandaboard has some task running on it then all is lost Overall though it’s a really nice piece of equipment and because of all the good work that has been done around it, I could recommend one to anyone with a bit of technical know-how (no ARM experience required!)

In my travels around Launchpad looking for bugs to triage, I came across an old one that I noticed (but not before others apparently) in the Alpha 1 release of Oneiric Ocelot. This was a problem with update-manager not ‘seeing’ that network-manager had a connection because the new version of network-manager (0.9) uses different codes to express ‘connected’.

This issue was bugging me, so I decided I’d take it upon myself to patch it up. Someone had done a similar patch in software-center so I already had all of the knowledge needed right there (i.e. what are the new codes). I jumped into my Oneiric VM, branched the update-manager code and hacked away at a couple of Python modules, tweaked, buffed and polished until lo and behold, on starting update-manager it picked up the connection! A few command lines (bzr stat, bzr commit, bzr push) and a few clicks in Launchpad later my merge request was with the update-manager project maintainer (Michael Vogt aka mvo). Minutes later it was merged and the next day with the help of my patched version of update-manager I was able to update update-manager with the patch.

Looking at my own name there in update-manager’s description of the change, I couldn’t help but think how awesome it is that I’m able to do this with my favourite operating system. That’s what makes OSS magic for me…

When I worked in the Symbian Foundation, as part of the (Symbian) Bug Squad activities that I helped run we would (try) to have regular get together on IRC where the community would come together and work on something in particular. Mainly this was getting triaging done. We didn’t have the benefit of a lot of experience, so this would be done in something of an ad-hoc way with everyone discussing each bugs status and priority until we reached a conclusion.

Now that I’m at Canonical and trying to participate heavily in Ubuntu’s Bug Squad activities, it’s comforting to know that something similar goes on here (maybe we were subconsciously influenced by it ?). It also happens to be on the same day (Thursday) of the week. I’m of course referring to Hug Days, which are co-ordinated by the QA team. I’ve been involved in them over the last few weeks as a participant (rather than an organiser) and I find the structure to be very good and very accessible. Quite simply there is a list of bugs with different statuses (New, Confirmed or Incomplete) and simple instructions on what to do with each bug. New bugs need to be either Confirmed or set to Incomplete if you find you need to ask the reporter for extra details to be able to reproduce the bug. Confirmed bugs themselves need to be revisited and a check done to make sure the bug is still happening, leading to the bug either being Triaged or set back to Incomplete if it’s not happening and you need the reporter to reconfirm. Lastly, Incomplete bugs should be checked for a response from the reporter to the information request. If they gave the necessary info then the bug should be Confirmed. If not a follow up question should be asked and the bug left as Incomplete.

Some tools that are handy to have to assist with the process of going through all these bug reports and updating them correctly are the Hug Day tools, which semi-automate the process of ‘closing’ Hug Day bugs (they aren’t being closed as bugs, but as tasks on the Hug Day), as well as the Firefox Launchpad Improvements, which are useful not just for Hug Days but any bug work. The improvements include canned bug comments for common scenarios such as when an inexperienced bug filer has provided little info on the bug and you need to tell them to provide simple steps to reproduce the bug.

Each Hug Day is based on a particular package (which helps to focus the effort) and this weeks Hug Day is on Nautilus, Ubuntu’s file browser. I have and will be participating in this as much as I can, so if you decide to participate in it then say Hi on Freenode IRC #ubuntu-bugs where there are lots of knowledgeable Ubuntu people waiting to help out newcomers with the task at hand. See you there!

As a means to control the activity on bugs, Launchpad implements a sort of permissions system for who’s allowed to change certain properties of a bug. Mainly this is applied to the ‘Importance’ field of the bug as well as some values of ‘Status’ such as ‘Triaged’. This is because these are used for managing workloads and it wouldn’t be desirable for people with little experience of managing bugs to just go changing them without having a fair idea of what they were doing.

For the Ubuntu project, the team which has these permissions (amongst others) is Ubuntu Bug Control. This is a group of Ubuntu contributors who have, over a period of time demonstrated their capability at bug analysis. Joining this group is done through a merit system, whereby you have to apply and back up your application with evidence. You need to state that you’ve read the relevant documentation on bug importance, status, assigning bugs and triaging. You are also quizzed on the requirements that need to be met in order to mark a private Apport crash bug public. Then you need to give a commentary on your ‘best’ five bugs, explaining what actions you took on them (status changes, any actions taking place around the bug such as working on a fix or discussing issues with fellow community members).

In all, the process is not too demanding for someone who has been doing this a while, so I plan to make my own application soon. I’m still working on my ‘bug resume’, since I have the slight disadvantage that a lot of the packages my team works with are private, so triaging I do there can’t be used as examples. So for a few more weeks I will do a few bugs each day and probably submit my application before the end of June.

One of the things I do on a regular basis is co-ordinating and executing the testing for the ‘Stable Release Updates’ which Canonical issues every few weeks. These contain important fixes to the kernel which we want to get out to users as soon as possible, but with these fixes come the possibility of regressions. We take it upon ourselves to run a small suite of tests, tightly focused on essential hardware such as graphics, memory, networking etc. across every system which we have certified for that release. This amounts to nearly two hundred systems when combining the systems we have for all the releases which need to be tested (at the moment it’s two, but this could be up to three). So the effort looks quite large, but luckily we have established a good process for it where I take care of servers which can be tested totally remotely and my colleague in Montreal takes care of laptops (which need a small amount of manual intervention). I also test systems which are located in the US with help from another colleague of mine who looks after the lab there. Between us we do a pretty efficient job of it.

What comes out of this process are reports (such as http://people.canonical.com/~hwcert/sru-testing/wk14_2011/lucid-proposed.html) which tell us how each system did with the testing. Failures are analysed and we check if they constitute a regression in the platform. Up until now we’ve been quite lucky (well for a tester we call it unlucky ) to not find any major issues. This probably means we need to beef up the test suite! After we’ve tested all the systems we report on the updates tracking bug to indicate our happiness with the state of the update. All being well in other places (QA do regression testing too, and the community gets a chance to test it as well) the update gets published and every Ubuntu user sees updates in Update Manager!

I’ve just spent the best part of a week in the beautiful city of Budapest attending the Ubuntu Developer Summit (henceforth, UDS), an event which brings together lots of different people involved in the Ubuntu community. This has been a really good experience, so I wanted to talk about it a bit.

I traveled to Budapest with my wife and baby daughter, arriving on Sunday evening. Almost immediately upon arriving things started happening as I needed to attend a Canonical pre-event session. I got to meet up again face to face with the people on my team who I interact with every day. This is one of the most important things about these events I think, as no amount of IRC and VoIP contact can replace good old fashioned human interaction.

The event kicked off the next morning with an introductory session hosted mainly by Mark Shuttleworth himself. As is well documented on the blogs and tech sites, he outlined the goal of reaching 200 million users for Ubuntu. This will be a difficult goal to reach anytime soon, some kind of huge breakthrough notwithstanding, but everyone loves a challenge so it’s good to have something to aim for! Of course he also talked about the short-term goals of this release, mainly the theme of stability and minor enhancement (Natty was the real jump, so time to take a breather). Of course my team doesn’t deal directly with the platform, so this could well be just as chaotic a cycle for us as the last one.

Immediately after the introductory session I had my own session to host. I was given the responsibility of owning the topic ‘Desktop Certification Scope’, which is basically a theme around enhancing the certification test suite to have greater coverage and clearer definitions of which tests are the most important and which are less so. The session went well, with a lot of input – most of it good. One lesson I learned though was that there will be people who come to these things with their own agenda and you need to steer them a path so that they don’t take the session off track.

Since I want to make this a short post, I’ll summarise the rest of the week by saying that it was very hectic! I participated as a contributor in a number of other sessions (both sessions run by my team and by others, such as one about revising the UI for Ubiquity, Ubuntu’s installer application), and learned most of the same lessons again! One thing that was really encouraging was reports by colleagues who had been to a number of UDS’s before that our sessions drew a lot of interest compared to previous years. It’s good to know people are starting to care about what we’re doing.

Since my family was with my I took advantage of the evenings to explore Budapest, which has lots of amazing locations to see. I’d heartily recommend anyone to visit, particularly the Castle District on the Buda side of the river (our hotel was in Pest, so it was a tiny bit of a hike to get there). Well worth it and primarily free (as in beer) to see.

UDS is basically a planning session for the next release, so I’ve got lots lined up for Oneiric Ocelot. I’ll keep my progress up-to-date here and perhaps intersperse it with a few topics of my own interest…

For a while I had a blog on WordPress which was related to Symbian and other mobile stuff. This has fallen by the wayside after I left Symbian. To get you all up to speed, I’ve been working for Canonical for the past 6 months and have become entrenched in the world of Ubuntu and Linux in general. My role there is as a Hardware Certification Engineer, which is mainly a QA type role. In general my team deals with running the the Ubuntu Certified program (http://www.ubuntu.com/certification) which awards ‘certified’ status to systems which pass our rigorous test suite.

My own role mainly focuses around the development and maintenance of the tests and tools used, as well as some operational stuff such as running tests on stable release updates (SRUs) across all of our certified hardware.

In a nutshell, that’s what I do. If you want more detail, keep your eye on my blog as I’ll be trying to be more active on this front. I just needed some time to get comfortable in my new role and get my feet on the ground before starting this again. I’ll follow this post up with a quick one about something I’ve been up to recently.