Canonical Voices

What Chris Johnston talks about

I blogged a couple of weeks ago about the addition of the key performance indicators to the Ubuntu QA Dashboard. Since that post the QA team has been hard at work. We have added a bootspeed KPI to the dashboard, giving you a quick look at today vs yesterday. Another cool feature that has been added to the dashboard is the addition of bug information. Previously the dashboard just provided a link to the bug in Launchpad. The dashboard now fetches the bug data from Launchpad and displays it for you when the mouse hovers over the link. No more having to click through to see what the bug is that is causing issues!

The other big things that we have added goes back to one of the most basic types of testing that we do, the smoke test. We have added two new big things to smoke testing in the past few weeks. The first is that you are now able to see, from the dashboard, which tests pass and which fail. The new test case results page shows you quite a number of things about the testing that was done. You get the basics that you see on the other smoke testing pages (total tests, pass/fail/error count, pass rate, as well as image and machine information) but you now see a list of test cases and their return codes. As you see on the results page I linked to, there were 19 test cases that ran plus four setup type commands. The setup commands are shown since it is possible for them to error. Clicking on the individual command types will give you more details about the specific test to include the test suite and the command that was run. This is all valuable data in determining the quality of Ubuntu each day and easily pinpointing any problems that there are.

The final thing that has changed with smoke testing isn’t so much a change in the dashboard but an addition of a type of test. The QA team is now running daily Ubuntu Touch testing with autopilot. Adding the autopilot testing provides us with a new and much better grasp of the daily quality of Ubuntu. Previously, we relied on manual testing for much of the functional testing of Ubuntu to determine if things where the way that they were supposed to be. While the manual testing is still an important part of the overall indication of quality, the addition of the autopilot tests provides us with the ability to test many more things at a much higher frequency than relying completely on manual testing. The autopilot testing results show up in the smoke testing results each day after they run.

Recently there has been a huge push for quality in Ubuntu. The goal is to have a stable and usable daily image of the development version of Ubuntu. To do this, the QA teams have spent a lot of time writing tests and manually testing everything in Ubuntu to ensure this high level quality. The QA Team is also developing the QA Dashboard to display all of the results from the testing that is being performed. The goal of the QA Dashboard is to start at high level overviews of testing results, and drill down to the small details that make up the overall big picture to give users and developers the ability to quickly and easily be able to identify that a problem exists, determine the cause of the problem and then fix the problem. Certain tests or types of tests are part of what we call ‘key performance indicators’ or KPI. These key performance indicators will give someone a very quick overview of the quality of today’s Ubuntu image.

Of the types of testing that is being done, the smoke testing is the most important one. Smoke testing is the most basic level of what to test. Essentially, does it install and after it installs does it work? If the image doesn’t install, none of the other testing can be completed. Infact, the daily builds part of cdimage now has a current and a pending. Current is the most recent image to pass the automated smoke testing whereas pending has not yet passed smoke testing. Because smoke test results are so important, it is now listed as a key performance indicator on the QA Dashboard. The Smoke Testing KPI shows each release that is currently being tested as well as the pass rate for the latest image of that release as well as which arches are being tests. It also will show you the number of bugs effecting a release by hovering over the pass rate.

The other test that is currently listed as a key performance indicator is Eventstat Wakeups testing. Essentially, this test is measuring the number of wakeup events each process has. The reason this is important is each wakeup prevents the kernel from going into low power mode, or idling. With the push towards cell phone and tablet devices battery life is a big deal. It is important that power consumption is properly measured and understood in order to make an operating system that performs well on these small factor devices which have a limited power source. The eventstat testing tells developers which processes are causing too many wakeups, allowing the developers to look into the root cause of all of the wakeup events. This KPI shows each machine that the tests are run on and the average number of total wakeups for all processes on that machine as well as the percent that it is higher or lower than the last image.

There are quite a few other types of power related tests that will hopefully soon be added as key performance indicators for the overall quality of Ubuntu. We have sort term goals of adding bootspeed testing results and memory usage as key performance indicators. If you have any questions about the QA Dashboard, feel free to jump into #ubuntu-quality on Freenode and ask away.

Last cycle we saw quite a number of changes to the way that planning works for Ubuntu. Some of these changes are causing issues that the current implementation of the Ubuntu Status Tracker is not easily able to handle. The main issue that I have noticed from helping people setup their work items and blueprints for tracking is that tracking needs to not be so closely dependent on the Ubuntu release cycles. This is causing issues in two ways. The first that I have seen is that a feature is planned to be released in stages essentially X number of cycles. It currently isn’t possible to track a single blueprint across different cycles, let alone multiple cycles. If you try to do this anyway by changing the cycle every 6 months, then the Status Tracker sends out what are essentially validation errors because as far as it is aware, any milestone that isn’t in the cycle that it is looking at is in valid (ubuntu-13.04 is a raring milestone and isn’t valid on a saucy blueprint).

In order to discuss these issues and hopefully come up with a solution, I have created a meeting for the virtual Ubuntu Developer Summit which starts tomorrow.

Next Tuesday, May 14 starts the second Virtual Ubuntu Developer Summit. Based on feedback, this vUDS will be three days long. Don’t forget to register for the event. A list of currently approved blueprints is available on Launchpad. If you find that one is missing, you can create your own. Keep an eye out this week for scheduling to start.

Tracks…

App Development: Alan Pope, David Planella & Michael Hall

Community: Daniel Holbach, Nick Skaggs & Jono Bacon

Client: Jason Warner & Sebastien Bacher

Server & Cloud: Dave Walker & Antonio Rosales

Foundations: Steve Langasek

Bugs…

One of the bugs that has long effected Summit has been that a blueprint had to have a specific status. This will finally no longer effect us! If a blueprint is marked anything other than Obsolete or Superseded it will now show up on the schedule as long as it is approved by a track lead. A huge thanks to Steve Kowalik and William Grant for getting this fixed!

Another issue that seemed to confuse people is having to register attendance in Launchpad in order to be able to use the features of Summit. This is no longer the case! You are now able to register as attending in Summit without any need to visit Launchpad (a Launchpad account is still required).

If you find that you have any issues with Summit or vUDS you can contact the track leads or you can contact me. If you find any bugs in Summit, please tell us about it!

I have been a Google Chrome user for a while now, and I have two different ‘Users’ in Chrome. The default user is my personal account, and then I have a work account. For my personal email I use a Google Apps Gmail account and just check my email with Chrome. I use Thunderbird to check my work email. For a while now I have had an issue where I click a link from Thunderbird and it tries to open in my default Chrome user. This doesn’t work very well as I am not logged into most of my work accounts on my personal user. This drove me nuts! Now I have to copy and paste the URL into the work user Chrome window. After a little Googling tonight, I was able to setup Thunderbird to open URLs in my work user Chrome browser. Life is much better now! To do this, I had to add two lines to prefs.js. On Ubuntu 13.04, prefs.js is located at ~/.thunderbird//prefs.js where is what appears to be a random set of numbers/letters followed by .default.

If the profile-directory for the Chrome user you are wanting the links to open in is different than what I have, you may need to edit the directory name. This worked for me on Raring (what will become Ubuntu 13.04) with Thunderbird 17.0.4.

Friday I decided to create a new OpenPGP key to migrate off of my 1024D keys to something a little more current as far as how secure it is. With this, I created a 8192R key. I have also created a transition statement. If you trust me and wouldn’t mind signing the key and uploading it back to the keyserver after you are complete, it would be very much appreciated.

Yesterday Anthony Lenton released django-openid-auth version 0.5 which includes support for Django 1.1 through 1.4. It also has quite a number of bug fixes in the release as well. The new version is available from PyPi today. I plan on getting the new version into Raring soon if someone else doesn’t get to it first. I will also work with the Debian maintainer to get it into Debian soon.

Most people are probably aware by now that this Tuesday, 5 March starts the first of the new virtual Ubuntu Developer Summit events. In order to handle the changes nicely, we have made some changes to the Summit Scheduler. The changes that we made allows a meeting and/or a summit to be set as ‘virtual.’ When a meeting is set as virtual, the meeting page will render with a new virtual layout. This layout includes the Google Hangout broadcast, the IRC channel via the webchat client, and the Etherpad, all embedded on in the page. The required participants for a meeting will also be given a link to participate in the hangout. Both the embedded broadcast and the hangout link have to manually be added by the track lead prior to each meeting. If the hangout broadcast isn’t available when you visit the page, please be patient and wait for the data to be added. Once the information is added to Summit, the meeting page should automatically load the hangout broadcast.

Please don’t forget to test using hangouts prior to UDS starting so that we can minimize the number of issues we have with the hangouts. A “best practices” guide has been created for use during the Ubuntu On Air events. Please take the time to look at these practices before Tuesday.

Over the past few months a lot of work has been done to the Summit Scheduler. This all culminated this past week when I sent in an RT to update the production instance of Summit. This included somewhere in the neighborhood of 8,000 lines of code change between Summit and the two themes that are run.

There are two major changes to Summit from the changes this past week. The first is that Summit has been rethemed to meet the new design guidelines for Ubuntu. This work was completed with major assistance by Alexander Fougner, Stephen Williams, and Brandon Holtsclaw who each put in numerous hours to make this happen.

The other major change, which affects the way that you will work with Summit is that you now have the ability to propose meetings in Summit removing the requirement that you create a blueprint. Also, as long as you’re using the new propose meeting feature in Summit, you no longer have to have a special name for your meeting, this is now done automatically. I also created a video which will walk you through the process of proposing a meeting in Summit.

The Summit Hackers have been hard at work again creating new features to make Summit easier to use. However, we need your help testing these features before the public release. This is where you, the user, come in! I have created a Test Summit on my server and created a test sprint in Launchpad and here is where I need your help with testing:

First, visit the test sprint in Launchpad, next mark yourself as attending. Please note, it could take up to 20 minutes before you are able to use summit after marking yourself as attending. You can either attend physically or remotely, it doesn’t matter. While you are there, take the time to create a couple of test meetings. Please be sure when you create these test meetings to follow the naming guidelines below.

Hey Chris, what are the new features? I am so glad you asked! The new features are:

For Attendees:

Summit now supports the ability for attendees to propose meetings right inside of Summit! On certain pages in Summit, you will now see an “Actions” area. Depending on what part of the cycle we are in, you may see different links. During the sponsorship application process, you will see links to apply for sponsorship. If you have the ability to review sponsorship applications, this is where you will find that link during that time period. During the scheduling phase of the cycle, you will be able to “Propose a Meeting.” What this means is that you will create a meeting in Summit, which will then go into a holding pattern waiting for a Track Lead or Summit Organizer to approve or deny your meeting. You may be familiar with this behavior, as it isThis is a very similar behavior as it has been with what you may have seen with Launchpad.

For Track Leads, Schedulers, and Organizers:

If you are listed as a Track Lead, Scheduler, or Organizer in Summit, you will have the ability to “Create a Meeting” and “Review Proposed Meetings.” When creating a meeting, you have the ability to mark a meeting as Private. If a meeting is marked as private, you will have to get it scheduled through a scheduler (Michelle and Marianna for UDS, Arwen for Linaro Connect), it will not be automatically scheduled. Non-private meetings that are created by a Track Lead, Scheduler, or Organizer will be automatically scheduled when slots are available.

The other option that this group has is reviewing proposed meetings. There are three different options for the status, Approved, Pending, and Declined, which you get to by clicking the current status. Pending and Declined meetings show up on this page, while Approved meetings are auto-scheduled and no not show up here.

Known Issues:
* All tracks are shown in the track choice list, not just the ones for the current summit
* Edit meeting links don’t work
* Issues with the form theme

Creating Meetings through Launchpad

The good news is–the old way still works! You will still able to create meetings through Launchpad Blueprints just like before. There are a few things you will need to be aware of as you create meetings this way, these are as follows:

Tracks:

I have created a couple of tracks for this Test Summit. They are as follows:

Name Slug
Community community
Linaro linaro
QA qa
Ubuntu ubuntu

Blueprint naming standards

When creating a blueprint in Launchpad, please follow the required naming standards whereis a custom name that you create for your meeting:

For the Community track:

community-

For the Linaro track:

linaro-

For the QA track:

qa-

For the Ubuntu track:

ubuntu-

Managers, Organizers, Track Leads, oh my!

If you are a member of ~summit-users on Launchpad, please take the time to approve/deny blueprints in launchpad.. I would like to have a couple denied, just to make sure everything still works properly. The same thing with the new meeting review feature in Summit. Please do not approve all meetings! Some of them need to be declined to ensure that everything is working properly!

IT’S VERY IMPORTANT that people who are in the role of Track Lead, Scheduler, or Organizer for UDS and Linaro Connect test things and give your feedback, please don’t wait until the week before your event to test this, please be considerate to those volunteers who are working to make this better for everyone. If you feel that you should have this access but don’t, please contact me and I will get you setup.

If you run into any problems, please file a bug and talk to me on IRC!

I look forward to your feedback and continuing to make summit the best possible scheduler it can be, and to do so requires your help. Thank in advance.

My buddy Dustin Kirkland pointed me to a neat little utility that he wrote with Scott Moser called ssh-import-id. Since he showed it to me a few months ago, I have used it many times and it has made my life quite a bit easier.

ssh-import-id fetches a the defined user(s) public keys from Launchpad, validates them, and then adds them to the ~/.ssh/authorized_keys file. That’s it, but if you need to add multiple people, or don’t know which key they are going to want used, this will save you time.

Dustin has tried to get it added to OpenSSH, but he hasn’t been able to succeed at this yet.

To use ssh-import-id, you first need to install it if it already isn’t:

sudo apt-get install ssh-import-id

Then to run it you would run:

ssh-import-id chrisjohnston

This would import my public keys. You are also able to import multiple users at the same time:

ssh-import-id chrisjohnston kirkland

If you are looking for the latest version of the code it is available in a ppa:

Yesterday was day one of Linaro Connect Q1.12. This was a very productive day of meeting people that I will be working with on LAVA, as well as other people throughout Linaro. I attended a couple of sessions during the morning, but I didn’t have anything noteworthy come out of them at this point. During the afternoon I worked with the Validation team in their hack session and was able to get LAVA running locally on my laptop and imported a bundle stream from Linaro’s LAVA setup.

Throughout the day I also did some work on Summit, pushing a couple of fixes and a new IRC link feature to Launchpad. While in the hacking session I met with Andy Doan and discussed the future of the Linaro Connect Android app, and we managed to get out a bug fix for it.

On day 2 I hope to do some more work with LAVA, hopefully figuring out how to play with the results that are displayed in the dashboard.

This week I will be attending Linaro Connect Q1.12 in Redwood City, California. Infact, I’m in an American Airlines plane at 34,000 feet heading there now. In-flight WiFi is awesome!

Over the past two months Michael Hall and myself have been doing a large amount of work on The Summit Scheduler to get it ready for Connect this week including modifying more than 2,400 lines of Summit code. You can find out more about that in my previous post.

I have a few things that I want to get out of Connect. The first is that I want to get feedback on the changes to Summit, as well as figure out what other things we may need to change. The second thing that I want to do is to learn more about the Beagleboard-xM that I have and how to use it for the many different things it can be used for. The third thing that I want to do is to learn about Linaro’s LAVA.

LAVA is an automated validation suite used to run all sorts of different tests on the products that Linaro produces. The things that I would like to get out of Connect in relation to LAVA are how to setup and run LAVA, how to set it up to run tests, and how to produce results and display those results the way that I want them.

If you are at Linaro Connect, and would be willing to talk with me about Summit and the way you use it and your thoughts on the changes, please contact me and we will set aside a time to meet.

During the months since UDS – P in Orlando, FL, the Summit Hackers have been hard at work. As of right now, we have modified more than 2400 lines of code and fixed 24 bugs. Most of those 2400 lines of code have been adding new features and finally getting around to adding a better looking schedule view when on your personal device. While we have made great strides towards getting Summit to be perfect, there is still more work that needs to be done. Our first big test for these new features will be the Linaro Connect Q1.12 event that starts next Monday, February 6. I have the pleasure of being able to go to this event, so I hope to obtain feedback while I am there on ways to further improve Summit for next UDS. Please feel free to poke around with the changes and provide any feedback you may have!

Also, thanks to the work of Stuart Langridge, the Summit agenda view will soon be mobile friendly, making it easy to follow your schedule on your cell phone!

Release Notes:

Added user roles of Scheduler, Manager, and Track Lead

Added the ability for Admins, Schedulers, Managers, and Track Leads to create private meetings through the UI

The creator has the ability to add people who are required to attend

The creator and any schedulers have the ability to edit the private meeting, as well as edit the attendee list

Added a way to create a link to a private meeting that has a random hash string in the URL.

If you have been given this URL you can view the meeting details without being marked as attending, or having a launchpad account