Menu Item Usage Study: Part I

For the last few days the Test Pilot team at Mozilla Labs has been running a test to explore usage of the Firefox menu bar. Ever since Mosaic 1.0 web browsers have had a standard menu bar–one that has always followed the design of a standard desktop publishing application, containing top level commands like File and Edit, even those these commands are not necessarily relevant to a web browser.

In order to streamline the Firefox user interface, and to match the overall interactive design of Windows 7, the Firefox UX team is exploring collapsing the menu bar into a single “application button” when Firefox is running on a modern version of Windows.

This menu item usage study will help guide the UX team as they create a fully optimized design by answering 3 questions.

Which menu items are the most commonly used?

Which menu items are the least commonly used?

How long do users spend exploring the menu bar contents before selecting each particular menu item?

In this post, we will discuss some preliminary findings regarding the first 2 of these 3 questions. Look for further analysis and a discussion of the 3rd question in our next post!

Experiment Results
The most obvious way to determine the most and least commonly used menu items is to simply aggregate the total number of menu item clicks for all users.

This graph shows just that, presenting each menu item’s relative use for all UI methods (both mouse and keyboard shortcuts). Even from this simple analysis, we can see some justification for a condensed toolbar as many of the items are used very infrequently compared to the other menu items. For example, the menu items from “Page Setup” to “Character Encoding/UTF-16″ each make up less than 0.01% of the total menu bar clicks.

While looking at the total number of item clicks can be informative, since menu bars are designed for mouse use, it is more relevant to look at item usage for just the mouse UI method (excluding keyboard shortcuts).

Examining the data in this way presents a slightly different picture: the top 5 most commonly used menu items are now “User Bookmark Item”, “Copy”, “Paste”, “Add-Ons”, and “Back”. In addition to “Add-Ons”, “Options” and “Bookmark This Page” are newly part of the top 10, replacing “Find”, “Open Location”, and “Find Again”.

Again these changes simply result from eliminating keyboard shortcut clicks and help us distinguish between mouse driven menu items and keyboard driven items. For example, by comparing the mouse UI chart (right) with the original all UI chart (left) we can clearly see that “New Tab” and “Close Tab” are predominately driven by keyboard shortcuts (as expected) and may not be the two most critical items to a mouse oriented toolbar (as suggested by the original chart).

Another interesting approach to these questions is to group the items by menu and visualize the data in this form (again, data is just for Mouse UI).

This visualization presents information on two levels: the area of the circles are proportional to the total number of clicks for the menu group as a whole, and the slices correspond to the share of clicks for each item within the menu group. Bookmarks and Edit are by far the most utilized menus, representing over 70% of total clicks.

The high use of the bookmarks menu is somewhat surprising; an obvious problem of looking at aggregated data like this is the potential for outliers to skew the data. It will be interesting to delve into this issue more in depth and determine if the Bookmark menu (and other menus and menu items) is genuinely an important menu group for all users, or if the high usage is driven by a set of relatively few users who interact with the Bookmark menu extremely frequently.

Wrap Up
Next time we will take our analysis further and move from answering questions about the frequency of item usage to examining how long users spend exploring the menu bar before selecting each particular menu item.

Thanks again to the Test Pilot Team and to all Test Pilot users for providing us with the data. Remember more information on Test Pilot studies can be found here. Anyone interested can also download data samples for this and other Test Pilot Studies from the website!

18 responses

It would be interesting to know the source.
I mean copy and past are the most common used action but do you take in account context menu? If yes I guess most of the time people use them from context menu on text field.

Very interesting results. You should keep in mind, however, that Test Pilot users are usually techie people (myself included) and not like the average Joe. For example, I can’t imagine normal users using a keyboard shortcut to close a tab.

It is not as interesting as you describe it.
My explanation (only for “CLOSE TAB” / “NEW TAB” problem):
Firefox default settings (like unchecked “Hide the tab bar when only one tab is open”) caused this. Users learned how to open new (close) tab via menu, because there was not other user friendly approach to do this (hotkeys is not user friendly, although I prefer this) with only one tab open. And this approach is usable () to open/close tab all the time while you are browsing, so why change it. I know, there is a reason to change e.g. because of speed of browsing you can use hotkeys, but you can still use attested approach (via menu).

My explanation (only for COPY problem):
Why can I still select the text without any following action/operation like copy/paste? To copy some text I have to do minimal 3 moves – (select -> edit (in menu) -> copy (in menu)) or (select -> do right mouseclick -> copy (in contextmenu)). There should be an automation – When I select some text then context menu should appear automatically.

If you change the traditional menu bar many like myself will remove Firefox from our computers. I had Chrome installed just long enough to find out there was no way to show the menu bar, then I uninstalled it. Look at the angry feedback Chrome is getting about this issue. Many, many people have answered no menu bar no Chrome. Don’t make the same mistake.

I always thought i’m in the %0.01th percentage.
When I need something it should be there.
I think working for 3 months, I would definitly need to some property, if it is not there after my 10.000 click becomes absolote. I would get angry. I would think to move other software.
%0.01 is nothing statistically but personally if im that number, it is important. I think features can be hidden but they should available, even if took 10 mouse clicks If it would worth, user clicks.
I like the idea of having much more pixels.