Ubuntu's HUD: Light-years beyond any menu system

Ubuntu continues to take Unity into the future of desktops, and with its new HUD system, the desktop has now gone boldly where none has gone before. Jack Wallen introduces the Head-Up Display.

Before I dive head-first into this week's post, I wanted to mention that this post is my 300th Linux and Open Source blog entry for TechRepublic. It's been such a wonderful part of my life to help the mighty TechRepublic nation stay in the know as well as grandstand on a podium made of soap boxes for Linux and open source.

Thank you all for continuing to stop by and read my words. I wouldn't be here without you! And now...let me hop back on my box.

HUD

Last week I raved about how much improved Ubuntu Unity was. During that love-fest I briefly mentioned the HUD (Head-Up Display). This new "intent-driven" menu system takes what Microsoft tried to do with the MS Office Ribbon interface, and made it actually work -- and work incredibly well. I thought I should expand upon my brief nod to the HUD and explain how this baby works. Once you've tried it, you will want to switch (or switch back) to Ubuntu 12.04.

What exactly is HUD?

Mark Shuttleworth said, of HUD:

This is the HUD. It’s a way for you to express your intent and have the application respond appropriately. We think of it as “beyond interface”, it’s the “intenterface”. This concept of “intent-driven interface” has been a primary theme of our work in the Unity shell, with dash search as a first class experience pioneered in Unity. Now we are bringing the same vision to the application, in a way which is completely compatible with existing applications and menus.

I think his term "intenterface" is dead on. Why? Because instead of having to dig through an endless amount of nested menus within your applications, you simply open up the HUD (by hitting the Alt key) and then typing what you want to do. For example: I'm in Firefox and I want to go the extensions page. I hit Alt (to open up the HUD) and type add.

Figure A

The results (Figure A) allow me to choose from any menu entry, within the Firefox menu system, that relate to the word "add".

You can then either click the correct entry with your mouse or use the arrow keys to select and then hit enter to open up the correct result.

The HUD is surprisingly accurate and makes using menu systems (especially larger ones) much more efficient.

Now, at the moment, not all applications have the HUD interface working properly. One such application is LibreOffice. The developers are still working out the kinks, so at the moment, it is not enabled. It can be enabled by installing the app lo-menubar. I'm fairly certain, by the time 12.04 is released into the wild, all major applications will enjoy HUD integration.

One of the goals for the HUD was to help further the integration between QT and GTK interfaces. Obviously, the HUD does far more than that. But, if you think along the lines of unifications, HUD does a perfect job of making everything on the Ubuntu desktop seamless. No more would developers have to worry about QT vs GTK menu appearance or functionality. Just develop with HUD in mind and you're good to go.

And, of course, HUD also makes using the Linux desktop even more easier. No more will the end user have to poke around menu systems trying to figure out where Conditional Formatting lives or where any given filter is on The Gimp. HUD takes care of all of that.

In the end, what HUD does is advance the Ubuntu Unity desktop ahead of the competition. I think we were all fairly certain it was nothing more than a matter of time before Mark Shuttleworth and the Ubuntu developers managed to make the Ubuntu desktop an obvious choice. But most never assumed they would take the Linux desktops light-years ahead of the competition.

Bravo to Ubuntu and Unity -- what you are doing is nothing short of incredible. You keep up this type of work and there will be no doubt who is the reigning King of the desktop.

About Jack Wallen

Jack Wallen is an award-winning writer for TechRepublic and Linux.com. He’s an avid promoter of open source and the voice of The Android Expert. For more news about Jack Wallen, visit his website getjackd.net.

Full Bio

Jack Wallen is an award-winning writer for TechRepublic and Linux.com. He’s an avid promoter of open source and the voice of The Android Expert. For more news about Jack Wallen, visit his website getjackd.net.

The reason is in Jack's own words... "For example: Im in Firefox and I want to go the extensions page. I hit Alt (to open up the HUD) and type add"
If you're identifying the object you wish to manipulate as "extensions" why wouldn't you type ext? And if ext is typed HUD doesn't end up anywhere near the desired function.
If the name of the destination is known seems HUD works. Generally not everything is recognized by one and only one name. Again see Jack's example. Or go from east coast to central US. In one place they'll put your groceries in a bag, in the other they'll put them in a sack. People just don't use terms consistently and so far as I can tell HUD only works if terms are used consistently.
The menu allows navigating the system to known commands, exploring the system for unknown features, e.g. "I want to do something like x I wonder if I can find something that does it," and supports finding infrequently used commands whose menu location might only be vaguely remembered and search term not remembered at all.
I guess if you know/remember the key terms HUD is great if you also want to use the keyboard. My own experience is I have better spatial memory then word memory. I can find things in a menu because I remember where it is not what it is named.
HUD doesn't pass go for me.
And oh, I'm male. Seems I've read reports for at least the last ten or fifteen years suggesting men have well developed spatial memories. If that's so seems HUD is running counter to the inate skills biology has provided me.
And, dare I say it, I like the ribbon. It reminds me of the Quattro Pro DOS menu. Seems the same idea except QP didn't have the icons only text.

Sometimes I think I'm missing something. Regarding your example above, if I'm in Firefox, and I want to go to the extensions page, why would I type add?
Wouldn't I type extensions? And if I did, would that take me to the same parts of the menu that you describe above?

I used Ubuntu as early as 2006, the version was 6.10, and updated every half a year, till 11.10. I think I failed in using it. I need to change windows of different programs frequently, which is not easy to find them in the Unity, when the number of programs are more than ten. You might be right. The menus become larger and larger. However, I don't think it's good way to type. First, Ubuntu is available for many languages. Is the keyword efficient in other languages? Second, if I don't know which command is correct, could the HUD give me a suggestion? Third, I need WINE to use some Windows programs, I'm afraid this design would make WINE more unstable.
I think Ubuntu should leave a traditional area where its new design breaks out. A traditional menu on the bottom won't take too much space.

Sounds great - searching through menus on applications to get quick access.
Now, let me think - which apps would I run on Ubuntu that I would need to search the menus.... hmmmm.
An email client, a browser, ummmm oh yeah thats it.
Any business app won't install and run on Ubuntu as they are all written for Windows!
Great idea - just need to put it on an OS that someone actually uses for work. I'm sure there are a couple of thousand people out there.

Not so long ago, Yahoo was king of internet content. If you wanted to find interesting web sites, you could go to Yahoo and crawl through their hiearchial categories. The same was true with file management and email management. Those who wanted careful order filed things in a hiearchial folder arrangement. But there came a time in which the internet just became too big to handle via a menu structure of categories. The same has become true with the files and email on our computers and is becoming true with the proliferation of functions in complex software applications. It is becoming more unworkable to rely on categories of menus to find functions. So, it seems to me to be a natural progression from searching the internet to find content, to searching our PCs to find files to searching an application to find a desired function. Of course, the devil is in the details. Google became king because they pioneered an easy to use search interface. I'll be interested in trying out 12.04 HUD to see if Ubuntu has implemented application function search effectively.

I don't know why people would bother with this type and that type of interface. Seriously, I don't want to wine about anyone is not thinking out of the box, but the future should be voice and hand gesture navigations. Such devices like keyboard, mouse, and trackball should be gone. There should minimal learning curve to use any equipment or computer.

I think the HUD is a brilliant idea as long as the normal menu is also there because you might not know all the options available in a certain app. I'm not a big Unity user myself but I have to say that I think HUD will be very popular

Windows 8 does something similar from the new start screen, Win 7 and Vista offered a varient on this from the start menus integrated search box, KDE has been doing the same thing for a while. The Unity HUD seems to be solving a problem that does not exist and fails to be simple for new users or powerful enough for experienced users.
The only real innovation in the area of HID's seems to be Siri. This is a painful admission as I am not an Apple fan but the idea that we can move on from keyboards and mice is one that needs to developed further.

Mac Quicksilver and even the Mac spotlight search works like this. It's well cool to not have to move your hands over to the mouse, and navigating menus is v e r y slow in comparison.
Yes, browseable menu options are important when familiarity is not there. Have they been removed? Aren't they still there?
This is an enhancment for experts. Some users can't even touch type but why should my interface be built just for them?

Question, I don't use GIMP very much same goes for Photoshop but when I'm editing a picture and I want that cool effect that I just don't know the name of. With a conventional menu you can browse the options and click until you find what you want (ah, sepia... that's the cool effect I was looking for). How do you do that using the HUD, if you don't know what you're looking for. Also goes for a new program, you don't need to know all menu items, often the most important options are in the GUI but when you need to tick that small radio button to continue with the program. PROBLEM! Most of the time developers bury small but sometimes important options somewhere in the menu. IMHO the HUD expects you to know a program in and out. What if someone wants to add an extension to a program and begins to type exte... instead off add?

There is nothing worse than an interface that assumes your both know how to spell and know the correct term name for the item you are looking up. This is why the hierarchical menu system was invented.
In fact, I mentioned to Microsoft years ago that the Windows key when held down for more than 5 seconds should to a HUD to overlay the names of the screen elements then allow either another key press or mouse click to start a tutorial about them.

The difference is that Ubuntu is much more well adopted linux among the desktop/business community in 1st world western nations not to mention 2nd and 3rd world nations. Although the other desktops do this, they don't use the Alt key to bring up the search. (I'm assuming this and I'm sure you can assign a hotkey to these functions, but regular desktop users aren't really familiar with hotkey assignment) Having to go to your mouse to open a program is a total pain. If you can just hit the Alt key and down arrows, then wham bam you're there. So back to the difference: Ubuntu will give much more exposure to this methodology than any other linux disto. I don't even use it because I'm on low powered computers. I use bodhi. When I upgrade to a computer that can handle Ubuntu/Unity I will try it. If they keep it up and continue to add the latest greatest UX/AI methodology then Ubuntu has the potential to become a leader in the desktop industry.

KDE has had a search option like that for years with a far more conventional start menu that any M$ user can grasp. Just pull up the start menu and type it in, just like in Vista. I'm sorry but even LXDE or xfce are more user friendly. Heck, I say get Mint with KDE or PC-BSD with the same. Using more then one desktop and switching between them is straight forward in Linux. You can find the exact names of your desktop that you want to add with google. If you're stuck with Gnome or Unity, escape is possible.

It's not for me me as I don't care for Unity (or Ubuntu fort hat matter) but this is just a variant on the old context-sensitive help search that has been around for many years. It's a nice additional feature to ahve available but it's nothing new or earth shattering.

1. Touch interfaces like tablets, that do not have Alt, or any, keys?
2. Our "muscle memory" that acquires reflexive knowledge for the right spots to tap/scroll/click for frequently used apps and functions? That is always a big issue for ANY kind of interface change.
I don't really need to "search" for items I use on a regular basis - like my car's controls, my hands/fingers know just where to go for what I want without looking (much). Now if it could read my mind for my "intent", that would be cool (and creepy...).

As someone who has used and enjoyed Enso (www.humanized.com) this sounds most promising. Great article. The last paragraph was, however, unsettling. It didn't border on fanboyism; it dove right in...head first...with it's mouth open. Scarey.

Installed 12.04 just yesterday to understand what everyone was talking about unity. I've been using Xfce since Ubuntu left GNOME. I couldn't figure out the HUD / interface. Even as an experienced user it wasn't intuitive. If Ubuntu is attempting to hit the novice market, they ought to make it apparent what the user is supposed to do. If the interface is drastically unlike Windows, help the user understand how to get there, and that doesn't mean ship with a user manual.

I hear what people are saying vis a vis keyboards vs mice but....
Surely the HUD would be a boon to desktop users rather than tablet users. Its not aimed at tablets since those with tablets will not want to use the keyboard so much. But for desktop/laptop users this provides a nice alternative to the traditional menus. It could be the missing link....
I myself use Cinnamon over Unity: but I will revisit unity when 12.04 comes out. There is a lot of talk about smaller footprints and faster working out there and I want to look again. This would mean that I could enjoy unity and would get around many of my reservations about the loss of traditional menus: leaving as my only objection the ugly and still not very configurable left sided dock.

Probably his best character ever, certainly the only conflicted one with an irredeemable dark side.
And probably the only time you could call the very talented Patricia Neal "hot!"
Great film. Don't make 'em like that any more.

My one big objection to Unity is that is puts the window menu at the top of the screen, ala MacOS. This may have been great when screen sizes were very small (the original Macintosh had a 9" screen!), but now that 19-24" monitors are commonplace, it makes little sense to have to move the mouse to the top of the screen to access a menu item, just to save a little screen space. With multi-monitor setups becoming increasingly common, it makes even less sense to have to move your mouse cursor from one screen to the top of a screen on a different monitor. Also it is just plain confusing for a menu to change context when you click on a different window. Somehow everyone bows down to Apple on issues like this. I beg to differ. This is the main reason I will not buy a Mac, and also why I will not use Unity. Also Unity breaks compatibility with applistions, they have to be modified to work with the Unity menu system. You will never get al applicaitons to work with Unity properly, because Linux is a free system, developers are not bound by the Unity interface requirements, indeed modifying them to work both with Unity and on another window manager is more needless complexity. One of goals of human interface design is consistency, this has always been Linux's achilles heel, there are too may different ways to do things, Unity is not helping here, look at LibreOffice, it still is not integrated with Unity. Stop the madness! Just say no Unity, at least until Shuttleworth removes the top of screen menu system.

No thanks Mr S! Unity had me scuttling to Linux Mint, via Vector and Fedora. HUD doesn't make me want to reconsider. A shame really, I've used Ubuntu as a first choice since Dapper, whenever that was. On a smartphone or tablet, I'm very happy with the touchscreen interface, but on a desktop, I want the speed and precision of a mouse and keyboard.

It would make finding things that you don't have a clue of where to start, much easier. The disconnect for interfaces has always been that you have to know exactly what you're looking for in order to use it. With HUD, you'll be able to type in something close and it will lead you to find what you're looking for.
For those that don't like to type to find stuff... why the world are you using Linux in the first place. You need to get back on Windoze and leave Linux alone.

I don't know why people are hating on this new HUD. It looks like it is taking the search in the start menu of Windows 7 to the next level. I'm going to be honest, one of the reasons why I don't use Ubuntu as often is because of that search functionality.
For The point and clicker, they will have to click on the start menu, click "all programs", then accessories, then click on "command prompt". For me I press windows key and then CMD. That takes a third as much time. People don't realize that keyboard shortcuts can be way faster than using a mouse.
Also in terms of OSX, You are likely just looking for nested menus in the OS. This looks to me like HUD will help you find nested menus within specific applications.

Just in response to the comments above. You can turn HUD off and go back to your comfortable command line. Remember that linux is about choice. Ubuntu's goal is about making the linux desktop a good alternative for newbies and putting linux in the mainstream. Wouldn't you be happy if linux became mainstream and more people used it? There are plenty of distros that cater to hard core linux people out there. Don't attack a distro because they want to go into another market that isn't yours. Just switch to another.

Anything that requires typing and gets you to have to switch to a keyboard from mouse is just plain stupid. Typing requires prior knowledge or it can lead you on wild goose chases. It slows you down because you change both eye level and hand movement as you switch from mouse to keyboard.
Typing on a touchscreen or tablet is painful, so Unity which is supposed to be headed towards tablets is headed in the wrong direction with both Dash and HUD.
News flash: most desktop users are not journalists or programmers. They click, drag and drop and do not type.

Voice commands/input are nice here and there, but a speech interface is not for "moving on" from touch interfaces, rather a supplement. Just imagine an office cubeville with everyone babbling at their computer AND their phones: "let me look that up - open customer accounts (not you, sir, the computer) ...". A modern tower of Babel ;-)
Don't know about you, but I am pretty sure I can type and mouse (and otherwise control tactilely) a computing device for a lot longer than I can talk to my computing device, and more accurately.

When he said, "I want to find EXTENSIONS... so I type in ADD,,," er, what does Attention Deficit Disorder have to do with... OH RIGHT, people can't find things on their own because IT'S TOO COMPLICATED for them?
On another note, the ONLY way I've found, to use the "ribbon" in ms Office is a little add-on (ha, got ya) that I installed where... you can type the name of the command you want, say "envelopes" and it brings up the most common choices for manipulation of envelopes. I ignore the ribbon entirely now, it's useless.

"Also Unity breaks compatibility with applistions, they have to be modified to work with the Unity menu system."
As long as you are using KDE 4.8 or Qt 4.8, all KDE and Qt apps are compatible with all Ubuntu features (HUD, DBus menu, menu bar at the top, whatever). Ask GNOME why didn't they integrate the Ubuntu patches into GTK mainline. In fact, there is a KRunner plugin to bring HUD to KDE, and, as long as you are running Kubuntu, or another distro with mainline Qt 4.8, that will work.

There's not a lot of novelty in touch UI design, regardless of who makes it. But yeah, it's unlikely you'll grasp that the futue of all devices is going to be at least partial touch interfacing until it smacks you in the face like a fresh tuna, so I'm not going to both to explain.

How does this HUD relate to the iPad at all? Honestly I don't see the connection - if anything, I would consider it the opposite!
As for Windows 8, it is a touch-based interface and nothing more. It is the least iPad-like I have ever seen, in fact, other than IKEA terminal stations and grocery store self check-out lanes ;).
And it is perfectly usable in day-to day life for work and play. It works great! I use It 100% as my main OS (once I got the sound drivers working).

Wow. I think you've got that backwards. When you have to leave the keyboard, you're wasting time. Granted, the hud may require you to remember how to spell words and stuff instead of clicking pictures. Maybe the Sesame St. Click and Learn would be better suited for some people rather than a computer.

Wow, that's an eye opener linuxcanuck! I didn't know there were users who didn't like using the keyboard. I guess I'm in my own little world here just typing messages and using words all day. I never thought that some users were only using their computers to watch videos and read content. I guess you're right: there is that class of user, and this user will probably find no use for the HUD. Ok, good point. Mind you, I don't think there's any indication this feature was aimed at this kind of passive web user? Just because non-professional passive users exist doesn't make the HUD stupid for those that do like to actually generate content and do some actual work?

Any non-tech type folks I know generate at least a bit of text for emails, FaceBook posts, and their jobs when using PC's. Even most GUI-based apps have the requirement here and there to put some text in a textbox widget. Not one size fits all in about any kind of user category.

None of you understand the concept of using the keyboard over a mouse. I can run circles around just about anyone using a mouse, while never lifting my keys from the keyboard. Once you figure this out, you will see how you can reap the benefits of a full-time keyboard interface.
Remember, Microsoft has gone to great lengths to keep keyboard shortcuts alive in their newer products because it is the most efficient way to work.
Do any of you even use the commandline interface? Sad if you do not.

News flash: I'm over 52, and still type at 74wpm, and you? No hunt and peck here (though in my teens I was clocked at 108).
But that doesn't mean I WANT to type, when using the mouse to navigate is SO MUCH EASIER not to mention faster. Unity and HUD are a huge step backwards for Linux desktops and laptops too, Unity and HUD may have some usefulness on touch screens such as iToys and small tablets. Not for serious users, and certainly not for people who use their desktops and laptops for actual WORK. Note to developers: hardware support (another gripe about Ubuntu, dropping support for very common wireless cards) is not bloat. Same for usability, functionality. Give us Gnome, or give us...something BETTER. PLEASE!

I have to agree...once a person actually commits to the keyboard and sees all the shortcuts, sticky key option, nav within page using cursor, function keys, etc among sooo many others, one gets annoyed with having to reach for the mouse. IMO anyway.
The mouse is still indispensable though, but, surely it must evolve soon, no? I'd hate to miss out on potential marketing but what about a joystick config? Or has that already been done I wonder...

When I play WarCraft 3, I used keyboard all the time when I use my mouse to click and right click. Remember, I just need to type one letter to tell my hero or army what to do, not "add" or "find" or "help". If you use this way to use your system, it's OK. However, you don't need to be sad. The advantages of the keyboard is the combination of few keys, not the full-length words. If you like this, I'd rather like to use shell to do the same thing, where I could use Tab to finish a command.

The state-of-the-art input interface is Touch... touch devices like smart phones and tablets have killed the mouse... the physical keyboard is gasping for air as we speak... on my laptop, I use the touchpad whenever I need to move the cursor; otherwise, I use the keyboard... I prefer a touch interface, but other than speaking to the device or having it read your thoughts, you'll need a way to input text... this will be done with a touch keyboard...