Language-Based Interfaces, part 3: Report Card for Ubiquity 0.1.1

I believe in tough love for my brain-children. It’s report card time. Let’s see how Ubiquity 0.1.1 holds up to my exacting standards.

For ease of learning, it should:

Accept input in something very close to the human language I’m already familiar with.

B. It’s cool that I can put the modifiers in any order and use “this” and stuff, but the commands-are-hyphenated-requirement means that I often find myself typing awkward stuff like “add-to-calendar this”, which is not natural at all.

Give me clues about what commands are available.

C. We’ve got the “command list” command and the context-menu to help people learn commands they don’t already know about, and you might notice a few commands in the suggestion list on your way to the one you want. But judging by the number of requests we get from the community for commands that already exist, I think this is still a major unsolved problem. And that’s just the commands you already have installed — finding out about a command that’s on the web somewhere that you might want is currently nearly impossible. We need a search engine for commands!

Give me clues about what I can type next.

B. I think the suggestion list does a pretty good job of this — but it will do a lot better when there are more specific nounTypes and better sorting.

Give me clues about what the current command will do if executed.

A. I think our preview feature pretty much nails this one.

Give me suggestions about other commands it thinks I might be looking for.

F. Nothing is implemented for this yet, at all. I have some thoughts about how to implement synonyms, which I’ll be explaining in an upcoming post.

Help me understand what ranges of arguments to a command are valid, and what the arguments mean.

C. The completions from nounTypes, combined with the preview pane, can give you a pretty good idea in some cases. But too many commands are using arbitrary-text arguments instead of more specific noun types. Mostly, we need to implement more noun types, and implement them better. Also, we don’t have a sensible system of defaults for command arguments yet, just a bunch of ad-hockery.

Propose commands appropriate to my working context or to the type of data I have selected.

D. While this is technically implemented in the form of noun-first completions, the generic arbText commands push the specific stuff off of the bottom due to lack of sorting, so it’s an unusable feature.

For efficiency, it should:

Allow the user to start with the noun or to start with the verb.

B. This works! But noun-first completion isn’t very useful right now, again, due to lack of sorting.

Let me autocomplete a partial word with a keystroke.

D. Not implemented yet at all, though I can get some of the same effect by leaving my partially-typed word in place, hitting spacebar, and going on to type the arguments.

Recognize words even if they’re super-abbreviated.

C. You can abbreviate the verb and most nouns, if you start at the beginning, and usually get the parser to recognize what you want in two or three letters. But you can’t abbreviate starting in the middle, or use a disjoint completion.

Remember what suggestions I’ve chosen in the past and pop them up next time I give the same input.

F. Not implemented yet at all. I’ve got some ideas for how to do it.

Let me partially enter something, see the suggestions, choose one as mostly-right, and edit that one some more before executing it.

F. Not implemented yet at all. However, I hope to implement the autocomplete-with-a-keystroke feature (see above) in such a way that it achieves this one too.

Guess, from my context and my selection, what I want, and fill most of it in for me, while letting me easily override it if it’s wrong.

C. Noun-first completion and interpolation of the selection are a good start. But it will be a lot better once the suggestions are sorted, and once there are a few more magic words, and once interpolation of the selection can automatically go to any argument in the sentence. Ubiquity should be able to guess what I mean from context much more often than it currently does, as I described in this post.

For expressiveness, it should:

Handle commands with multiple arguments, including optional arguments, that can take various data types.

A. We handle this pretty well! We just need arguments to be able to define defaults and whether they’re optional or not.

If I have data selected, let me use that selection as an input for any of the multiple arguments — or for none of them.

B. Easy to do this with “this”. Need to make it better at automatically figuring out where the selection should go when there’s multiple arguments, though.

Let me chain commands together, with the output of one going to the input of the next, like Unix pipes.

If my input could mean more than one thing, give me a sensible way to resolve the ambiguity.

C. Suggestion list. It errs on the side of putting too many possible parsings in the suggestion list, though, so potentially useful completions are pushed out by trivia.

Let me compose a complex command out of small parts, in the flexible way that natural language does.

D. You can’t do it yet, but I think we’re laying the foundation for it. The command chaining is a step in this direction, for instance. The potential is there to do a lot more of this.

Let me save a complex command that I’ve created and give it a simple name so I can re-use it in the future.

F. Not yet implemented at all.

Give me an easy way to create my own commands — and to share them with others.

A. I’ve been amazed by the quantity of third-party commands which have appeared and the speed with which they did so. We only launched last week and we already have a thriving third-party development community. What we need most now is a search engine for commands so that people can find what’s out there.

Needs Improvement

Given all that, here are the improvements which I think are most important to make to the parser next:

Sorting the suggestion list, both by the inherent quality of a match, and by what completions the user has chosen before for similar input.

Get rid of the hyphenated-command-names!

Add an autocomplete function, perhaps triggered with the tab key.

Allow abbreviations to match to the middle of words, not just the beginning. Also allow disjoint abbreviations, such as allowing “yts” to match “youtube-search”.

Implement synonyms for commands.

Additionally, we need to build that search-engine for commands, which is not technically a parser improvement but will make the Ubiquity as a whole much more usable.

Finally, there’s the internationalization issue. Making the parser easily localizable to various languages is a huge priority, but it deserves a post (or several) of its own, so I’m not going to get into it here.

One thing with sorting that I think would be a great way to get rid of some of the junk suggestions would be to only show the “search” command with arbitrary text, as opposed to all of the various search engines separately. The search command already does a good job of figuring out which search engine you want, so you should capitalize on this in order to keep the suggestions clean.

I think another good future for Ubiquity should to be integrated straight into the url bar.
I’ve been testing Chrome for a week know and the Omnibox’s integration of search, history…is really faster than going to a special search field in FF or doing Ctrl for Ubiquity.

I hope that Ubiquity will eventually find its way to be integrated in Firefox straight into that bar.

Kourge: Thank you very much for that link! I will be looking at that code for inspiration as I improve the ubiquity matching algorithm — or even incorporating parts of it wholesale (since it is under a license that I believe is compatible.)

FaroCastiglo: That’s a good idea too. I’m going to have some stuff to say about this in my upcoming article “Anarchy in the Namespace”.

D. Not implemented yet at all, though I can get some of the same effect by leaving my partially-typed word in place, hitting spacebar, and going on to type the arguments.

If I could make a suggestion specific to this: one of the things that is most irritating about completion, as anyone who has used a lousy cellphone texting program knows, is that when something is autocompleted but is wrong, backtracking is a real pain. I would like to suggest that I never have to delete more than one character from something I didn’t type. For instance, let’s say I have Ubiquity open and I’m trying to type “tab”. I type the letter “t” and press space, only to remember that Ubiquity doesn’t (yet) remember what I chose last time when I typed “t”. I only have to backspace once to get back to what I’m typing – and that is great! I don’t want to lose that ability when you implement autocomplete and command memory. Perhaps you could have a bolder font for what I have typed and a lesser font for what you have autocompleted. Then when I backspace once into the “lesser font,” it all disappears and leaves me at the tail end of what I have actually typed, ie the bold font.

Something else unrelated I was wondering about the other day – what if you could tie Firefox’s menu bar into Ubiquity. You know how Eclipse has a pretty fantastic options editor with it’s own text bar for searching for options quickly? Well, what if there was a command in Ubiquity called “menu” or “configure” or whatever would be appropriate, and that command was tied directly into the GUI for File/Edit/View etc. So I could type “menu Saved Passwords” and inside Ubiquity there would be the Saved Passwords button, which I could then click. Or I could type “menu Passwords” and all the Passwords-related options would be there all in a row. The reason I was hoping this is possible is because then that could eventually lead to Firefox getting rid of the titlebar and menu bar the way Chrome has done, which feels like a surprising amount of screen space in practice. Come up with a good way to tie Bookmarks into Ubiquity and you could get rid of that toolbar too.