The Personified User Interface Trap

Personified user interfaces, like chat bots or agents, are the new thing once again. But despite advances in artificial intelligence, they still have many issues and drawbacks compared to direct-manipulation interfaces. There was a debate around these interfaces in the 1990s, and it seems to be bound to repeat itself.

These bots act like people, in that they talk to you via text chat and try to understand free-form text. They don’t attempt to pass the Turing Test, but they promise to be smarter (and more attentive) than an overworked call center worker tending to a dozen people at the same time.

Bots’a Problems

So what’s not to like? Bots have a number of problems that have been solved, or that are at least understood well, in graphical user interfaces.

Bots are not discoverable. It is very difficult to find out what a bot will respond to, and takes a lot of trial and error. Making it easy to find things is the first task in designing user interfaces (GUIs). Even if you can’t tell what something does in a GUI, you can see that it’s there and can try it. You don’t know what sorts of constructs a particular bot will understand.

Bots hide information. A graphical user interface lets you browse. You don’t have to specify exactly what you want, you can look at pictures or a list and figure it out as you go through it. A bot can list things, but it can’t easily present an endlessly-scrolling list with lots of items. Getting a long list from a bot is about as useful as having a call center agent list your options on the phone. By the time they’re done reading them all out, you’ve forgotten half of them.

Bots require syntax. You need to specify your operations in a sort of syntax. That can either be a single sentence (if you know how to construct that so it’ll be understood), or a back-and-forth between you and the bot where it asks you for missing information. In graphical user interfaces, you can drag and drop, select from lists, swipe, shake, and use lots of other gestures.

Bots are never as smart as you think. We tend to assume that because we can talk to a thing, it will understand and have all sorts of external knowledge that it can’t possibly have. That is not an issue with graphical UIs. Assuming the bot will understand and then having it come back with what is basically Huh? is frustrating, especially because it’s often hard to impossible to figure out what you could have done instead.

There are a number of points in that article, not all of which apply to chat bots. But the gist is that Shneiderman wants people to see as much data as possible and be able to touch it directly (using direct manipulation), whereas Maes argues that for large amounts of information, you need agents to sort through it all.

The problem is that despite Maes’ insistence that agents learn and understand what the user wants, they are still extremely dumb. All I need to do to see the state of the art is to look at my Amazon recommendations: Oh, you bought a battery recently? You must be into batteries! Here, look at all these other batteries we have! The same is true of things that you don’t typically buy very often, like cameras. Look at all these other cameras! This one would go great with the one you just bought! Also this completely incompatible lens!

Maes also closes by saying that agents are not an alternative to direct manipulation, but require it as their own user interfaces. That may be the case for some of them, but especially with chat bots, the bot is there instead of the graphical interface.

There is also a strangely elitist vibe to these bots that bothers me tremendously. The people creating these things assume that we like to think of computers as our servants. They ask what we wish them to do, how we’re doing, etc. I don’t want a servant, and I don’t care if it’s a person or a machine. I don’t want to have to go through some real or pretend intelligence when performing a task that could just as easily be done with a few mouse clicks. That whole notion of a personal assistant just rubs me the wrong way.

It also takes control out of my hands. Instead of being able to just perform a task, I have to go through an intermediary. Why? All the uses I’ve seen so far could have just as easily (and much better) be done with a simple app or website. The indirection a bot creates interferes with my sense of control and involvement. I’m no longer the active party, I’m just yelling instructions. That seems like an odd way of interacting with a machine.

Bot Niches

Chat bots are a reasonable solution when all you have to talk to a service is a text connection, which is why they are marketed as a great way to reach people in the developing world who have cell phones and can text, but whose connections are slow and unreliable. But with a reasonably good connection and screen, it’s much more comfortable to be able to use a well-designed user interface than talk to a bot.

I can also see bots being useful for people with vision or other deficiencies, since they can help guide them when they can’t easily see the interface or interact with it.

Outside of these niches, I really don’t understand the appeal of bots. The way they act and force me to interact, and the way they restrict my access to information, in no way make up for whatever advantages they might have. More than that, they strike me as disempowering the user and thus taking many steps back from modern graphical user interfaces that literally let you touch the actions and data you’re dealing with.

Robert Kosara is Senior Research Scientist at Tableau Software, and formerly Associate Professor of Computer Science. His research focus is the communication of data using visualization. In addition to blogging, Robert also runs and tweets. Read More…

(1) All interactions — direct or not — are governed by a syntax and I have yet to see an intuitive one for a complex system. Mobile device syntax is quickly becoming a nightmare. Look at the Apple Watch. Complexity overload.

(2) The idea that all information I need can be right in front of me is a powerful, seductive idea. But is it true for complex systems? I find myself using Google (a bot of sorts) to discover how to use other pieces of software.

(3) That which makes things better for less abled bodies will benefit everyone. When it works there is a *huge* advantage to hands and vision free interaction and I suspect that as we move to always on and wearable devices with no screens (watches, embedded systems, wearables, Amazon Echo, etc.) we’ll be increasingly reliant on bots.

The good news is that tvs and phone screens keep getting bigger so it probably won’t be one or the other.

On your point 1: That’s a moot point isn’t it? If everything needs a syntax, then replacing one syntax with another isn’t the issue, it’s whether it’s any better for the tasks at hand that counts.

On point 2: You’re going to be hard-pressed to convince anyone that they’re going to use something they don’t know is there. Given a sufficient level of complexity in a system under use, there is of course an argument for visual simplification, but if you observe people who use complex systems, complexity per se doesn’t rank very highly in their problems list. Have you seen Facebook recently? They don’t do anything in the UI that doesn’t pass stringent usage metric multivariate tests. The result? Show everything. Facebook users are happy. They’re not attempting to solve a UI complexity problem with their bot experiments.

On point 3: You seem to be implying voice control. If so, then that’s a bit of a different issue if there is no opportunity for direct manipulation at all.

This article expresses very well-made, entirely sensible observations while offering historical context on the issue of bot design. A pity then that it will be completely ignored by designers who neither consider, much less know or care about such issues.

Those who concern themselves seriously with interaction design as a discipline will observe that product managers and self-styled “designers” value perceived novelty and the dominance of form over function above just about everything else. They will consistently ignore all the boring old academic stuff that Robert is talking about. The people who would use such systems seem to be as highly tolerant of poor experiences as those who design them as well. This is, sadly, the reality we are failing (collectively) to deal with I think.

Quartz is in fact quite fun for a few minutes, although quite why Amazon does what it does with “recommendations” is completely beyond my comprehension. Could it be that this too is a result of them simply not caring about function?

Well, I don’t think their experiments are a bad thing anymore than I dissaprove of data art. I think novelty IS a part of it and I think it’s great that people are experimenting with form over function. That’s art.

And if you’re designing a chat bot in 2016, you clearly you have different expectations about your userbase.

lastly, I enjoy using Siri and Ok Google to interact in limited ways now and I look forward to future bots which supplement direct manipulation (hey read me what’s on the screen) or in some cases completely replace it (i want to watch —— on Netflix).