Bret Victor's Outspoken Views Against the Current State of Interaction Design

Even if you've never heard of him, Bret Victor ought have some credibility as a former Human-Interface Inventor for Apple, where he worked on "pervasively direct-manipulation interfaces (where the user does his thing by moving and gesturing with meaningful objects, instead of relying on verb buttons and other indirect controls)" among other things. Remember the Microsoft "Vision of the Future" that made the blog rounds, including ours, two weeks ago? Victor's frustration with it is palpable:

I had the opportunity to design with real working prototypes, not green screens and After Effects, so there certainly are some interactions in the video which I'm a little skeptical of, given that I've actually tried them and the animators presumably haven't. But that's not my problem with the video.

My problem is the opposite, really—this vision, from an interaction perspective, is not visionary. It's a timid increment from the status quo, and the status quo, from an interaction perspective, is actually rather terrible.

On his blog, Victor lays out a well-considered and well-communicated diatribe against current touchscreen technology, dismissing it as "Pictures Under Glass [which sacrifices] all the tactile richness of working with our hands, offering instead a hokey visual facade." His suggestions for what interface designers ought be looking into for the future is pretty exciting, and we won't spoil it for you by revealing it here. Go read his essay. Now! (Even if you're on a touchscreen.)

The latest design news, jobs & events. Straight to you every other week.

Join over 300,000 designers who stay up-to-date with the Core77 newsletter...

9 Comments

One could argue that if the objects with which we need to interact were all touch screen based, man would not have developed such intricate instruments as hands.
Fact is: we have and might as well use them, and so the rest of our body.
Victor does not say touch screen is flawed. He says it should not be the central feature around which a vision for the future is build.
Well, call me a worrydream fan. I think his brief rant makes Microsoft's vision look rather shortsighted indeed.

I can't even begin to imagine what the future might hold for interaction design.
I think for the most part the world of human-computer interaction is advancing quite steadily and positively. Manipulating actual things on the screen (representations of objects atleast) is still more natural and intuitive than using a mouse and keyboard to manipulate those same representations.
Bret says that manipulating "Pictures under glass" is unnatural and absurd, given the vast movement capabilities of the hand. Although I can see potential uses of 3D cameras to allow for hand movements that go beyond manipulating mere 2D images, at this point I think we've got as much as we need from computer interaction.
Why do we need to chase the perfect way to interact realistically with simulations of real world things. Why
Touchscreen devices are a better way to interact with phones and the internet, but it doesn't replace actually doing the thing that touchscreen device is simulating.

Good comments here: Pictures Under Glass is fine for 2D objects. He seems to skip over the fact that this is pretty early days for devices like this. But what about full-body interfaces like the Kinect? How can that work for a hand-held device (the return of the PowerGlove?).

As far as the performing additional functions I think apple has pretty much hit it's uselefulness limit with the 2 fingers to scroll and double tap(which is annoying).
When you get into actually manipulating things other than photos.. you are talking about design work which after an 8 hour day would require your arm laying down RESTING and using 1-2 fingers. Instead you are saying it would be be better to have our entire upper body in use. Like carpal tunnel isn't bad enough.

"With an entire body at your command, do you seriously think the Future Of Interaction should be a single finger?"
First, pads and tablets directly relate to pieces of paper and clipboards, things we've used for a very long time. Touch eliminates the pen, which is simply a tool for manipulating the paper, and allows direct manipulation.
Second, as I recall the iPad supports multi-touch, so you can use a finger, or two, or three, to swipe, pinch, rotate, and perform additional interactions.
Third, I think that voice is going to play a major role in the "future of interaction."
Fourth, that single finger interaction is so simple and easy that even a toddler can use it. Anyone remember the story where the toddler who'd used an iPad got upset when she came across paper magazines that were "broken"?
Finally, I was lying in bed last night reading. I had one arm behind my head, and held the iPad in the other, propped on my stomach, and used my thumb to scroll up, down, and through articles in Zite and Instapaper.
Single finger (thumb) interaction was perfect for that use case. It worked and worked well.

At this point in time touch screen interfaces FAIL.
They are a simple reproduction of what was once there physically.
In this sense the physicality and pleasure from pushing a button is much more effective than a touch screen. Also in some instances creating more work for the user. Plus this push for touch screen computers. I'm sorry but I can move my hand a few cm and get it across a 27" screen. If this were to happen i'd physically be doing 10,000 times more work physically. I consider that a fail. Plus I can text message on my phone incredibly fast without looking but now that will never happen in the future because buttons will be obsolete. Not to mention Apple's auto correct feature is absolute garbage.

Here's what I tweeted today after reading the entire piece:
I get Bret Victor's rant [http://worrydream.com/ABriefRantOnTheFutureOfInteractionDesign/]. It makes a lot of logical sense. My prob though is it doesn't match my experience ...
... I love touching glass and obviously so do a gazillion others. My son took to it amazingly quick @ 2yo (as have many a toddler) ...
... a focus on tangibility is interesting. There are examples of pure tangible or hybrid UIs, but a condemnation like this is overstated.
Where I most certainly agree, is that we are desperate for OTHER visions of the future besides this one. I still love this one: http://www.youtube.com/watch?v=IX-gTobCJHs
Make me a Nokia Morph ... Hell 1/2 a Nokia Morph and I'd be like a pig in shit! It is a hybrid of Interface under glass and tangibility presented in a future scene that while not the MOST inspirational, should definitely inspire.

Great read. I can't help but think of the craft from District 9 and the tubs filled with gel in which the the pilot must submerge his hands to manipulate it. Makes me wonder if haptic or other sensory feedback would be used.

Reporting by Yin HoIn less than two weeks, judges start reviewing design submissions for the Fast Track to the Mobile App contest—an international Windows Phone app design challenge. The competition, launched last month by Core77 and the Windows Phone design team, challenges designers to create work productivity solutions tailored for...

All, an unconventional retrospective of artist Maurizio Cattelan opened at the Guggenheim in New York on November 4, and with it, appropriately, the museum released its first mobile application to support the exhibition. Although a little late to the app game (with Google, galleries, and other museums paving the way),...

Designing the shopping experience is a large challenge that we're excited to watch the progress of, since nobody knows what it will really turn out to be. Recently Apple and Adidas have both proposed and are now beginning to advance new shopping experience designs that make very different bets on...

The BBC World News has launched a great interactive piece in anticipation of the world's 7 billionth person, whose birth we will celebrate this coming Monday, October 31st. Enter in your birth date and the program will tell you, of the people alive that day, what number person you were...