A collection of observations, news and resources on the changing nature of innovation, technology, leadership, and other subjects.

July 11, 2005

3rd Generation User Interfaces

One of the most exciting areas of innovation is emerging around what I'd like to call 3rd Generation User Interfaces or 3G/UI. 3G/UI, inspired by game players, promises to bring highly visual, interactive interfaces to all sorts of applications in health care, education, science and business. The reason this is such a huge deal is that every time a new paradigm emerges in the way people interact with computers, we've seen all kinds of new applications begin to appear, qualitatively better and different from anything before. Furthermore, innovation in user interfaces soon gives rise to innovation in the programming and computing models needed to develop and run the new round of applications that they enable.

As a student at the University of Chicago in the '60s, I entered my programs and data onto punched cards, then brought the deck of cards to an operator and waited for the batch job to be run and the output printed. Many a night, while waiting for the job to run, I went out with friends to Uno's or Due's, amazingly still serving deep dish pizza in Chicago's near North side, or to Taquerias de Mexico, a long gone little restaurant in what is now the Chicago Circle campus of the University of Illinois. When I got back several hours later, I was hoping that my job had finished, and that I had not made a silly error that caused the job not to run. Those were clearly slower -- if not necessarily kinder, gentler days.

I realize that to many, this sounds like someone talking about the days of horseless carriages and gaslight lamps. That is why I think of the true 1st generation of user interfaces as the text-based interfaces that we used with interactive computer terminals starting in the 1970s, and that you can still find in the DOS Command Prompt in Windows. Interactive computing ushered in all kinds of new applications, from transaction processing in mainframes to office systems in PCs.

Text-based interfaces gave way to Graphical User Interfaces or GUIs. This 2nd generation of user interfaces was invented at Xerox PARC in the late '70s, was popularized by the Apple Macintosh in the mid '80s, and has been part of every Windows system since the early '90s. Object oriented programming was particularly suited to GUIs, making it easier to manipulate programs (or objects) by pointing, clicking, dragging, dropping and so on. I don't need to go over the huge numbers of new applications ushered in by these 2nd generation graphical interfaces in all their various manifestations.

And, while we generally associate the success of the Internet with advances in networking and standards, we would be remiss not to mention the equally important role that the GUI-based Mosaic browser played in the explosion in popularity of the World Wide Web.

But, 25 years is a long time between user interface paradigms, when everything else in these intervening years has advanced at warp speeds. Visual interfaces, which first appeared with scientific applications, and have been increasingly perfected (along with the accompanying sounds) with digital animation and video games promise to finally usher in the next round of new and innovative applications. Video games are particularly important in this regard, because in addition to their very realistic visual images and great sound, they are also highly interactive and increasingly collaborative, and thus a good launch pad for thinking about how people should best interact with all kinds of computer applications as well as with each other in the future.

The timing for such a new round of applications with innovative user interfaces could not be better. First of all, both Microsoft and Sony are expected to come out with new game consoles, the new Xbox (http://en.wikipedia.org/wiki/Xbox) and PlayStation 3 respectively, that promise an order-of-magnitude improvement in performance for highly visual, interactive applications, especially when coupled with the much sharper images and sound of HDTV and surround sound. Efforts are already underway to adapt the underlying capabilities of the Xbox and PS3 for non-game applications, such as with the announcement we made a few weeks ago about providing key hardware and software specs of the PS3's Cell processor to open communities, so they can start developing new platforms and applications. The continuing performance and price advances in computer technologies -- not just micros, but memory, storage, bandwidth and everything else -- makes supporting these new kinds of visual applications increasingly affordable, much as advances in PC capabilities helped graphical interfaces take off more than ten years ago. Finally, as computing becomes ubiquitous and integrated into just about everything around us, there will be huge market pressure to make interacting with those devices as easy and natural as possible.

While these advancing technologies are the enablers of the move to 3G/UI, the real innovation will come from the many thousands of application developers -- and, increasingly, users themselves, from gamers, to medical technicians (and patients), to educators (and students) -- who will compete in re-thinking how to best integrate these new kinds of visual interfaces into existing applications, as well as come up with whole new kinds of applications that we cannot even envision today. Our brains are wired for sight and sound. It is about time for computers to finally be able to deal with us on our terms.

Comments

As a movie buff I'm sure you saw "Minority Report" by Steven Spielberg.

I really liked the user interface that Tom Cruise and others used when searching for clues to the future crimes foreseen by the pre-cogs. For those of you who haven't seen the movie, basically the user "conducted" a set of video images with his or her hands. It's hard to describe in writing so just rent "Minority Report" if you're haven't seen it.

Interestingly, while searching for photos of this movie interface (no luck), I found an article that says that Raytheon is creating a real life version for the US military (see link below). Wonders never cease!

A very interesting article. The company I work for does a lot of work with both larger enterprise customers in the PLM space as well as the game industry. With enterprise customers (The Boeings, Panasonics, etc of the world) we help them apply their existing 3D CAD data to training and configuration problems. In the game industry, we provide tools and technology for authoring 3D game user interfaces.

In my experience there is a remarkable overlap between the two markets that is not immediately apparent. The game developers are interested in displaying highly complex real-time information regarding the in-game simulation in an intuitive fashion. A poor interface can ruin a game. The enterprise needs are not so different. They require an equally capable method of displaying complex enterprise information and tying it into highly interactive 3D applications built upon 3D CAD data.

What surprised me was that these two completely different customer groups both share such remarkably overlapping feature requirements. When you have an inexpensive new technology solving a wide range of problems that were previously unsolvable in a cost effective manner, the futurist in me begins to pay attention.

A piece by Seth Schiesel in the Aug. 6 NY Times about "World of Warcraft" (http://www.nytimes.com/2005/08/06/sports/06game.html)is very resonant with the themes you're touching on in this post, Irving: new forms of user interface, and collaborative online activities. He writes:

"Most people never experience interdependence second by second. Even if your employer says you are working on something larger, many people spend all day in a cubicle or at a counter and never really know if they are making a difference. That's what 'Dilbert' is all about.

"But gamers get to know that their team killed the beast and got the treasure, much as athletes get to know that their team won the game and soldiers get to know that their side defeated the enemy. Those are powerful, even addictive emotions, and many people who experience them can never truly give them up."