Interview with 2011 SIGGRAPH Asia Featured Speaker, Bill Buxton

If you have experienced Microsoft, Maya and/or SketchbookPro … you have (knowingly or unknowingly) come in contact with Bill Buxton. Before joining Microsoft, as a Principal Researcher at Microsoft Research, Bill's journey took him down paths that were filled with a variety of jobs, degrees, and adventures … including the title of Chief Scientist at Alias|Wavefront, where he was instrumental in contributing to the development and success of Maya and SketchbookPro.

Known as a man who wears a variety of well-worn, yet extremely well-fitting hats ... what Bill lacks in ego, he makes up for in brilliance (he holds degrees in both Music and Computer Science). Although he is the quintessential professional in all areas of his life, he never takes himself too seriously.

His innovative thinking in the arena of Computer Graphics and Interactive Techniques earned him an invitation to the 2011 SIGGRAPH Asia Conference, in Hong Kong, as a Featured Speaker.

I invite you to explore the world of Bill Buxton: award-winning computer innovator, progressive musician, author, scientist, wilderness explorer, advocate, and avid collector of interactive devices.

Dee Marie: Regrettably, many of our readers were unable to attend the 2011 SIGGRAPH Asia Conference, in Hong Kong. Please give us a brief summary of your SIGGRAPH presentation, "More than what the Eye Sees: Interaction and Graphics?"

Bill Buxton: My talk went to SIGGRAPH’s roots. While many, if not most, think of SIGGRAPH as the ACM’s Special Interest Group for Graphics, its correct name is: the ACM Special Interest Group on Computer Graphics and Interactive Techniques. And, its stated mission is, "…to promote the generation and dissemination of information on computer graphics and interactive techniques."

This combination of graphics and interactive techniques is also reflected in all editions of the two most important text books in the field, Newman & Sproull’s, Principles of Interactive Computer Graphics, and Foley & van Dam’s Fundamentals of Interactive Computer Graphics.

This combination of graphics and interactive techniques was a fundamental component in making the field so strong, and SIGGRAPH so important and exciting. Yet, over the years — especially since about 1981 — interactive techniques have played a significantly smaller role at SIGGRAPH. My feeling is that this is more than unfortunate.

My thesis is that over the past few years there has been a significant change in the nature of computer graphics. This is reflected in where we see CG manifest, in what kind of situation, and who it involves. At the heart of this is a strong shift in balance from CG that is to be watched (as in feature films), to CG that is to be interacted with in all kinds of rich contexts on a wide variety of devices.

Hence, my thesis is that interactive techniques are not just an important part of our legacy, they are fundamental to the health of our field in the future. If we cannot bring interaction back into the fold, in a balanced way, the relevance of SIGGRAPH will be seriously threatened. I think that would be a bad thing.

Dee Marie: From speculating about the future to reflecting on the past … in all the years that you've attended SIGGRAPH, what has been your most memorable moment?

Bill Buxton: There are two sides to my answer. First, what stunned me the most, was seeing things like the first ray-traced images of Turner Whitted; Ed Emshwiller's 1979 film, Sunstone; and Loren Carpenter’s fractal landscapes in his 1980 film, Vol Libre. At this point you may be wondering, “What about what you just said about the importance of interaction?”

Dee Marie: Well, now that you asked …

Bill Buxton: … remember, my specialty is interaction, so I knew better what to expect. But the real response to such a question is contained within part two of your original question. The second thing that has stunned me, even more than the first, is that all of the techniques that we saw in ray tracing, Sunstone, and Vol Libre, are now in the domain of interactive systems — something that I suspect none of us imagined at the time.

This feeds into something that was central to my talk in Hong Kong — the need to not only know Moore’s Law, but to embrace it (no matter how farfetched the consequences seem at the time), in every decision that you make.

Dee Marie: Moore's Law, states: that the number of transistors that can fit on a chip will double every 18 months. How does Moore's Law apply to CG interaction?

Bill Buxton: For example, our inability to internalize something as basic as the fact that the algorithms that gave us ray tracing, fractals, etc. would run in real-time of inexpensive hand-held devices, at photo-realistic resolution reflects our collective lack of belief in Moore’s Law and its consequences.

The rule is this: Every graphics technique you develop today — no matter how computationally intensive — will be interactive tomorrow. Consequently, you need to take that into account now, rather than after-the-fact, down the road.

Dee Marie: Along that line of thought; an influx of “touch devices” has recently flooded the marketplace, yet many would be surprised that interacting with computers, by means of touch, has been around for nearly half a century. What major advancements (either technical or social) do you attribute to this old technology being re-purposed at this specific time in history?

Bill Buxton: Yes, touch screens were being used in air traffic control systems in the UK shortly after 1965; and on desk-tops in kindergarten [through] high school classrooms in Illinois as early as the early 1970s. By 1984, Casio was making wrist watches with capacitive touch screens that were coupled with character recognition, thereby enabling you to enter data into your watch-based address book, or operate your calculator, by printing characters on the watch crystal with your finger (all for under $100!).

By 1985, there were already multi-touch devices around in the research community, including my lab, and a great capacitive multi-touch screen at Bell Labs. And, already by 1993, IBM had created the world’s first so-called “smart phone” that had only two buttons, volume and on/off, with everything else (including address book, calendar, email maps, notebook, etc.) activated by touching icons via the touch screen that covered the display that constituted the front of the phone, and then doing everything by touch.

I go through all of that in detail not to take a walk down memory lane — although my memories might constitute many of your reader’s “new news,” so to speak. Rather, I mention these things to point out that what we are seeing today is less a re-purposing of those technologies, but rather, already demonstrated applications crossing a tipping point, and becoming main-stream. Their doing so is very much a manifestation of my notion of Long Nose of Innovation. That is, it takes on average 20 years for a technology to make the transition from first articulation/demonstration to maturity (defined as becoming a $1billion industry). Despite everyone who saw it knowing that it was the future … the mouse, for example, took 30 years.

This all reflects the reality that technology or interactive techniques … rarely a successful product or business make. Rather, products and services that really take off generally require a whole ecosystem to be in place in order to create the conditions for the “perfect storm” that enables the “break through brand new hit” to happen. It has always been thus, and despite appearances to the contrary, things are moving no faster today than they were 50 years ago. The perception to the contrary is confusion between volume of product and technologies versus speed of change.

Dee Marie: Do you feel that the modern generation takes “touch devices” and “interacting with Artificial Intelligence” for granted? On a similar subject … what is your opinion in regards to the pros and cons of society being overly-dependent upon computerized technology?

Bill Buxton: In response to your second question on the pros and cons of society being overly-dependent upon computerized technology, my thoughts are largely shaped by the historian of technology, Melvin Kranzberg, whose first law states, “Technology is not good, it is not bad, but nor is it neutral.” If you introduce a technology as simple as a paper clip into an office, you will be changing that office’s culture in some combination of a negative and positive way. And, when it is something like the kind of digital technologies that we are making today, rather than a paperclip, that is being introduced, we must act in a way that assumes that the cultural impact will be significantly larger — and again, some combination of good and bad.

I believe that it is incumbent on those who design, make, sell and adopt such technologies to not only know Kranzberg’s first law, but to take it into account in making decisions. It is simply not acceptable to take the attitude that, “I just make the tools. I can’t be responsible for how they are used.”

The reality is, the design can have a huge impact on channeling usage along certain paths. Those of us who design such things need to make best efforts to make sure that those paths conform to our ethical compass. Of course, that implies that we have to have some sense of what our values are.

As for the first part of your question, about the prevalence of touch in today’s devices, I believe that that is largely a reflection of a lack of design, a lack of imagination, a lack of understanding, and a sheep-like mentality that reflects the too broad lack of innovative spirit in our field.

Dee Marie: I agree with your philosophy, however, I'm rather surprised at your response, can you elaborate?

Bill Buxton: Strong words, so let’s go a bit deeper. Yes, in 2007, products like Apple’s iPhone and Microsoft Surface raised a public awareness of multi-touch. Both reflect the best of innovation.

But the ensuing stampede transformed such well-considered decisions into a mere marketing-driven “must-have” feature that needed to be checked off for almost every product, regardless of whether it was appropriate or not, or brought any value whatsoever.

I am a designer. I care about design. Stripped of everything else, design is choice. But as Gloria Steinem has said, "... if everybody does it, it's not a choice." By the same token, it is not worthy of the name “design.”

Everything is best for something and worst for something else. The competent product designer and manager know the who/what/when/where/why/… of this and take it into account in the decisions that they make. There is a great and appropriate place for multi-touch in our world. Front console mounted car navigation and entertainment systems, for example, are not one of them.

Our field is driven by a chase for the next hot thing, the next fad — typically some new technology. What we need to wrap our mind around is the fact that the hottest, coolest, most amazing thing around is the people who use our tools. One of the reasons that I am so passionate about the importance of SIGGRAPH, SIGCHI and research, is to strengthen how we counter the natural, but nevertheless counter-productive — obsession with technology over that with people — that dominates our industry.

Dee Marie: As you previously stated … innovation in our industry is almost always characterized by a long nose – with 20 plus years being the norm. In the year 2020, what do you predict will be the top two technological innovations that were originally planted in 2000?

Bill Buxton: I hope that by 2020 we may finally be able to walk into a room and not have the planned presentation delayed by 10 minutes while we figure out how to connect to the wall mounted displays. But I won’t bet on it [pause for laughter].

Of course I am largely joking— but only partially. The most revolutionary thing that we can do over the next 20 years is just make this stuff work, work together, and do so seamlessly. We have crossed a line where the key challenge is not how to make a great product, but to make products that work together— at the very minimum — with the same fluency and seamlessness as my mobile phone does with the SYNC system in my car. And, I wouldn’t mind if having slates, phones, and laptops connecting to meeting room displays were high on the list of where to start.

Dee Marie: Your answer touched on one of my pet-peeves as well. Switching gears … I would be remiss if I didn’t ask you about your impressive collection of input and interactive devices. What first triggered you to start this collection?

Bill Buxton: Like many things, there is not a single, or a simple, answer to this. Don’t confuse my plea for focusing on people rather than just technology, with a denial that technology is fascinating.

Dee Marie: Have no fear, the fact that you are focused on people, only emphasizes your passion for technology.

Bill Buxton: I love the creativity and ingenuity in these devices, and that is partially why I have collected them over the past 30 years. But that is kind of a side benefit. The primary reason for acquiring them, and keeping them, is that there is an important message relevant to the design of future things embedded in every single thing in the collection. And, the lessons are not just in individual devices (which is why I bristle a bit when the collection is characterized as being of my “gadgets”), but in the relationship of one to another.

The collection is full of insightful stories — stories about design, the nature of innovation, human motor control, how richly varied the answers to seeming simple questions can be, and how things that seem impossible today were already shipping 20 years ago. The collection is a tangible trove of such stories, lessons and reality checks.

When I draw a blank, I spend some time in the collection, and frequently find an answer that I didn’t know I already knew. And, when I want to explain an idea to someone who would otherwise be skeptical, I just pull out some old device that manifests the concept so that they can experience it directly, and form an informed opinion, rather than make a decision based on some ill-founded prejudice or opinion. And, by the way, sometimes, that prejudiced and ill-founded opinion is mine, and the collection serves to keep me honest, and mend my ways.

Finally, the collection is part of our history — a history that is very much at risk of being lost. This concerns me, since ours is one of the only design or creative disciplines where practitioners are not deeply based in the history of the craft. This is to our serious detriment — in my opinion. Rather than whine, I figured that making the resources of the collection available was a way to do something constructive in terms of changing that.

Dee Marie: Ok, we've touched on the serious side of things, time to lighten it up a bit. Share with us something about yourself that even your closest friends don’t know.

Bill Buxton: This is a hard question to answer, since it presumes that I know not only myself, but also my closest friends, well enough to give a coherent answer. Perhaps that is my answer — that someone who seems to have opinions about almost everything, knows so little about that with which he is most intimate: himself.

Dee Marie: Nice answer. Let's try another … do you do your best work in silence, or surrounded by chaos?

Bill Buxton: To answer your question, in a word, “Yes.”

Dee Marie: Come on Bill, you know the ground rules of interviews, no one-word answers.

Bill Buxton: I’m not being cute. I know that I am far smarter and far more creative when I am working in a small group of people who I respect, trust, and who are different than me. I love being challenged, and I would prefer to be wrong and learn something from others, than to have my view taken as “right.” And I love these kinds of social interactions to occur in a loosely-structured, quasi-chaotic jazz kind of way. And, what I have described is probably the most publicly visible side of my work style.

However, for the above to work, I need huge amounts of private, quiet time, to both absorb and develop what happens in such public forums, and to think, read, study, and write. I have a terrible time concentrating, but once in a zone, I can go hard. I consciously shape my working life and environments to accommodate both needs. Each feeds the other.

Dee Marie: Now that we are on a roll … who has been the most influential person in your life?

Bill Buxton: I worry about naming names, since I will forget someone. There is no single person. There is a group. What I can say is that throughout my career — and I mean right back to as early as when I was five years old — there were people who — for some reason — took sufficient interest in me to not only share their passions with me, but to coach me, and point me in directions that I didn’t know existed, such that would kindle my own passions.

It just seems that despite an often large smokescreen behind which I hid — these people recognized something in me, and took a chance, often at times when things could have gone a very different way. This has happened repeatedly, and still happens, and the more it has happened, the less I took it for granted … just the opposite. And, one of the main consequences is that if I took it in any way, it was as an obligation on my part, a responsibility, to do the same in return as best I can.

Dee Marie: Getting back to SIGGRAPH … over the years, you've been involved in numerous conferences. What differentiated the 2011 SIGGRAPH Asia conference from past events you've attended?

Bill Buxton: What turned out most interesting was less any single thing … and more an unexpected thread … that formed by the sequence of talks by, Don Greenberg and Ken Perlin, in combination with my own. We didn’t discuss anything amongst ourselves in advance — despite all knowing each other. And yet, we all seemed to land on a similar theme — each approaching it from a different angle — a different and complementary perspective.

This surprised me as much as it interested me. One result is that I came away with a much deeper understanding of my own talk. And, I must say, I love that: due to the opportunities provided by the context that SIGGRAPH Asia afforded, me, one of the keynote speakers, may have been one of the ones who learned the most from my own talk.

But see? This reflects right back on what I said earlier about learning from others. When I prepared it, my talk was something that I crafted in silence, on my own (okay — alone as well as some consultation with friends like Andy van Dam, for example). But in the context of the conference, it morphed into something different — no longer “just” a talk, but rather part of a conversation and theme. I like such surprises.

Dee Marie: In regards to serendipitous moments … I was intrigued to discover that you were not only an award-winning computer innovator, but also a musician, author, scientist, wilderness explorer, advocate, and avid collector. You must be extremely proud of your accomplishments.

Bill Buxton: I’m not sure how pride fits into the equation.

Dee Marie: Let me rephrase that … do you agree that during you lifetime you have racked-up a very impressive eclectic resume of achievements?

Bill Buxton: Yes, I’ve done a lot of different things. I have a lot of different interests. Discovering new passions is a pattern in my life, and one that keeps me alive, motivated, and helps me avoid growing stale. But I am also almost 63 years old. So the number of different things is partially (even largely) a consequence of the amount of time that I have had to do them.

Dee Marie: Were your most valued achievements the ones that you were most passionate about?

Bill Buxton: Yes, most of these things were passions. I just don’t see how anyone can be motivated enough to be good at anything if they didn’t love it. So, I have loved these things, but I’m not sure how much pride factors into these things. I do not want to come across as being falsely modest. But the truth is that I constantly struggle with the thought that I have not used my time well, and that I have failed to accomplish what I could have and should have. And, I measure that more from the perspective of what I still want to do, but have not yet done, than from that of past “accomplishments” (some of which are as bad as others are good).

Dee Marie: By definition, a Renaissance Man, has broad intellectual interests; is accomplished in areas of both the arts and the sciences; and most importantly, he has the ability to "look" into the past as well as the future … all at the same time. Although I suspect that you will disagree … to me, you are theepitome of a Modern Renaissance Man.

Bill Buxton: When I was a kid, a key instruction from my father was, “Make a difference.” When I joined Microsoft Research, Rick Rashid (who hired me) gave me my job description using the same three words. But as for being a “Renaissance Man,” no, I don’t buy the description. That directs far too much credit onto the individual … me.

This brings us right back to the earlier conversation that touched on working methods, and collaboration. There is almost nothing that I have done that did not involve the very active help, participation and encouragement of others. I believe in “Renaissance Teams” but not modern “Renaissance Man or Woman.”

If I appear accomplished, I would say that my two key accomplishments are: first, generally having excellent judgment in identifying people to work with; and second, having outstanding creativity in terms of finding ways to convince them to do so. Ever the musician, it is all about the band you play with!

Dee Marie: Thank you for those parting words of inspiration, and thank you also for taking time out of your busy schedule to allow us a glimpse into your world.