The Past, Present, and Future of Touch

Welcome back to our three-part series on touchscreen technology. Last time, Florence Ion walked you through the technology's past, from the invention of the first touchscreens in the 1960s all the way up through the mid-2000s. During this period, different versions of the technology appeared in everything from PCs to early cell phones to personal digital assistants like Apple's Newton and the Palm Pilot. But all of these gadgets proved to be little more than a tease, a prelude to the main event. In this second part in our series, we'll be talking about touchscreens in the here-and-now.

When you think about touchscreens today, you probably think about smartphones and tablets, and for good reason. The 2007 introduction of the iPhone kicked off a transformation that turned a couple of niche products—smartphones and tablets—into billion-dollar industries. The current fierce competition from software like Android and Windows Phone (as well as hardware makers like Samsung and a host of others) means that new products are being introduced at a frantic pace.

The screens themselves are just one of the driving forces that makes these devices possible (and successful). Ever-smaller, ever-faster chips allow a phone to do things only a heavy-duty desktop could do just a decade or so ago, something we've discussed in detail elsewhere. The software that powers these devices is more important, though. Where older tablets and PDAs required a stylus or interaction with a cramped physical keyboard or trackball to use, mobile software has adapted to be better suited to humans' native pointing device—the larger, clumsier, but much more convenient finger.

The foundation: capacitive multitouch

Enlarge/ Many layers come together to form a delicious touchscreen sandwich.

Most successful touch devices in the last five or so years have had one thing in common: a capacitive touchscreen capable of detecting multiple inputs at once. In this way, interacting with a brand-new phone like Samsung's Galaxy S 4 is the same as interacting with the original 2007-model iPhone. The list of differences between the two is otherwise about as long as your arm, but the two are built upon that same foundation.

We discussed some early capacitive touchscreens in our last piece, but the modern capacitive touchscreen as used in your phone or tablet is a bit different in its construction. It is composed of several layers: on the top, you've got a layer of plastic or glass meant to cover up the rest of the assembly. This layer is normally made out of something thin and scratch-resistant, like Corning's Gorilla Glass, to help your phone survive a ride in your pocket with your keys and come out unscathed. Underneath this is a capacitive layer that conducts a very small amount of electricity, which is layered on top of another, thinner layer of glass. Underneath all of this is the LCD panel itself. When your finger, a natural electrical conductor, touches the screen, it interferes with the capacitive layer's electrical field. That data is passed to a controller chip that registers the location (and, often, pressure) of the touch and tells the operating system to respond accordingly.

This arrangement by itself can only accurately detect one touch point at a time—try to touch the screen in two different locations and the controller will interpret the location of the touch incorrectly or not at all. To register multiple distinct touch points, the capacitive layer needs to include two separate layers—one using "transmitter" electrodes and one using "receiver" electrodes. These lines of electrodes run perpendicular to each other and form a grid over the device's screen. When your finger touches the screen, it interferes with the electric signal between the transmitter and receiver electrodes.

When your finger, a conductor of electricity, touches the screen, it interferes with the electric field that the transmitter electrodes are sending to the receiver electrodes, which registers to the device as a "touch."

Because of the grid arrangement, the controller can accurately place more than one touch input at once—most phones and tablets today support between two and ten simultaneous points of contact at a time. The multitouch surfaces of the screens allow for more complex gestures like pinching to zoom or rotating an image. Navigating through a mobile operating system is something we take for granted now, but it isn't possible without the screen's ability to recognize multiple simultaneous touches.

These basic building blocks are still at the foundation of smartphones, tablets, and touch-enabled PCs now, but the technology has evolved and improved steadily since the first modern smartphones were introduced. Special screen coatings, sometimes called "oleophobic" (or, literally, afraid of oil), have been added to the top glass layer to help screens resist fingerprints and smudges. These even make the smudges that do blight your screen a bit easier to wipe off. Corning has released two new updates to its original Gorilla Glass concept that have made the glass layer thinner while increasing its scratch-resistance. Finally, "in-cell" technology has embedded the capacitive touch layer in the LCD itself, further reducing the overall thickness and complexity of the screens.

Using the coordinates from this grid of electrodes, the device can accurately detect the location of multiple touches at once.

None of these changes have been as fundamentally important as the original multipoint capacitive touchscreen, but they've enabled thinner, lighter phones with more room for batteries and other internal components.

94 Reader Comments

Interesting read Ars. I particularly liked how you covered the "Pavlov" effect that the widespread adaptation of the touch screen has had. (I constantly find myself resisting the urge to touch the screen of my work PC, ATMs, or other devices that don't have a touch screen built in.

Given that the baby boom generation is retiring and will make up the largest (and richest) demographic in the U.S. population, touchscreen UI computers are the stupidest idea to come out at this time.

The mouse + keyboard, heck just the keyboard is the ideal solution because the lowest common denominator is entering text and as long as we need to do that, then keyboards will rule. (ever try to dictate to a machine all day?)

Touchscreens are lousy at entering text, there is no tactile feedback so we can look at something else (like the screen) when we are typing, especially if our big mitts are in the way.

Touchscreen tablets have their place and purpose, apx 3/4 of their usage time is for playing games according to Business Insider, the touchscreen UI has no place on desktop type computers because frankly we are on them all day to get things done.

I just appreciate the well-written article on touchscreens... call me simple, but I don't particularly care how much he touched on specific product lines and their popularity. I just liked the article.

But I don't want to touch my computer screen.Touchscreens are lousy at entering text, there is no tactile feedback so we can look at something else (like the screen) when we are typing, especially if our big mitts are in the way.

«Another important element [...] scrolling that speeds up, slows down, and gains momentum based on how you're touching the screen. When you can't scroll anymore, the OS will usually move and snap back into place [...] (commonly called "rubber-banding")»

I find it amazing how you managed to squeeze in references to other OS in other paragraphs, and on this one, which refers to unique elements introduced by Apple,

...which is the combination of a feature common in media playback devices coupled with a superficial bit of animation. A lot of mechanisms translate quite directly from one form of input to another. So some of the hype around the new shiny shiny is a little overblown both in general and how it relates to a particular vendor.

And I still have UX designers that propose to show controls only on "mouse-over". After using an Android Transformer it became obvious that all OS' should allow and plan for touch screens. I can't tell you how many times I've been upset with my wife's netbook because I touch a link and the computer doesn't do anything.

Now I just can't wait for the manufactures to get rid of the trackpads on laptops and just go with the touchscreen.

But I don't want to touch my computer screen.Touchscreens are lousy at entering text, there is no tactile feedback so we can look at something else (like the screen) when we are typing, especially if our big mitts are in the way.

«Another important element [...] scrolling that speeds up, slows down, and gains momentum based on how you're touching the screen. When you can't scroll anymore, the OS will usually move and snap back into place [...] (commonly called "rubber-banding")»

I find it amazing how you managed to squeeze in references to other OS in other paragraphs, and on this one, which refers to unique elements introduced by Apple, apparently the acceleration and rubber banding interface elements materialized in the industry out of thin air.

Inertial scrolling has been around for over two decades. Rubber-banding was something the list controls on the original Surface (now PixelSense) tables did as well.

But I don't want to touch my computer screen.Touchscreens are lousy at entering text, there is no tactile feedback so we can look at something else (like the screen) when we are typing, especially if our big mitts are in the way.

While that is amazing, and can improve screen typing, nothing can replace a full size clickety-clack keyboard when it comes to an ideal typing experience. I think OldManGuy doesn't like Desktops and Laptops going touch because it's the "thing everyone's doing".

The 1st article seemed to avoid mentioning the iPhone/iPad, but I thought this one was balanced. It gives Apple credit where it's due, which is designing software specifically tailored to true touch, and shows that while others have tried before, there's was an all-in-one solution and therefor fell short. And in the case of Office 2013, is still falling short.

Tactile feedback is one thing, but these devices are still no more ergonomic than when they were invented. These flat screens and tiny keyboards and miniature mice are destroying our hands and wrists. Even the standard office keyboard still isn't the shape it really should be.

Given that the baby boom generation is retiring and will make up the largest (and richest) demographic in the U.S. population, touchscreen UI computers are the stupidest idea to come out at this time.

The mouse + keyboard, heck just the keyboard is the ideal solution because the lowest common denominator is entering text and as long as we need to do that, then keyboards will rule. (ever try to dictate to a machine all day?)

Actually, given these factors, a touchscreen actually makes a fair amount of sense.

The main thing is that the mouse and keyboard aren't going away with touchscreen desktops and laptops, they're all different inputs for different uses. You're not going to type on an upright screen (hopefully not ever, that would destroy wrists), but lets say you're watching a movie and an alert pops up in the corner (ignoring that it would be horrible to have that happen), do you remember the keyboard shortcut, do you reach over, find the cursor on the screen by wiggling it about then click on it, or simply tap it with your finger?

And while the keyboard is still the quickest way to use a computer, how many people do you see click through the start menu? The keyboard being the best way doesn't mean it's the most common way nor the most liked. With computers just beginning to spread to many parts of the world (and phones being the way they're spreading in many regions) a touchscreen is going to be the easiest to learn and interact with for many people. Touch interfaces are seen as the easiest to many people.

Plus with an aging population, the cursor makes less sense, especially for those visually impaired. It's a tiny thing they need to track, rather than giving them big elements they can interact with. I've seen way too many people wiggling the mouse around trying to find the cursor when simply being able to tap where they want it would save time and effort to think it's a good idea.

And in regards to all the Apple posts above, this isn't politics guys, it's informing people, no one is trying to convert anyone to a specific OS, there's no winner out of this. It's about learning about touchscreens and their many implementations. Sure, Apple is all the rage in the area and contributed a lot but learning just about Apple would be less informative than learning about every implementation, thus it's scattered pretty evenly to show all the different ways it's used. And on rubber-banding, Apple had their patent nulled by the patent office a few months ago, so I'd drop that one as an argument.

All jokes aside, I do recall testing and using the XP Tablet PC Edition on some (at the time) 'amazing' Fujitsu hardware. Putting aside the heat issues, pathetic processors, narrow keyboards, and lack of sufficient RAM, trying to use Windows XP with a stylus was a pain. Sure, it had some extra bells and whistles, but it just didn't work well.

With so many attempts to release a tablet PC that ran Windows, I wonder why hardware manufacturers didn't put more pressure on M$ to release a proper tablet OS. XP was released in 2001, and Tablet PC Edition in 2002.

Interesting read Ars. I particularly liked how you covered the "Pavlov" effect that the widespread adaptation of the touch screen has had. (I constantly find myself resisting the urge to touch the screen of my work PC, ATMs, or other devices that don't have a touch screen built in.

Cripes, I tried to two-finger zoom a fuzzy picture in a newspaper a couple weeks ago. Absolutely true story. I was very tired.

Tactile feedback is one thing, but these devices are still no more ergonomic than when they were invented. These flat screens and tiny keyboards and miniature mice are destroying our hands and wrists. Even the standard office keyboard still isn't the shape it really should be.

There have been a number of ergonomic keyboards and mice introduced, buy, just like more efficient letter arrangements, haven't caught on.

Given that the baby boom generation is retiring and will make up the largest (and richest) demographic in the U.S. population, touchscreen UI computers are the stupidest idea to come out at this time.

The mouse + keyboard, heck just the keyboard is the ideal solution because the lowest common denominator is entering text and as long as we need to do that, then keyboards will rule. (ever try to dictate to a machine all day?)

Touchscreens are lousy at entering text, there is no tactile feedback so we can look at something else (like the screen) when we are typing, especially if our big mitts are in the way.

Touchscreen tablets have their place and purpose, apx 3/4 of their usage time is for playing games according to Business Insider, the touchscreen UI has no place on desktop type computers because frankly we are on them all day to get things done.

<luddite>Actually, I don't like touchscreens for playing games. I use dozens of binds & macros for most MMOs, and when I was playing RTSs competitively, there's no way you can compete w/the speed of a kb + mouse pairing. Hell, just the first 10 sec. of a high level RTS game can determine the outcome, and you need to be able to hit about 20~30 shortcuts & micro commands in that span. Good luck doing that on a touchscreen.

And that's really the failing of the touchscreen interface for, ugh, "power users," and I hate that phrase, but it more or less describes people who look for shortcuts and efficient I/O techniques. I use a ton of kb shortcuts every day on my work station, w/a full sized keyboard, to the point that I don't even like working on a laptop. Actually, I'm working on one right now... w/a full keyboard plugged in. Can't live w/o my numpad, Win button, function keys, etc, and better still are specialized gaming kbs w/user-defined macros and the like.

Touchscreens are great on small devices, since it's impractical to have a full I/O device attached to, say your phone. Most tablets are sort of in-between, but I prefer my tabs w/removable kbs anyway, ala the Asus Transformer line. Due to their form factor & interface, touchscreen-optimized apps & games are all the rage now, but I honestly don't enjoy any mobile-sized game. There are a few, like Baldur's Gate, that have been modified for touchscreens that I can get into, but I still miss kb shortcuts. As for productivity, I don't even try to create content on my phone or tablet. I am planning on getting a Win 8 based tablet soon, but if I do any real work on them, it'll be attached to a usb kb.</luddite>

I loathe touchscreens. I hate that every last smarthphone HAS to be touchscreen based, especially since I remember seeing an early version of android on a non-touch device and thought it was pretty cool.

I liked my Palm PDAs (everything from a handspring visor all the way to a Tungsten T5) because the stylus kept my screen clean, was an accurate pointing device and didn't obstruct my view of my screen. I lamented how fragile the screens were, even then, though. And it was that fragility that doomed my last PDA, I was cleaning the screen by rubbing it with my shirt, looped over my thumb. When i finished cleaning the dust off, the screen no longer took input from anything. But, until that point, I was quite pleased with the device, especially because it had an aluminum clamshell case that completely enclosed the device in my pocket. That clamshell had several horrific dents in it, each one a case where having the shell had saved my PDA

But there's nothing like that for the smartphones I buy. and my fingers are a poor poor substitute for a stylus. My fingers are far larger, far less precise. they, and the hands they are attached to obscure my view of the screen. They tend to get dirty. Nobody wants heatsink grease, or engine grime, or that evil gunk that builds up in a smoking client's computer, or barbeque sauce, or any of the things my hands encounter in a day, on a touchscreen. I also think its horrible design to have the screen face my face on a smartphone. we should be talking into the back of the phone, not the front.

I've got a motorola photon q, I insist on having a physical keyboard, but because I do, there are no durable cases for it. I would fight a hobo bare-fisted for a chance to have a metal clamshell that enclosed my phone when not in use. And these touchscreens are ridiculously fragile. Just this past sunday, I had my phone in my pocket as I drove across the state. When I got home, despite having not moved from where I sat, despite doing NOTHING with the phone, I found the screen cracked and unresponsive. I can only guess that an earlier drop the week prior weakened it, and some pressure from my leg made it crack. I don't know, its baffling.

Actually it is possible to provide tactile feedback using microfluidic touchscreens

That looks very interesting.

However the sad fact of the matter is hardware makers (especially Apple) will gravitate towards producing less expensive hardware with less complicated (and non-moving) components.

Once there, they are unlikely to add something to satisfy only a small portion of users, we learned all about that with the glossy/matte screen issue.

After all Apple has that angle covered with the iPad keyboard/dock connector to satisfy those users.

As much as the tech tries, it still comes back to a real desktop hardware UI features because the human doesn't change.

Well, civilization existed for thousands of years without keyboards. But, we were using tablets and other writing surfaces for most of the span of civilization.

It comes down to preference and context.

Typing is a learned process--touch is natural.

But we've always used tools to write, and those methods are learned processes. Very few things in modern society are truly natural. Typing solves the problem of illegible handwriting and it can be done MUCH FASTER than with pen ... and pen is much faster than with Chisel.

Technological evolution is about give and take. In order to gain some things, like speed you may lose longevity. Or mobility may lose ergonomics. It's not a constatantly upward climb to success, there are a lot of examples where touch is being forced into places where it's not ideal and that's why people get defensive about it.

Of course, if we were living in an ideal world we'd all be typing on DVORAK keyboards, but we're mostly using QWERTY ... that doesn't mean DVORAK isn't available. The choice is out there, but sometimes there's a cost to going against the grain.

Well, civilization existed for thousands of years without keyboards. But, we were using tablets and other writing surfaces for most of the span of civilization.

It comes down to preference and context.

Typing is a learned process--touch is natural.

Touch is indeed natural. So is fingerpainting. It takes no effort to progress beyond that to painting with a brush.

Learning to touch-type is somewhat unintuitive. It takes practice & effort to get good at it. Some never do.

Those who make the effort to improve their input abilities, whether it's oil painting or typing, will reap the benefits of efficiency and optimization. Those who don't will pressure companies to continue to dumb down devices, to make I/O more "ergonomic" and "intuitive". I suspect this trend will make the more "difficult" I/O options increasingly in the minority, but I doubt they'll ever go away, so at least a few of us will continue to kick your butts at online games of your choice.

Unless I'm mistaken, I haven't seen any reference to how Leap Motion (or the likes of) technology will affect the current landscape of touch interface, and interface in general. I can imagine this technology built into future techs fairly simply and seamlessly, and almost as important- cheaply. This deserves a mention, as it not only alleviates several issues with touch, but will also allow displays to ditch the touch substrate (correct me if I'm wrong, people) and have even more thin, responsive, and immersive screens. Plus, no finger prints and germs. Touchscreens at the WaWa make anyone else cringe and wash hands before eating??

Given that the baby boom generation is retiring and will make up the largest (and richest) demographic in the U.S. population, touchscreen UI computers are the stupidest idea to come out at this time.

The mouse + keyboard, heck just the keyboard is the ideal solution because the lowest common denominator is entering text and as long as we need to do that, then keyboards will rule. (ever try to dictate to a machine all day?)

Touchscreens are lousy at entering text, there is no tactile feedback so we can look at something else (like the screen) when we are typing, especially if our big mitts are in the way.

Touchscreen tablets have their place and purpose, apx 3/4 of their usage time is for playing games according to Business Insider, the touchscreen UI has no place on desktop type computers because frankly we are on them all day to get things done.

Dear God, please bring back the Xerox PARC guys.

Who exactly are you speaking for here? My parents have almost completely abandoned their desktop PC in favor of touch screen devices (iPad, iPhone, and Android tablet). My fiance's skeptic father has become an evangelizer for iPad in the way it's revolutionized his computing needs. Their best friends independently returned a desktop PC that was "too complicated" in favor of an iPad and keyboard.

All of these people are of the Baby Boomer generation, and that's just scratching the surface of my own personal experience spread out from friends and family over 3 states, certainly not accounting for the country at large. Something tells me there's a similar trend going on.

Tactile feedback is one thing, but these devices are still no more ergonomic than when they were invented. These flat screens and tiny keyboards and miniature mice are destroying our hands and wrists. Even the standard office keyboard still isn't the shape it really should be.

Ditto. I use computers every day as part of my job (and hobbies), and try to be conscious of ergonomics so as to avoid RMI. Yet since I got a smartphone, I've developed a ganglion cyst in my right hand wrist.

With the proliferation of handheld computing devices that are demanding more and more of our attention, the risks of developing RMIs have only gotten larger.

Given that the baby boom generation is retiring and will make up the largest (and richest) demographic in the U.S. population, touchscreen UI computers are the stupidest idea to come out at this time.

The mouse + keyboard, heck just the keyboard is the ideal solution because the lowest common denominator is entering text and as long as we need to do that, then keyboards will rule. (ever try to dictate to a machine all day?)

Touchscreens are lousy at entering text, there is no tactile feedback so we can look at something else (like the screen) when we are typing, especially if our big mitts are in the way.

Touchscreen tablets have their place and purpose, apx 3/4 of their usage time is for playing games according to Business Insider, the touchscreen UI has no place on desktop type computers because frankly we are on them all day to get things done.

Dear God, please bring back the Xerox PARC guys.

Check out an Atom-powered Windows 8 tablet and you'll change your mind about touch. A real computer but weighs in at just about 1.25 pounds, so you'll hold it in one hand and use touch with the other. I agree about tactile feedback but audio feedback is just as good.

I'm not saying it replaces a desktop PC, and I'm much happier to be typing this on a physical keyboard than I would the on-screen one. But instead of having a 4-pound laptop with a bag of accessories ending up at 30 pounds, I've got a little, light-weight tablet that lets me leave the power brick behind (or in checked baggage). I can add a portable bluetooth keyboard and still not be up to the weight of the laptop alone.

Touchscreen tablet PC hybrid devices are a game changer. I've been saying this for months and still hold to it.

I got mine back around the beginning of November (activation says 11/1/2012) and use it daily (and heavily) for work and recreation. I rarely ever pull out my old Thinkpad anymore. And my desktop at home is still primarily a gaming machine.

Given that the baby boom generation is retiring and will make up the largest (and richest) demographic in the U.S. population, touchscreen UI computers are the stupidest idea to come out at this time.

The mouse + keyboard, heck just the keyboard is the ideal solution because the lowest common denominator is entering text and as long as we need to do that, then keyboards will rule. (ever try to dictate to a machine all day?)

Touchscreens are lousy at entering text, there is no tactile feedback so we can look at something else (like the screen) when we are typing, especially if our big mitts are in the way.

Touchscreen tablets have their place and purpose, apx 3/4 of their usage time is for playing games according to Business Insider, the touchscreen UI has no place on desktop type computers because frankly we are on them all day to get things done.

Dear God, please bring back the Xerox PARC guys.

Who exactly are you speaking for here? My parents have almost completely abandoned their desktop PC in favor of touch screen devices (iPad, iPhone, and Android tablet). My fiance's skeptic father has become an evangelizer for iPad in the way it's revolutionized his computing needs. Their best friends independently returned a desktop PC that was "too complicated" in favor of an iPad and keyboard.

All of these people are of the Baby Boomer generation, and that's just scratching the surface of my own personal experience spread out from friends and family over 3 states, certainly not accounting for the country at large. Something tells me there's a similar trend going on.

But you have to love the "I'm getting too old to lift my arms" part. I wonder if OldMacGuy's Mac feeds him?

Personally I think that Apple could make an interesting combo tablet/ultrabook device like others have on the market only... more Apple. Will they? Who knows?

Interestingly I am waiting to see what the new Mac Pro looks like later this year.

Well-researched article on touch, however two years from now it's all going to be gesture.

Gesture-based televisions are debuting, and that will change the way we interact with screens (as well as reducing germ transmission from dirty screens).

Left-right scrolling will dominate, as it is easier to move your hand right and left in relation to a vertical screen.

Voice is already working - think Siri - but it has a long way to grow before it becomes useful. Gesture is already working in video games, imagined in movies, and currently being manufactured on the assembly line.

Given that the baby boom generation is retiring and will make up the largest (and richest) demographic in the U.S. population, touchscreen UI computers are the stupidest idea to come out at this time.

The mouse + keyboard, heck just the keyboard is the ideal solution because the lowest common denominator is entering text and as long as we need to do that, then keyboards will rule. (ever try to dictate to a machine all day?)

Touchscreens are lousy at entering text, there is no tactile feedback so we can look at something else (like the screen) when we are typing, especially if our big mitts are in the way.

Touchscreen tablets have their place and purpose, apx 3/4 of their usage time is for playing games according to Business Insider, the touchscreen UI has no place on desktop type computers because frankly we are on them all day to get things done.

Dear God, please bring back the Xerox PARC guys.

Who exactly are you speaking for here? My parents have almost completely abandoned their desktop PC in favor of touch screen devices (iPad, iPhone, and Android tablet). My fiance's skeptic father has become an evangelizer for iPad in the way it's revolutionized his computing needs. Their best friends independently returned a desktop PC that was "too complicated" in favor of an iPad and keyboard.

All of these people are of the Baby Boomer generation, and that's just scratching the surface of my own personal experience spread out from friends and family over 3 states, certainly not accounting for the country at large. Something tells me there's a similar trend going on.

I think he's speaking for himself. But Baby Boomers are "generally" computer skeptical, and Tablets are kind of like a wolf in sheep's clothing for them. I'm sure OldMacGuy loves his computers and has stories to tell his grandchildren about each of them. When he was YoungMacGuy or MiddleAgedMacGuy he would have been seen as an enthusiast, nerd, and part of the fringe. At the time your parents may have thought of that "computer thing" as some sort of trend.

Andrew Cunningham / Andrew has a B.A. in Classics from Kenyon College and has over five years of experience in IT. His work has appeared on Charge Shot!!! and AnandTech, and he records a weekly book podcast called Overdue.