Apple looking at dual-mode touchscreen desktops and laptops

Apple has considered how touchscreen-equipped Macs could switch from …

With Mac OS X and iOS both sharing a lot of underlying code, and touchscreen interfaces becoming more popular, there is certainly reason to believe that Apple could leverage an iOS-like touchscreen interface for future computers. Patently Apple recently discovered a patent application filed with the World Intellectual Property Organization earlier this year, which reveals that Apple has considered how both desktop and laptop computers could switch from a more traditional desktop UI best suited for use with a keyboard and mouse to a UI geared for touch input. Such machines could use a sort of hybrid between Mac OS X and iOS, switching UI layers for the most appropriate context.

When we spoke to a cadre of Mac OS X developers earlier this year, they universally agreed that iOS was having more influence on the direction of Mac OS X than the other way around. Cabel Sasser, co-founder and developer at Panic, especially felt that Mac OS X 10.7 could potentially be a hybrid between the two OSes. "I could see a gradual, slow merger between iOS and Mac OS X styles and approaches," he told Ars. "It doesn't make sense for them to be developing two of everything, one good, one not as good—two calendars, two address books—it's got to merge somehow."

Ars' John Siracusa, in a recent editorial for Macworld, expressed his hope that Mac OS X 10.7 could power new touchscreen-based Macs. However, he wrote, the standard WIMP interface just isn't going to cut it for touchscreen use:

Unfortunately, the Mac user interface is not designed for touch. Standard Mac OS X controls like scrollbars, buttons, and checkboxes are far too small to be comfortably manipulated with an adult-size finger. Worse, some common operations—such as hovering a cursor over an interface element without actually clicking it—can’t be done at all using touch alone. On the hardware side, poking at the vertical screen surface of, say, a future touch-sensitive iMac would quickly produce arm fatigue.

Like Sasser, Siracusa also believes that a future version of Mac OS X could switch UI paradigms depending on user needs and the configuration of a hypothetical transforming MacBook/iPad device. Such a device would be "a light, fully capable Mac laptop when you need all the power it provides, and a slightly bulky (but screamingly fast) iPad when you don't."

Apparently Apple has considered giving users exactly that. The WIPO patent application, titled "Transitioning Between Modes of Input," describes how a iMac-like computer could be tilted from a vertical to more horizontal position. When a suitable change in screen angle is detected, the UI could shift into a mode geared toward touch input. Switching the display back would trigger a switch to mouse and keyboard input.

The patent also considers laptops with similar functionality. When the display is reoriented to lay flat—essentially converting a laptop into a flat, touchscreen tablet—the OS would switch to a touch-based UI. When the display is switched back, you would again be able to use traditional keyboard and trackpad input.

Though the patent doesn't describe particularly how application software would handle the context switch, one possibility is that applications could be bundled with both a "regular" UI and a "touch" UI, with shared underlying code to handle program logic and data. The application could then present the appropriate UI depending on the configuration of the machine. Additionally, apps built specifically for iOS devices could easily run in a compatibility layer, just as iPhone and iPad apps now run on Mac OS X machines in a simulator used during development.

90 Reader Comments

I can see having touch screens to a point (phones, tablets), but I think at the PC level, it's just not what I am looking for. You see, that's the beauty of the PC/laptop; it picks up where touchscreens leave off--accuracy and speed. You can be very precise with a mouse (or even trackpad), and most can type much quicker on a keyboard that offers actual feedback. Maybe a touch interface is quicker in some respects, but I don't want to edit spreadsheets or type every email or document on a touchscreen. I guess it could have it's uses, but I don't want yet another screen to clean off every day.

All I have to say is that if they make it easy and smooth to tilt the screen like that and seamlessly switch between mouse/keyboard-driven UIs and a touch one, it could have some potential. One reason why I never got into the all-in-one touchscreen desktops (looking at you TouchSmart) was I don't necessarily want to reach up and touch the screen — I want to have it closer to me.

Can someone explain to me why an OS would have to switch modes to exclude keyboard and mouse input while handling touch input? When did it become impossible to handle more than two modes of input? What's the reason to prevent using the mouse, keyboard, and touch all at the same time? Multiple modes of input are what make computers easier to use!

I seriously hope they don't make computers that look like whats in the drawings of that patent. You'd have to stand up at your work station to use the touch interface! That's ridiculous!

my thoughts exactly. This kind of patent should basically be thrown out as obvious.

There are a lot of ideas that seem obvious after they've been patented. Lego, or intermittent windshield wipers, for instance. I'd say that's what makes them valuable ideas in the first place.

ok, so the sensor thing may be clever (makes one wonder what the clever heads over at microsoft have been doing all these years). But the hinge is something i have wondered why noone have bothered trying already.

Can someone explain to me why an OS would have to switch modes to exclude keyboard and mouse input while handling touch input? When did it become impossible to handle more than two modes of input? What's the reason to prevent using the mouse, keyboard, and touch all at the same time? Multiple modes of input are what make computers easier to use!

Not all of us are blessed with more than two hands. Personally, I don't see myself ever using a mouse, keyboard, AND touch at the same time... I don't want to use my nose as an additional input device. (please note how hard it was to keep it family-friendly with the third appendage...)

As most people before me, I have to say that a touch-based desktop doesn't appeal to me for ergonomic reasons. Even when it's implemented like described in the patent I have great reserves, because the screen is too thick to comfortably work with when it's lying on the table. A better implementation of a touch screen desktop would be a real touchscreen desktop, such as that amazing table MS made. Throw in a keyboard and an extra screen to mimic a typewriter and it's back to before the 1980s in the office. I'm not sure if that would be more ergonomical than the PCs we have now, though.

As most people before me, I have to say that a touch-based desktop doesn't appeal to me for ergonomic reasons. Even when it's implemented like described in the patent I have great reserves, because the screen is too thick to comfortably work with when it's lying on the table. A better implementation of a touch screen desktop would be a real touchscreen desktop, such as that amazing table MS made. Throw in a keyboard and an extra screen to mimic a typewriter and it's back to before the 1980s in the office. I'm not sure if that would be more ergonomical than the PCs we have now, though.

Like the slight angle the ipad can have when in the apple brand case? Or basically a touch screen drafting table?

Not trying to troll here but didn't Steve Jobs basically call touch-screens on PC's stupid? Now the following will praise this as the greatest thing since sliced bread.

It's more likely that he called current implementations stupid. They are at least doing something different with this (granted it's little), but it's hard to say what it will really be liked until the software to support it is shown and any other bells and whistles that haven't been revealed

A limit switch coupled with an OS hook... but I guess they'll make it fancy and use the accelerometers built into everything smartphone-ish instead.

Chris Foresman wrote:

Like Sasser, Siracusa also believes that a future version of Mac OS X could switch UI paradigms depending on user needs and the configuration of a hypothetical transforming MacBook/iPad device. Such a device would be "a light, fully capable Mac laptop when you need all the power it provides, and a slightly bulky (but screamingly fast) iPad when you don't."

Funny, seriosuly suggest the same thing for Windows, you get howls of protest about how that's impossible even though the levels of effort required would be fairly similar.

Actually, I'm thinking that the Magic Trackpad might be Apple's way of sneakily getting more and more OS X devs to support multitouch. We already had it on the laptops and now it came to the desktop. So once it's become a bit more ingrained and many applications use it the transition to this semi-touch-based Mac OS X will become even easier/obvious.

Actually, I'm thinking that the Magic Trackpad might be Apple's way of sneakily getting more and more OS X devs to support multitouch. We already had it on the laptops and now it came to the desktop. So once it's become a bit more ingrained and many applications use it the transition to this semi-touch-based Mac OS X will become even easier/obvious.

Not trying to troll here but didn't Steve Jobs basically call touch-screens on PC's stupid? Now the following will praise this as the greatest thing since sliced bread.

Steve Jobs says a lot of things. I remember him saying portable video was stupid (a year before releasing iPod with video), and also saying nobody reads any more when asked about ebooks.

I reckon you could always rationalize it away by interpreting these statements as a disingenuous "Every implementation out there TODAY of Feature X sucks." Even if that weren't the case, Steve Jobs isn't obligated to tell us what he REALLY believes at any time. I take sweeping statements of his with a pretty massive shaker of salt. What's probably more interesting is what he doesn't talk about. The dude can lie all day as long as Apple keeps making stuff people want.

As most people before me, I have to say that a touch-based desktop doesn't appeal to me for ergonomic reasons. Even when it's implemented like described in the patent I have great reserves, because the screen is too thick to comfortably work with when it's lying on the table. A better implementation of a touch screen desktop would be a real touchscreen desktop, such as that amazing table MS made. Throw in a keyboard and an extra screen to mimic a typewriter and it's back to before the 1980s in the office. I'm not sure if that would be more ergonomical than the PCs we have now, though.

I think you're talking about something like this (scroll to the bottom):

But then again, why would it even need to have any switching? As I said, let's assume that the Magic Trackpad and the multitouch pads on the Macbooks are the gateway devices! If you can consider the screen to be a big extension of that you can type-type-type-mouse-whatever, "whoops!" *drag screen a bit closer* *multi-touchery* then go back to typing some more.

Does certainly take me back to the G4 lampshade iMac, which I sincerely loved up-down-left-right then give it a little twirl!

my thoughts exactly. This kind of patent should basically be thrown out as obvious.

There are a lot of ideas that seem obvious after they've been patented. Lego, or intermittent windshield wipers, for instance. I'd say that's what makes them valuable ideas in the first place.

Well it is slightly different. The intermittent wiper patent was for a particular way of making the wipers intermittently wipe not for the concept. Also it had not been done before. This apple patent seems to be patenting the concept of switching ui which I am sure has been done before in various guises. I mean surely the whole point of UI is to switch. Some programs change UI when they detect a certain peripheral for instance. (How much of a UI has to change before it would infringe this patent?)

I would imagine that Apple's solution to mixed-mode would be something along the following lines:

New applications are written as iOS applications. You can't access current MacOSX APIs when in this mode, only APIs that are made available in iOS.

Old applications, or legacy applications remain in standard Mac OSX mode. These won't have access to the touch screen, or only in a limited way. They are still interacted with by using a mouse.

The hinge of the screen itself probably doesn't actually need to detect it's position. The user simply positions the screen so that it is comfortable to use for whichever paridigm the current app is using.

Anyway, that's how I see the platform evolving. It seems obvious to me that Apple will want to merge it's computer/consumer electyronics lines as time goes by, but they are going to need to maintain access to a relatively standard API if they wish to retain compatability with major software packages such as Photoshop and Microsoft Office. Hence mixed mode.

The Apple Haters are secretly crapping their pants at the thought that Apple might produce this and worse, have a patent on it! Where's your Windows god now?

Don't be ridiculous! People are unhappy as companies seem to be able to patent any concept nowadays and that is not a good thing whether it is Apple, Microsoft or little back water company. It is also seemingly becoming more and more common to be able to patent something which is already being done or has been thought of publicly a long time ago as this patent shows along with the apple patent for slide to unlock.

Apple should be perfectly able to patent new original ideas that they have in order to protect their products and business, however they seem to be abusing the patent system recently. (Although I blame the patent system and politicians rather than Apple).

Just to enhance the point, would you be happy if oracle patented the typing of text in to an on screen text box? or IBM patented the use of signals to identify future or present movement?

It's on the same topic, but they remark on a single insight that you missed: that, even though it is so absolutely obvious, NOBODY thus yet has implemented this on any of the currently available touch-enabled desktops.

It's on the same topic, but they remark on a single insight that you missed: that, even though it is so absolutely obvious, NOBODY thus yet has implemented this on any of the currently available touch-enabled desktops.

-dZ.

But I have used monitors that allow very similar manipulation. you can tilt them backwards and lower them. Many wall mounts also allow a similar degree of manipulation and I have seen POS screens do the same.