Optimizing Apps for the 2 in 1: A Better User Experience

A recent Lenovo Yoga 2 Pro review at Ars Technica ostensibly focused on the incredibly high resolution of this 2 in 1 device; but seemed to be more focused on something besides the size of the pixels:

“The device now comes with a staggering 3200×1800 of those pixels, packed into the same 13.3-inch screen as the original model. Coupled with a slight reduction in chunkiness and a bump in specs, this could set this device up as the ultimate drool-inducing portable in the $1,000 range.

After extensive testing, the Yoga 2 Pro’s size and screen certainly earn it that river of drool. However, in spite of its best qualities (did we mention all of those pixels?), the biggest drawback is the operating system they're tied to.”

What’s the problem here? Apparently, some apps are not optimized for different modes and/or form factors:

“The Metro interface, found in the Start menu half of the OS, works wonderfully at such a resolution. Its every element—and compatible app—is set to render at a size befitting the screen’s inches, not its pixels. Tapping through apps like the default mail client, Skype, Evernote Touch, and more just plain worked.

But Windows 8’s most useful apps—namely, Web browsing and office productivity—are still locked to the desktop. That wouldn’t be so bad if the applications smartly treated that space as a pixel-quadrupled 1600×900 space, as they do on the Retina line of MacBooks.”

This is a pretty strong call to action for developers. Developers who are able to optimize apps for these new flip devices in whatever mode the user decides to go with will definitely be ahead of the curve as the computing ecosystem moves more decisively towards these more versatile devices.

One developer who’s got this covered is Ideum, whose GestureWorks Gameplay software was featured in a recent case study. Gameplay is a customizable touch overlay that can be added to non-touch enabled desktop games as an example of how more apps can enable touch capability. This doesn’t require the game developer to have to change any of their code. It also offers 2-in1 state awareness, which is the ability for applications to switch modes with the 2-in-1 platform has been converted to either clamshell or tablet modes. In this case, the touch overlay only came up when the 2-in-1 was in tablet mode. More from the Ideum website:

“GestureWorks Gameplay is a revolutionary new way of interacting with popular PC games. Gameplay software for Windows 8 lets gamers use and build their own Virtual Controllers for touch, which are overlaid on top of existing PC games. In addition, gamers can use hundreds of personalized gestures to interact on the screen.” - “GestureWorks Gameplay Revealed Today in San Francisco”, Ideum Blog

Unlike traditional game controllers, virtual controllers can be fully customized, and gamers can even share them with their friends. Gameplay works on Windows, Ultrabooks, 2-in-1 laptops, all-In-ones, and even multitouch tablets and large touch screens. Here’s a quick video that explains how this works:

"The Virtual Controller is real! Gameplay extends hundreds of PC games that are not touch-enabled and makes it possible to play them on a whole new generation of portable devices," says Jim Spadaccini, CEO of Ideum, the makers of GestureWorks Gameplay. "Better than a physical controller, Gameplay's Virtual Controllers are customizable and editable. We can't wait to see what gamers make with Gameplay."

What about 2 in 1 support? They’ve got it:

“Early on in Gameplay development, we decided to include basic support for the 2-in-1 transition (going from clamshell to tablet mode) available on some new Ultrabooks. The vision was to hook into the game as usual but not display the overlay if it was launched in clamshell mode. Then, during game play, if the system was switched to tablet mode, the Gameplay Virtual Controller overlay would immediately appear to allow touch-only game control. You can see this capability in action on any virtual controller run on an Ultrabook with 2-in-1 support. In the virtual controller edit mode, just enable 2-in-1 mode switch support in the experimental section of the settings tab.”

Another app that manages to pull off this dichotomy is Krita Gemini, “a fusion between Krita Sketch and Krita Desktop…. switches automatically and seamlessly between the full-featured desktop/notebook user interface and the sketch interface, which is optimized for tablets”; focusing on 2-in-1awareness, use of the stylus, Intel ®AVX2 optimization, and V Tune™ optimization.

Intel has been supporting the work on Krita Gemini as a focus point for the new generation of Intel-powered 2-in-1 devices which can switch between desktop and touch mode. During development of this technology, Intel commented, "Krita Gemini is a brilliant example of how developers should utilize the option of a convertible device by using both tablet and desktop mode. The switching between the two modes works seamlessly." See how Krita Gemini works by clicking on the image below to watch a short video:

A recent case study on Krita Gemini explains extensively how developers should consider multiple scenarios and form factors while modifying their applications to take advantage of and conform to 2 in 1s. For example:

“Just as there are multiple ways that the 2 in 1 can transform between laptop and tablet modes, software can be designed in different ways to respond to the transformation. In some cases it may be desirable to keep the UI as close to the laptop mode as possible, while in other cases you may want to make more significant changes to the UI. For example, Intel helped KO GmBH combine the functionality of their Krita Touch application with their popular Krita open source painting program (laptop application) in the new Krita Gemini application. The Krita project is an active development community, welcoming new ideas and maintaining quality support. The team added the mechanisms required to provide seamless transition from the laptop “mouse and keyboard” mode to the touch interface for tablet mode.”

For more detailed resources on modifying apps to run on 2 in 1s, please consult the following case studies, tutorials, and articles:

Touch Developer Guide for Ultra Mobile Devices: This Guide contains information about the APIs that application developers need to use when they are developing apps targeted for Ultra Mobile Devices (PCs, Ultrabook™ devices, 2-in-1s, tablets) and Adaptive All in Ones, which have touch screens and multiple usages, such as a monitor or a multi-user tablet that may be easily moved to communal locations.

All-in-One PC: What are the Developer Possibilities?: All-in-One PCs are unique in that they deliver the full functionality of a desktop PC, but also deliver complete multimedia functionality with a touch screen interface in addition to a keyboard and mouse. An All-in-One PC is a high-performance computer with a near-instant wake up time, and is always on and always connected.

Windows 8* Store vs. Desktop App Development: When developing apps for Windows 8 and Intel devices, you have a unique opportunity and decision regarding which type of Windows 8 app you would like to develop. The following article helps define and point to resources to help you decide which type of app development is for you.

Intel® Graphics Performance Analyzers : The Intel® Graphics Performance Analyzers 2014 (Intel® GPA) is a suite of graphics analysis and optimization tools to help game developers make games and other graphics-intensive applications run even faster.

Developing Power-Efficient Apps for Ultrabook™ Devices: Power-efficient performance is a key component of UltrabookTM devices and mobile devices in general. While the new generation of Ultrabook devices comes with built-in hardware features that enhance performance, power efficiency is not the realm of hardware alone. Software plays a critical role in making Ultrabook devices as energy-efficient as possible. In this section, you will find resources to help you design and develop "green" software.

Windows* 8.1 Preview – What’s New for Developers: Windows 8.1 Preview is out, and many people are testing and checking the new features and capabilities. As an update of a major version, 8.1 has no big changes in the way developers create their applications, instead it has more capabilities and a few small additions to API functionalities. All the changes are well documented by Microsoft on MSDN here and here, but we will highlight a few interesting additions or changes that caught our attention below.