Screen Play

My daughter sits cross-legged on the ﬂoor, surrounded by toys: a dollhouse, a pile of wooden blocks, a princess gownin a forlorn heap, a softened old box of crayons. The rug creates a blank space—an opening for imagination. As she plays, everything becomes something else: She is a teacher,a doctor, a ballerina, an explorer; the beige rug turns into a beach; stuffed animals acquire life histories; thumb-size ﬁgurines tell jokes. It often feels like we have joined an animist cult, in which everything in the material universe—but especially cats and pigs—is poised for conversation. “Preschool children,” the pediatrician/child philosopher Dr. Benjamin Spock once pointed out, “are virtuosos of imagination.”

Enter the smartphone, inevitably lurking nearby. The adults treat it as the most interesting thing in the room—a truth that never escapes children. If preschoolers are virtuosos of imagination, wizards casting spells, then adults are the opposite: we’re zombies, lost in our crystal slabs. We stare at it, fondle it, keep it close. Its materiality is odd. At rest, it is a black mirror. Alive, it is different in kind from everything around it. Its screen asks for a response, but its tactility is strange; things happen when you touch it, but it always feels the same. It appears to be of inﬁnite capacity. It sings unexpectedly, and then the voices and faces of faraway people appear, and they can hear and see you, via an almost-hidden camera. It responds to movement; it can appear transparent; it seems to be connected to everywhere, and to know everything. In the words of its most famous creator, it is a “magical” device.

All of these characteristics have become banalities for adults. For children they are expectations. That strikes me as new and strange. The conversation about smartphones and children usually centers on the risks and consequences of too much “screen time.” There are books with titles such as iBrain: Surviving the Technological Alteration of the Modern Mind, and longitudinal studies about what happens to children who watch three hours of television a day. But separate from brain activity or social dysfunction or violence or the litany of claims about what media does to our children, I’m hung up on a more fundamental question of how we understand the objects around us. With smartphones and tablets, the very basics of ownership and ontology have been pulled out from under us like a rug—or, perhaps, swiped away. In the digital landscape of apps, online media, and multipurpose devices, the three primal questions of small children—

What is it?
Where does it come from?
Is it mine?

—are difﬁcult enough for adults to answer for themselves. For children, they are mystifying:

A phone is a camera.
The message ﬂies invisibly through the air.
It is yours—after you watch this zit-cream commercial.

This is a design problem. The best design has always intertwined function with some greater truths about the object. Except objects are no longer objects; they are part-digital, part-physical constructions—that nevertheless can be seen and touched. With smartphones especially, the entire framework of ownership, interaction, and mechanism is changed. Adults have quickly, if sloppily, adapted to this. (We mangle our tendons. We unwittingly reveal our deepest desires to large corporations. We butt-dial.) But seen through the lens of children’s apps, the strangeness of it all becomes apparent again. This “magical device” is clearly a vital competitor to the magic already present in their imaginations. The consequences of this strike me as a bigger question than when television was presented to today’s great-grandparents. We’re in the horseless-carriage phase, bringing the old metaphors to a new medium. As Scott Chambers, Sesame Workshop’s senior vice president for worldwide media distribution, put it to me: “We’ve spent forty years perfecting this linear video experience, and in fact proving that it’s effective as a teaching tool. So now what happens with all of that.

My ﬁrst inkling of this was arrival of Balloonimals in 2009—design-powerhouse IDEO’s early foray into the kids’ app world. Rather than feeling like an old-style video game in which the “buttons” had simply migrated to the touch screen, it cannily reﬂected the newness of the device. The premise of it is to “blow up” a balloon, twist it into an animal, play with the animal, and then inﬂate it more until it pops. Then do it again.The gimmick is that you blow up the animal by blowing into the phone’s microphone, lifting it to your lips and holding it sideways, like a whistle. Adults do a double take to remind themselves that it’s not their breath going into the phone, but the rustling of its wind through the microphone. I have no idea what children think. They smile, invariably. It seems to conﬁrm the tendency of the world around them toward mystery and surprise. To turn the balloon into an animal, you shake the phone—an old magician’s gesture. The animal arrives on the screen with a pop, and then it responds to your touch: dinosaurs roar, ducks swim in a circle, kangaroos hop—their landing marked by a vibration from the phone, as if the animal were really shaking around inside. The mix of the physical and the digital is uncanny. (As I write this I am lightheaded from blowing up so many balloons.) As IDEO’s Adam Skaates explained, it all came out of their initial research into how kids use the phones. It was all about big, physical gestures, which inspired the blowing, shaking, and tapping.

At ﬁrst it seemed charming, quirky, a sly use of the iPhone’s sensors. But, as I’ve begun to appreciate the pervasiveness of these devices—and as my own children have begun, slowly, to make sense of the world around them—I have grown increasingly perplexed at this recurring slippage between the physical and the digital. Interactive designers are fond of talking about the pros and cons of the “skeuomorph,” those little cues that connect the things on our screens to their physical precursors, such as yellow Post-it notes, calendar rings, and lined note-book paper. But the physicality of Balloonimals is a whole other animal. It suggests that the world inside the machine is seamless with the world outside—which I suppose is partially true. But only partially. The rules are inconsistent. A real balloon animal shrivels, over days, into a sad heap. A virtual balloon animal disappears, and another instantly takes its place, sliding in from off-screen, and bouncing into place—the kid version of the iPhone’s inertial scrolling.

These are no longer the rules of physics, but of Silicon Valley. “As we continue to work in this space, we see that kids are digital natives now,” Skaates explained. “In their minds there’s not a real change in experience in going from blocks to a digital device. They’re both toys and they’re both entertaining in different ways, and they’re happy to use them together if they make sense.” The hang-up, apparently, is my own.

But there is an in-between: a way of recognizing the differences between the digital and physical worlds, while still playing at them. Duck Duck Moose, one of the most prominent kids’ game studios, takes a more straightforward approach to its creations, carefully framing the world inside the game. Cofounder Caroline Hu Flexer—herself an IDEO veteran, with an architecture degree from Princeton—recalls the moment she realized how readily the magic of the phone blurs with the magic of the physical world. They were testing a drawing app, and she watched as a child picked a purple paintbrush, and then changed her mind. “So what did she do?” Hu Flexer says. “She actually wiped her ﬁnger on her pants. For kids this seamless blurring of the digital and real world return is not necessarily apparent.”

Duck Duck Moose’s games have a appealing simplicity. The drawing games are a stripped-down evolution of MacPaint, the key interactive tool of my own childhood; a “Wheels on the Bus” song app is more like a digital version of a lift-the-ﬂap book. Tap a stuffed bear and he pops up in his seat; touch a cat and it purrs. This simplicity of cause and effect is deliberate. An early iteration of another game, Draw and Tell, had virtual stickers that behaved like stickers do in real life—which is to say, once you put it down, it was difﬁcult to move. “That’s how things work in the physical world,” Hu Flexer says. “But what we found is when children are playing with these devices, they expect everything to be interactive and moveable.” It is an expectations game, which is to say it’s about the trial and error of understanding the world. And with apps, sometimes things get strange.

But what else do they expect? IDEO’s work on Balloonimals—which was sold directly via the iTunes App Store—got the in-house Toy Lab noticed by Sesame Workshop, the nonproﬁt educational organization behind Sesame Street. Their collaboration led to 2012’s Elmo Calls, which is a wonderful, if thoroughly surreal, extension of the iPhone’s phone-ness. Its icon shows Elmo holding a classic Ma Bell corded phone headset. But when you open it, the game mimics the iPhone’s phone interface, with big speed-dial buttons, and a voicemail indicator—marked, similarly anachronistically, by the tra-ditional symbol of a reel-to-reel tape. Tap one of the dial buttons, and Elmo calls you, with the game mimicking the FaceTime interface, with your own image in the bottom corner. The illusion is complete: just as Grandma might appear on the screen for a chat, so too does Elmo. A hidden menu allows Elmo to call at preset times with speciﬁc messages: get in the bath, go to bed, brush your teeth. It’s dif-ﬁcult to parse all the layers going on here—to pin down the fact and ﬁction of a video-chatting furry red monster. But I’m struck again by the ﬂuidity of how children understand what this device is and does. From IDEO’s perspective, it doesn’t matter much. “Our research showed that kids didn’t totally understand the metaphor of the phone,” Skaates said. “The phone is a little bit of an adult construct. It’s a way to tell the story of the app that parents will ﬁnd entertaining.”

We can brush all this off as a harmless technological effect—no more surprising or magical than furry monsters themselves. And yet the design mode at the moment is to keep ﬂexing the metaphors until they become just the ordinary way the world is. We are interacting with machines, on multiple and multiplying levels. Video chatting with Grandma is a legible stand-in for a phone call, which is itself a stand-in for being face-to-face. But what is video chatting with a monster? And what happens when that monster costs $0.99, plus an additional 99 cents for the Happy Habits pack? What is it then? How do you share a non-object? How do you take pride in its ownership? How do you care for it? If you’re a child, how do you let the objects in your life help you construct an identity, in the way of a lovey or the tacked-up poster on a wall? These are questions that are difﬁcult to ask of apps and the current generation of digital toys. They’re questions that seem vital to me now, but may in a generation seem the last gasp of an old way of looking at the world.

What’s clear is that things are getting still stranger. This winter, Sesame Street announced the impending arrival of Big Bird’s Words, its ﬁrst foray into augmented reality, in collaboration with Qualcomm. In the app, Big Bird asks kids to ﬁnd letters in the world around them. Using the phone like a camera, they point it at the back of a cereal box, at a sign, or the cover of a book—and Big Bird acknowledges their success at ﬁnding the letter. It is thrilling, yet whiplash-inducing. The phone becomes a monocle through which to look at the world. It is pretend layered on top of pretend—instantiating on the screen the imaginary world that already seems to be inside their heads. We are creating new worlds. With what effects?

IDEO’s Skaates recognizes augmented reality as, ironically, potentially limiting. “Their imaginations are so evolved, they can move the doll or the vehicle around and have this imagined storytelling unfold in their minds,” he says. “But in our experience with some of the stuff we designed, it’s almost like it limits their imagination. It conﬁnes it and constrains it and tells it how it’s supposed to be—which is magical for an adult, but I think sort of awkward for a kid.”

My son, three years younger than his sister, is days away from crawling. He sits up beside her on the rug, ﬂailing his arms and chewing on the legs of a rubber giraffe. Put a tower of blocks in front of him and he knocks it down, startled at the crash. Kids have always played at life—the cooking sets, the dolls, the tumbles and breaks. This shift to digital experience is profound, yet so new that no one, not even the ostensible experts, knows how it will evolve—on the part of both the toys and the children. But watching from the edge of the rug, it’s difﬁcult to imagine physical play won’t fundamentally endure.