Thursday, October 26, 2017

Questions America Wants Answered: "Is AI Riding a One-Trick Pony?"

Just about every AI advance you’ve
heard of depends on a breakthrough that’s three decades old. Keeping up
the pace of progress will require confronting AI’s serious limitations.

I’m standing in what is soon to be the center of the world, or is
perhaps just a very large room on the seventh floor of a gleaming tower
in downtown Toronto. Showing me around is Jordan Jacobs, who cofounded
this place: the nascent Vector Institute, which opens its doors this
fall and which is aiming to become the global epicenter of artificial
intelligence.

We’re in Toronto because Geoffrey Hinton is in Toronto, and
Geoffrey Hinton is the father of “deep learning,” the technique behind
the current excitement about AI. “In 30 years we’re going to look back
and say Geoff is Einstein—of AI, deep learning, the thing that we’re
calling AI,” Jacobs says. Of the researchers at the top of the field of
deep learning, Hinton has more citations than the next three combined.
His students and postdocs have gone on to run the AI labs at Apple,
Facebook, and OpenAI; Hinton himself is a lead scientist on the Google
Brain AI team. In fact, nearly every achievement in the last decade of
AI—in translation, speech recognition, image recognition, and game
playing—traces in some way back to Hinton’s work.

The Vector Institute, this monument to the ascent of ­Hinton’s
ideas, is a research center where companies from around the U.S. and
Canada—like Google, and Uber, and Nvidia—will sponsor efforts to
commercialize AI technologies. Money has poured in faster than Jacobs
could ask for it; two of his cofounders surveyed companies in the
Toronto area, and the demand for AI experts ended up being 10 times what
Canada produces every year. Vector is in a sense ground zero for the
now-worldwide attempt to mobilize around deep learning: to cash in on
the technique, to teach it, to refine and apply it. Data centers are
being built, towers are being filled with startups, a whole generation
of students is going into the field.

The impression you get standing on the Vector floor, bare and echoey
and about to be filled, is that you’re at the beginning of something.
But the peculiar thing about deep learning is just how old its key ideas
are. Hinton’s breakthrough paper, with colleagues David Rumelhart and
Ronald Williams, was published in 1986. The paper elaborated on a
technique called backpropagation, or backprop for short. Backprop, in
the words of Jon Cohen, a computational psychologist at Princeton, is
“what all of deep learning is based on—literally everything.”

When you boil it down, AI today is deep learning, and deep learning
is backprop—which is amazing, considering that backprop is more than 30
years old. It’s worth understanding how that happened—how a technique
could lie in wait for so long and then cause such an explosion—because
once you understand the story of backprop, you’ll start to understand
the current moment in AI, and in particular the fact that maybe we’re
not actually at the beginning of a revolution. Maybe we’re at the end of
one.

Vindication
The walk from the Vector Institute to Hinton’s office at Google,
where he spends most of his time (he is now an emeritus professor at the
University of Toronto), is a kind of living advertisement for the city,
at least in the summertime. You can understand why Hinton, who is
originally from the U.K., moved here in the 1980s after working at
Carnegie Mellon University in Pittsburgh.

When you step outside, even downtown near the financial district,
you feel as though you’ve actually gone into nature. It’s the smell, I
think: wet loam in the air. Toronto was built on top of forested
ravines, and it’s said to be “a city within a park”; as it’s been
urbanized, the local government has set strict restrictions to maintain
the tree canopy. As you’re flying in, the outer parts of the city look
almost cartoonishly lush.

Maybe we’re not actually at the beginning of a revolution.

Toronto is the fourth-largest city in North America (after Mexico
City, New York, and L.A.), and its most diverse: more than half the
population was born outside Canada. You can see that walking around. The
crowd in the tech corridor looks less San Francisco—young white guys in
hoodies—and more international. There’s free health care and good
public schools, the people are friendly, and the political order is
relatively left-­leaning and stable; and this stuff draws people like
Hinton, who says he left the U.S. because of the Iran-Contra affair.
It’s one of the first things we talk about when I go to meet him, just
before lunch....MORE