News Main Menu

A Penn State-led team has received a $10 million Expeditions in Computing award from the National Science Foundation (NSF). The five-year award is for the project "Visual Cortex on Silicon" is one of only two announced by the NSF.

UNIVERSITY PARK, Pa. -- A Penn State-led team has received a $10 million Expeditions in Computing award from the National Science Foundation's (NSF) Directorate for Computer and Information Science and Engineering (CISE).

The five-year award is for the project "Visual Cortex on Silicon" with Vijaykrishnan Narayanan, professor of computer science and engineering and electrical engineering, as the lead principal investigator. The award is one of only two announced by the NSF.

"The Expedition in Computer program represents the single largest investment in computer science research the NSF makes. The immense technical challenges spanning multiple disciplines and the potential for wide societal impact make this silicon on visual cortex project very unique," said David Wormley, the Harold and Inge Marcus Dean of the College of Engineering.

The institutions collaborating on the effort include the University of Southern California; Stanford University; York College of Pennsylvania; the University of California, San Diego; the University of California, Los Angeles; the University of Pittsburgh; and the Massachusetts Institute of Technology.

The project envisions a holistic design of machine vision system that will approach or exceed the capabilities and efficiencies of human vision, enabling computers to not only record images, but also understand visual content, at up to a thousand times the efficiency of current technologies.

While several machine vision systems today can each successfully perform one or a few human tasks -- such as detecting human faces in point-and-shoot cameras -- they are still limited in their ability to perform a wide range of visual tasks, to operate in complex, cluttered environments and to provide reasoning for their decisions. In contrast, the visual cortex in mammals excels in a broad variety of goal-oriented cognitive tasks, and is at least three orders of magnitude more energy efficient than customized state-of-the-art machine vision systems.

Narayanan said, "We have already been collaborating with colleagues at USC and MIT in developing smart camera systems for the past five years and demonstrated vision systems that operate with two to three orders of better energy efficiency than existing approaches. With this expedition, we are aiming to leapfrog the intelligence of these vision systems to approach human cognitive capabilities, while being extremely energy efficient and user friendly."

The expedition seeks to understand the fundamental mechanisms used in the visual cortex, with the hope of enabling the design of new vision algorithms and hardware fabrics that can improve power, speed, flexibility and recognition accuracies relative to existing machine vision systems. The interdisciplinary effort covers several domains, including neuroscience, computer vision, hardware design, new device technology, human-computer interface, data analytics and privacy.

Smart machine vision systems that understand and interact with their environments could have a profound impact on society, including aids for visually impaired persons, driver assistance capabilities for reducing automotive accidents and augmented reality systems for enhanced shopping, travel and safety.

"This project is well aligned with the new brain initiative launched by President Obama. Expedition is considered one of the most prestigious large-scale research grants from the NSF," said Raj Acharya, head of computer science and engineering. "The breadth of expertise that this project brings to designing visual cortex on silicon underlines the value of cross-disciplinary research in solving grand challenges. Projects of this magnitude that cut across research domains and institutions are vital for our nation's technology research."

In addition to Narayanan, the Penn State effort includes John Carroll, professor of information sciences and technology; Chita Das, distinguished professor of computer science and engineering; Suman Datta, professor of electrical engineering; Lee Giles, professor of information sciences and technology; Dan Kifer, assistant professor of computer science and engineering; and Mary Beth Rosson, associate dean for undergraduate studies in information sciences and technology.

The first expedition awards were made in 2008. As of today, 16 awards have been made through the program, addressing subjects ranging from foundational research in computing hardware, software and verification to research in sustainable energy, health information technology, robotics, mobile computing and Big Data.

"The Expedition in Computing program catalyzes large-scale, far-reaching and potentially transformative research motivated by deep scientific questions," said Farnam Jahanian, CISE assistant director. "These two new awards aim to apply our understanding of natural, biological capabilities to the development of revolutionary new computing and information technologies with tremendous potential for societal benefit."

Basu Mitra, the NSF manager for the Expeditions in Computing program, said, "We just completed the fifth expeditions competition. Year after year, we see ideas that not only address some of the most challenging problems in computer science and engineering, but also hold promise for making momentous impact on society."

Das, who also serves as the project's outreach leader, added, "In addition to the large societal impact, the multidisciplinary nature of this project can have significant broader impact on education and outreach activities. It should rejuvenate interest in computer science and engineering, facilitate technology transfer and stimulate K-12 education. Early prototypes of these ideas have been hugely popular in science fairs across the nation including the State College Exploration-U science days."