RIT emerges as leader in drone research

Remote Sensing

Faculty/Staff

Graduate

Mar. 22, 2015

Sean Lahman

(Photo: MAX SCHULTE/@maxrocphoto/ / STAFF PHOTOGRAPHER)

On a cold March afternoon, RJ Garma is flying a small quad copter in a parking lot at the Rochester Institute of Technology. The craft, about 2 feet wide, hovers several hundred feet in the air as Garma controls its movements using an application on his tablet. A camera mounted to the bottom of the copter sends overhead images of the campus back to him.

But Garma is not just a student out for a few hours of fun. He's a U.S. Air Force captain, a doctoral candidate in the school's imaging science program, and one of a handful of RIT students and faculty piloting a revolution.

The school has long been known for its expertise in aerial and satellite photography — a science known as remote sensing. So it's no surprise that RIT has now emerged as one of the world's leading centers for research on drones, small unmanned aircraft.

David Messinger, interim director of the school's Center for Imaging Science, says he gets calls almost every week from companies seeking this expertise, and graduating students are in extraordinarily high demand. Messinger says he can't recall one in the last 10 years who walked across the stage without having a job lined up.

"Our students don't bother going to the job fair," he says, "because when employers want our students they come here and talk to them directly."

The Digital Imaging and Remote Sensing Lab is the biggest group within the center, and about three-quarters of the DIRS students are working on masters and doctorate programs. They're highly sought after by government and industry alike.

The development of inexpensive flying platforms — as well as ever smaller and cheaper digital cameras — has suddenly made aerial photography something anyone can do. For a few hundred dollars, you can walk into the mall or go online at Amazon and get a simple drone capable of taking pictures. With a modicum of skill, just about anyone can take overhead pictures of an urban landscape or a natural wonder.

Those breathtaking photos are nice, but what's really driving interest, what's turning this into a science and big business, is the idea of converting those aerial images into useful data.

"If I launched a drone and over the course of half an hour it covered the entire RIT campus at a 1-inch resolution, I'm not going to be able to physically look at all of that data." Messinger explained. "You've got to have some back end processing schemes that try to extract information out of the data.

"That's what you really want. You don't want the pictures. Nobody cares about the pictures. You want the information that you can get out of it," he said.

One of the first areas to leverage this new technology is precision agriculture, and researchers at RIT have been developing systems to address issues like drought management and disease detection.

Here's how it works. A farmer launches a small drone by simply throwing it into the air. The aircraft circles a couple of times to orient itself, then it goes back and forth until it has taken high-resolution pictures of each individual plant in his entire field. When it lands, the information is downloaded to a computer that can begin analyzing the images, asking questions about what the images show.

Questions like: Is that plant healthy? Is there a gap in my irrigation system? Is there a broken pipe someplace or an infestation of something that's moving across the field?

Messinger says that the process usually starts with researchers going to customers and saying "this is what we can do, tell us if it's useful." If you could look at each plant in a thousand-acre field every day, what could you learn from that?

Salvaggio developed an inexpensive imaging system that can transmit live images of an area about an acre in size. It's designed for law enforcement or first responders who want to get an overhead view of what's going on in real time.

"A user can simply point at a spot on Google Maps," Salvaggio said. "The system will figure out where it is, turn and keep the camera trained toward that point on the ground the user selected."

In addition to the technology push, there's also an application pull: folks who come to RIT with a specific problem they need help solving. And perhaps the biggest problem is the one faced by the Federal Aviation Administration, which is charged with developing a plan for getting drones integrated into the national airspace.

They're concerned about these inexpensive fliers getting in the way of commercial aircraft, of course, and there are all sorts of technical and logistical issues that need to be addressed.

As one of the lead test centers for NUAIR, researchers at RIT are working on solutions for these challenges.

"With manned aircraft, we're pretty good at navigating from point to point," said Agamemnon Crassidis, a professor in RIT's Kate Gleason College of Engineering and the academic director for NUAIR.

Commercial aircraft use sophisticated navigation systems with an array of high-tech sensors.

"Those systems are large and they're expensive. You're not going to put an $80,000 inertial navigation system on a small unmanned aircraft," Crassidis said. "We're trying to develop sensors that are just as accurate but much cheaper, weigh less, use less power, and obviously are a lot smaller."

Collision avoidance is a major concern because drones are flying at lower altitudes than traditional manned aircraft. Crassidis says it's pretty easy to avoid buildings or hillsides, but that smaller obstacles — electrical wires or tree branches — present a more complex challenge. Part of the solution is developing better "detect and avoid" algorithms, but the real advances will be driven by those new sensors.

"The variety of potential applications for these unmanned systems is amazing, but we have to be able to do the testing to figure out how we can do those things safely," Crassidis said. "In terms of the technology, we're pretty close."

High-Resolution 3-D Scans Built from Drone Photos

Remote Sensing

Faculty/Staff

A drone spent hours swarming around Rio’s iconic Christ statue to show a cheap way to capture highly accurate 3-D scans.

Mar. 19, 2015

Tom Simonite

Left: This drone flew around Rio’s Christ the Redeemer statue to capture photos later used to make an accurate 3-D model of the monument.

The 30-meter tall statue of Christ overlooking Rio de Janeiro from a nearby mountain was under construction for nine years before its opening in 1931. It took just hours to build the first detailed 3-D scan of the monument late last year, using more than 2,000 photos captured by a small drone that buzzed all around it with an ordinary digital camera. The statue’s digital double was unveiled last month, and is accurate to between two and five centimeters, enough to capture individual mosaic tiles.

The project was intended to help efforts to preserve the statue and to demonstrate how drones could lead to a dramatic increase in high-resolution 3-D replicas of buildings, terrain, and other objects. Being able to easily and frequently capture detailed 3-D imagery could have many uses, such as speeding up construction projects and helping Hollywood make better special effects, says Christoph Strecha, CEO and cofounder of Pix4D, the Swiss company that led the project. It collaborated with drone manufacturer Aeryon Labs and researchers at PUC University of Rio de Janeiro.

Mapping tools from companies including Google and Apple have made outdoor 3-D imagery commonplace in recent years. But they are mostly built with 3-D data from aircraft carrying expensive equipment. The 3-D shape of buildings and terrain is most often captured using a technique called lidar, which uses lasers. The photo-real 3-D models of cities in Apple’s “flyover” mode are made by processing images captured by a complex and expensive array of cameras (see “Ultrasharp 3-D Maps”). Both techniques rely on very accurate GPS technology.

Drones don’t have very reliable GPS fixes by comparison, and can’t carry large sensors or cameras. But they are cheap, and Pix4D’s software can build highly accurate models by comparing many different overlapping photos, says Strecha. In fact, using lidar would have been impossible with the Rio statue, Strecha says, because of its size, shape, and location.

Other projects have also been weaving 3-D models from drone photos. Researchers led by Horst Bischof, a professor at the Technical University of Graz, Austria, are developing software that extracts information from such images. For example, for a company that restores old buildings, the researchers made a version of the software that calculates measurements necessary for producing custom-fit thermal insulation.

With the image processing more or less a solved problem, the ambitions of drone scanning will depend more on how well drones can be controlled or coӧrdinated in challenging conditions like the winds around Rio’s Christ, or to cover larger areas, saysCarl Salvaggio, a professor at Rochester Institute of Technology’s Center for Imaging Science. “Drones are good for small-scale projects but traditional aircraft offer the time in the air to collect whole cities,” he says. “Perhaps when there are ‘armies’ of drones in the air, we will see a different landscape emerge.”

Researcher talks of RIT’s role in finding new words on a 500-year-old map

Cultural Artifact and Document Imaging

Faculty/Staff

Chet Van Duzer will speak at RIT about findings on the Columbus-era map

Mar. 13, 2015

Greg Livadas

A historically significant map from circa 1491 is yielding new information not visible until a researcher from Rochester Institute of Technology’s Chester F. Carlson Center for Imaging Science processed images of the map collected under various colors of light.

Chet Van Duzer of California, a historian of cartography who is studying the results, will speak about the faded map and his findings Wednesday, March 18, at RIT.

His talk, at 7 p.m. in the Carlson Auditorium (Building 76), is free and open to the public and sponsored by RIT’s College of Liberal Arts and the College of Science.

The Henricus Martellus World Map, actually a painting about 6 ½ feet wide and 4 feet high, resides at the Beinecke Library at Yale University. Roger Easton, professor at RIT’s College of Science, said it is believed Christopher Columbus had a copy of this map when he sailed to America.

“The next significantly historical map was in 1507 and says ‘America’ on it,” Easton said. It is believed the Henricus Martellus map greatly influenced the 1507 map, but it was difficult to determine because the earlier map had faded.

“We went to Yale last year to image the map and Chet has been working on it ever since,” Easton said. “We took pictures under many colors of lights and you can use different combinations of the images under different lights to try to enhance the visibility of the things that have faded. He found all kinds of surprises,” including many words in Latin not visible to the eye.

The project also involved the University of Mississippi, MegaVision of Santa Barbara, Calif., and was sponsored by the National Endowment for the Humanities.

Plenty of Eyes in the Sky, Not Enough Minds on the Ground

General

Remote Sensing

The U.S. Intelligence Community Must Address a Workforce Gap in Remote Sensing Analysis

Mar. 5, 2015

Dr. Darryl Murdock, USGIF Vice President of Professional Development

In today’s world of light-speed satellite communications, advanced remote sensing, and supercomputers—and the mega-data they produce—we seldom think about who is applying all of this technology to meet our national security needs. It’s easy to act as if all available data is put in one end of a computer with the necessary information emerging at the other end.

While the Intelligence Community improves the technology needed to interpret this high volume of data and information, the sheer volume of data being consistently collected around the world mutes our existing analytic capability. At the intersection of technology and human intelligence are the GEOINT analysts who pore over the data retrieved by our increasingly sophisticated remote sensing technologies, assign the data context, and create actionable knowledge. GEOINT analysts regularly apply their skills to multiple national and international threat scenarios, military operations, and natural and manmade disasters. Without these dedicated GEOINT analysts we would have eyes (in the form of satellites and UAVs) on our tumultuous world but would not understand what the images they produce mean.

The Intelligence Community faces a rising demand for highly trained geospatial and remote sensing analysts. At a time of burgeoning and unprecedented threats including terrorism, asymmetrical warfare, and social unrest across the globe, the GEOINT Community is especially challenged as its workforce ages at a rate much faster than qualified analysts enter the workforce. Steps should be taken immediately to address this widening GEOINT analyst gap. Further delay will only make this current staffing problem more difficult and costly to address in the future.

Historical Precedent

The problem is simple to explain. In the aftermath of World War II and with the onset of the Cold War, the United States realized the pressing need for intelligence gathering. Aerospace and satellite technology developed in the ’50s and early ’60s gave the U.S. the necessary tools for this effort. Addressing the need for a highly skilled aerospace and intelligence analyst workforce, Congress passed the National Defense Education Act in 1958. The act provided U.S. universities with resources to improve technical education and graduate programs in order to produce an engineering workforce for the aerospace and intelligence gathering units of the federal government and commercial industry. This workforce was developed to observe and—more importantly—interpret the data that was just becoming available.

These first generations of intelligence analysts did a superb job during the Cold War and subsequently developed many of the remote sensing methods and technologies still used today. However, a large number of these pioneering geospatial analysts are heading into retirement and are not being replaced at a rate sufficient to bridge the mission performance gap.

A 2013 report by the National Academy of Sciences on the Future U.S. Workforce for Geospatial Intelligence claims qualified “GIS and remote sensing recruits are already hard to find.” It concluded that the nation needs to ensure an environment exists to create a STEM workforce trained specifically in remote sensing—with remote sensing being identified by the academy as one of the “five core areas on which the current production and analysis of geospatial intelligence relies.” An earlier report by the House Permanent Select Committee on Intelligence came to the same conclusion with respect to the projected analyst gap and recommended the U.S. government partner with universities to prepare more students for space and remote sensing analysis careers.

Positioning universities to produce more GEOINT engineers and remote sensing analysts is a national security imperative. A new national strategy is needed to ensure the health of U.S. GEOINT analyst education. However, any amendments to the National Defense Education Act to recognize the modern challenges facing the GEOINT Community and our universities would require substantial financial support. Universities should be incentivized, as was done in the ’50s, to greatly expand education and training programs and work with NGA and other U.S. intelligence agencies.

STEM-to-GEOINT

At USGIF, we strongly believe, given the current state of global security, that maintaining and expanding our nation’s GEOINT capabilities is critical—and addressing the GEOINT analyst workforce shortage is essential to doing so. We support the strengthening of the strategic relationship between the U.S. defense and intelligence communities and the U.S. academic community, particularly in STEM disciplines focused on addressing the technical collection and analysis efforts required by our nation.

To that end, I am in favor of a STEM-to-remote sensing pilot program focused on doubling the number of remote sensing analysts entering the Department of Defense and Intelligence Community within the next five years. This program could include tuition funding for up to four years, be open to both undergraduate and graduate students, and offer funded summer internships with industry, during which exposure to and work on hard problems should be the focus. All selected participants should also receive security clearance during their second program year, which would allow them to work within government facilities and on classified projects at their home school. When shown to be successful, such a program could be expanded to other geospatial intelligence sub-disciplines to develop cross-cutting, broad-based GEOINT analysts capable of producing finished intelligence products derived from a combination of remotely-sensed data, geographic information systems data, and open-source information.

Though some may argue such an initiative is not possible, similar programs have already been established in pockets around the globe. As our nation’s needs and priorities change, a STEM-to-remote sensing program could become a STEM-to-GEOINT program, which would include a strong and flexible remote sensing component.

Biomedical Imaging

Faculty/Staff

The beginnings of artificial vascular networks created with ultrasound waves. The frequency and intensity of the waves organize the cells into position.

Feb. 2, 2015

Susan Gawlowicz

Circulating oxygen-rich blood through artificial organs and tissues is a bioengineering conundrum without an easy answer. The problem is in the plumbing. Biomedical teams around the world are working on different solutions to the problem of creating a synthetic vascular system.

Scientists at Rochester Institute of Technology and the University of Rochester are looking to ultrasound technology to create tiny blood vessels needed to nourish organs and tissues grown for reconstructive and surgical applications.

Developing complex vascular systems with high-frequency ultrasound waves is the goal of RIT imaging scientist Maria Helguera ’99 (Ph.D., imaging science), and UR’s Diane Dalecki, professor of biomedical engineering, and Denise Hocking, associate professor of pharmacology and physiology. The UR-led project is funded by the National Institute of Biomedical Imaging and Bioengineering, part of the National Institutes of Health.

Helguera provides the team an expertise in ultrasound imaging and image processing through high-frequency ultrasound techniques and quantitative analysis of microscopy images of the tissue samples.

“We can use ultrasound in every stage of the process while creating artificial tissue,” said Helguera, an associate professor at RIT’s Chester F. Carlson Center for Imaging Science. “Ultrasound is a clean way of doing things in the sense that we are not delivering any harmful radiation. We can manipulate the cells and image them without hurting the samples.”

Ultrasound standing wave fields are generated in a tissue-culture plate containing endothelial cells embedded in collagen. Endothelial cells form the insides of blood vessels. Pressure from ultrasound standing wave fields nudges the cells into predetermined positions. The frequency and intensity of the waves organize the cells and control the density and spacing of cell bands. Samples are then polymerized in an incubator and locked into place. Their close positioning encourages cells to signal to each other and sprout three-dimensional blood vessels.

Tissue constructs are visualized with multiphoton microscopy, a technique that captures specimens in three-dimensional sections.

“Distinct and interesting formations can be seen depending on the frequency and intensity of the ultrasound stationary wave fields,” Helguera said. “We decided to quantitatively analyze these three-dimensional data sets to extract parameters characteristic of each exposure regime.”

Imaging science Ph.D. student Mohammed Yousefhussien developed an image-processing tool for evaluating the structures of the blood sprouts. Third-year imaging science student Amy Becker is modifying the tool to capture details that will help manipulate the sprouts’ growth. Determining the preferred direction in which the vessels branch outward will lead to networks resembling the vascular system within an organ.

“The goal is to design a quantitative protocol that will allow us to create a more complicated structure that is closer to a real system,” Helguera said.

N.Y. application for a national photonics center advances to final round

General

Photonics

RIT joins UR, SUNY Polytechnic in leading consortium that could receive $110 million federal investment

Jan. 30, 2015

Ellen Rosen

Rochester Institute of Technology is among the leaders of a consortium named as one of three finalists in the country competing for a multimillion-dollar federal investment in a regional photonics center.

The application from the Research Foundation for the State University of New York, which includes RIT, University of Rochester, SUNY Polytechnic, Massachusetts Institute of Technology, University of Arizona, University of California at Santa Barbara and other academic and industrial partners, has advanced to the final round, Democrat New York Sens. Charles Schumer and Kirsten Gillibrand and Congresswoman Louise Slaughter announced. Full proposals are due March 31 and a winner will be announced in June, they said.

“This project would allow us to build national capabilities to support this strategically important industry,” said Ryne Raffaelle, RIT vice president for research and associate provost. “With one of the largest photonics manufacturing hubs in the nation, Rochester is uniquely positioned to take the lead, and RIT’s renowned work in microelectronic systems, imaging science and packaging will play a major role. We are very grateful to Senators Schumer and Gillibrand and Congresswoman Slaughter for championing our application, and to President Obama for his leadership in recognizing the importance of the industry to our nation’s future.”

Raffaelle said advancing photonics is essential to the nation’s manufacturing capabilities in such areas as high-speed data and telecommunications. These new technologies will allow for more information to be transmitted easier, faster and with less energy.

“Rochester is home to the world’s greatest concentration of companies, university programs and expertise in the field of photonics, and this proposed partnership would further position Rochester as a global leader in this cutting-edge industry,” Schumer said in a statement.

Gillibrand said in a statement that Rochester “would be the perfect home for the new National Institute of Photonics, and this selection to move to the final phase of consideration shows that Upstate New York’s strong community of manufacturers and innovators is prime for these types of investments.”

The application was in response to a program announced by Obama last October in which the Department of Defense will take the lead in constructing an Integrated Photonic Manufacturing Institute with a $110 million federal commitment, Slaughter said.

“Today, we are one step closer to securing a federal photonics manufacturing innovation institute. I will continue to be relentless in my efforts to secure state and federal investments for an industry that is synonymous with Rochester because I know what it means for our economy and for local jobs,” Slaughter said in a statement.

Earlier this year, RIT was named a core partner in the Chicago-based Digital Manufacturing and Design Innovation Institute and is slated for an investment of up to $20 million.

RIT has contributed to advances in the design, fabrication and manufacturing of electronic and photonic devices for more than 30 years as technology generations have progressed from the micron-scale to the nano-scale.

RIT’s leadership includes:

The microelectronics program, created in 1982, was the nation’s first Bachelor of Science program specializing in the fabrication of semiconductor devices and integrated circuits.

The microsystems engineering Ph.D. program began in RIT’s Kate Gleason College of Engineering in 2002.

The university’s first doctoral program was imaging science in 1990, the first of its kind in the nation.

More than 2,000 RIT engineers have been placed into related engineering positions across New York state and throughout the U.S., Europe and Asia.

RIT assets in this area include:

Semiconductor and Microsystems Fabrication Lab: This includes more than 10,000 square feet of cleanroom space dedicated to manufacturing support and workforce development. http://www.smfl.rit.edu/

The Center for Electronics Manufacturing and Assembly: The Center is an academic research lab offering the electronics packaging industry research services, failure analysis, training, process development, consulting and laboratory rental.http://www.rit.edu/cast/cema/

RIT Nanophotonics Group: The group aims to demonstrate optoelectronic chips that will revolutionize future computing, communication and sensing systems.http://www.rit.edu/~w-nanoph/photon/

The Center for Detectors: The center designs, develops and implements new advanced sensor technologies through collaboration with academic researchers, industry engineers, government scientists and university/college students.http://ridl.cfd.rit.edu/

Imaging Science Sophomore Named Liberty League Field Athlete of the Week

Undergraduate

Track and field standout Nick Ng garners Liberty League Field Athlete of the Week honor

Jan. 27, 2015

Joe Venniro

ROCHESTER, NY - Sophomore jumper Nick Ng (Wayland, MA/Wayland) of the RIT men's track and field team, was named the Liberty League Field Athlete of the Week on Monday, for the week ending Jan. 25, 2015. It is his first weekly honor.

Ng placed fourth in the long jump with a leap of 6.50 meters at the Brockport Golden Eagle Invitational on Saturday. It was a personal best for the sophomore.

The Tigers are back in action on Saturday, Jan. 31 at the Robert J. Kane Invitational, hosted by Cornell University.

Remote Sensing

It's a significant step, since the Federal Aviation Administration generally prohibits commercial use of these devices, commonly referred to as drones or UAVs. They're concerned about these inexpensive fliers getting in the way of commercial aircraft, of course, but the agency has been slow to adapt existing rules to accommodate the new technology.

In 2012, Congress ordered the FAA to develop a plan for getting drones integrated into the national airspace, but progress has not come quickly. To date, the FAA has granted exemptions to just 13 companies, many of them in the motion picture business.

"Our aim is to get beyond hobby-grade equipment and to establish what options are available and workable to produce high quality video journalism using various types of UAVs and camera setups," Vigilante said.

Look, we know all about photography here. Even in the digital age, we've got a tremendous aggregation of experts in imaging science working in this region, some of the brightest minds in the world.

But the coming boom isn't simply about capturing cool images. It's about harnessing computing power to do things with those images. And we've got the experts in that field as well.

Pictometry International, a Henrietta-based company, developed the technique of stitching together aerial photos from low-flying airplanes to create overhead images that look three-dimensional. They've also developed software and algorithms that can pinpoint locations in those photos by latitude and longitude, and even make precise measurements of things like the square footage of a building's roof.

The folks at Exelis Geospatial Systems in Gates work from even greater heights. Their researchers have designed and built the camera systems for the majority of commercial imaging satellites that have been launched, starting with the first one in 1999. Even from 373 miles in the air and traveling at 17,000 mph, they can pinpoint a spot on the ground to within a few meters.

It's this sort of technology that's really going to drive the commercial applications of drones. Software that can analyze images taken from drones to do new things, or to do old things in new ways.

Farmers could use drones to look for crop or irrigation problems, or even to keep a watch on their livestock. Utility companies could use drones to inspect pipelines or electrical wires. Imagine how a drone could change the job of a building inspector, for example, by using its camera to take measurements and identify trouble spots in areas that are difficult and dangerous for a person to go.

CIS High School Intern Program Now Accepting Applications

Interns

The 2015 application cycle for the CIS High School Summer Intern program is open through March 6.

Jan. 6, 2015

The summer of 2015 marks the sixteenth year of the high school summer internship program at the Center for Imaging Science. This unique program offers a limited number of highly qualified juniors the opportunity to work side-by-side with world class scientists on a variety of imaging-related research projects.

These unpaid internships give students the chance to get valuable hands-on experience in a real laboratory setting as contributing members of a research team. The internship program also provides an opportunity for interaction with other students from surrounding school districts who have similar interests and ambitions. Professional development activities, team building exercises, and at least one field trip are additional benefits.

Participation in this program is free, and upon successful completion of research students are provided a certificate of completion as well as letters of recommendation.

All current high school juniors* are eligible to apply. Application instructions as well as more information are available via our High School Internship webpage.

*Notice regarding non-local applicants: While this internship is not limited to local high school students, it is not within the bandwidth of the program to offer any housing or housing assistance. If a non-local student wishes to apply, they must prepare appropriate living arrangements independently.