Data Flood Feeds Need for Speed

Share

Data Flood Feeds Need for Speed

Scientists can break records just like athletes, but new speed and data-transfer milestones in high-performance computing may prove to be more than just fodder for the record books.

An international team of physicists who set a world record for the amount and speed of data transferred over a broadband network will attempt to best their own mark in the next few weeks. The advances they make, researchers say, promise to greatly enhance their future work, as well as their ability to collaborate with scientists all over the world.

The team set the Internet2 Land Speed Record in November at SC2002, a high-performance networking conference. Internet2, a coalition of universities that have developed a high-performance research and education network separate from the public Internet, modeled the contest after the Land Speed Record competition, where drivers compete to set speed records on Utah's salt flats.

The team sent 6.7 GB of uncompressed data at 923 megabits per second in just 58 seconds from Sunnyvale, California, to Amsterdam. That's the data equivalent of four hours of DVD-quality movies, and a transfer speed 3,500 times faster than a typical household broadband connection.

Data was transmitted by packets called jumbo frames, which are 9,000 bytes – six times as big as the packets normally sent over the Internet. The team used PCs running Debian GNU/Linux in Amsterdam and RedHat Linux in Sunnyvale. Cisco provided a router.

The record is another step forward for high-performance computing.

"High-energy physics is a global enterprise where data needs to be available and analyzed at many sites around the world," said Les Cottrell, director of computer services for the Stanford Linear Accelerator Center. "We're creating terabytes of data a day – that's a million million bytes per data – and it's increasing each year. We need to share it with our collaborators around the world."

Higher data-transfer rates are crucial, Cottrell said, because they must keep up with the rates at which the data is produced.

Cottrell said the ability to send massive amounts of data over high-speed networks will benefit other fields, including telemedicine, the Human Genome Project and astronomy.

For example, doctors in far-flung locations will be able to discuss a particular patient's X-rays by quickly sharing the information over these fat pipes. The Human Genome Project might transfer its enormous stores of data faster on more robust networks. Likewise, astronomers can share telescope data located all over the world.

Internet2 spokesman Greg Wood said the Internet2 Land Speed Record is an indicator of networking capabilities researchers will need in the coming years.

For example, the Large Hadron Collider – located at CERN in Switzerland – will be coming online in the next few years, he said, and it's a resource that all physicists in the United States will want to have access to.

"The near-gigabit-per-second speeds over transcontinental distances are crucial to providing U.S. researchers access to the data that LHC will produce," Wood said.

The record also demonstrates the power of networks that researchers already use. But often, Wood said, the PCs and local networks connected to the Internet are not configured to support high-speed networking.

"In our day-to-day lives, when we use the commercial Internet, we sort of assume that the Internet is going to be the bottleneck," he said. "There's never a focus on the other pieces of the networking puzzle – the computers and the local area networks."

One physicist not involved in the project said such advances in networking will help the work of scientists all over the globe.

"The research that we do involves large international teams scattered all over the world," said Bob Cahn, senior physicist at Lawrence Berkeley National Laboratory. "We work with enormous quantities of data and we have to share the data around the world.