Friday, August 28, 2009

Data Deluge Swamps Science Historians - WSJ.com

Interesting times

Usually, historians are hard-pressed to find any original source material about those who have shaped our civilization. In the Internet era, scholars of science might have too much. Never have so many people generated so much digital data or been able to lose so much of it so quickly, experts at the San Diego Supercomputer Center say. Computer users world-wide generate enough digital data every 15 minutes to fill the U.S. Library of Congress.

In fact, more technical data have been collected in the past year alone than in all previous years since science began, says Johns Hopkins astrophysicist Alexander Szalay, an authority on large data sets and their impact on science. "The data is doubling every year," Dr. Szalay says.