Big Data Drives 'National Day Of Civic Hacking'

Nationwide hackathon this weekend encourages coders to use publicly available data to tackle problems ranging from poverty to poultry handling.

5 Big Wishes For Big Data Deployments

(click image for larger view and for slideshow)

What do big data, software developers, and poultry farmers have in common?

More than you might think, actually. Organizers of this weekend's National Day of Civic Hacking hope to inspire socially aware coders to invent new apps that utilize publicly available data in innovative ways.

More than 95 Civic Hacking events will take place June 1-2 across the United States. Some 6,000 participants are expected, according to the event's official site and Intel, one of the hackathon's primary sponsors. Some 20 government agencies are participating as well, including the White House Office of Science and Technology Policy, NASA, the Census Bureau and FEMA.

At the weekend's events, developers, entrepreneurs, and other participants will use publicly released data and code to solve problems relevant to neighborhoods, cities, and the country, organizers say. About 75 data sets and resources will be available, many from federal, state, and local government agencies.

"[The National Day of Civic Hacking] is designed to demonstrate what value can be created through combinations of data sets, and what business models might be viable in this emerging data society," explained Brandon Barnett, director of business innovation at Intel, in a phone interview with InformationWeek. "Groups of people are going to come together to apply their expertise to solve civic problems, society's problems."

They'll do this by writing apps that use digital information -- personal data combined with big data. For instance, one challenge organized by the U.S. Census Bureau has the ambitious goal of leveraging statistics about every neighborhood in the country to create tools that look at everything from commute times to median income to poverty rates.

"This is going to be the largest number of sites of any kind of coordinated hackathon in the United States," Barnett said. "It's also the largest coordination of national government agencies that are supplying data and participating in challenges."

Another challenge, Backyard Poultry Farmer, aims to spur development of a poultry management system for people who raise chickens at home. These urban and rural farmers, many of whom have little or no agriculture background, face many difficulties, such as learning how to handle eggs properly so that they're safe to eat.

For Intel, the hacking event is the latest in a series of initiatives designed to draw attention to big data's impact on society. WeTheData, a research program the company conducted last year in collaboration with Vibrant Data Labs, a collective of scientists, artists, and designers, posed technical challenges that focused on the democratization of digital information.

The WeTheData research made it clear that data literacy is a fundamental principle that must be in place for a new "data society" to emerge, Barnett noted. Events like the National Day of Civic Hacking are designed to move that process by making hard-to get (and often inaccessible) data sets available to everyone.

Great. But why is a chipmaker like Intel involved in such philosophical undertakings?

"Even though historically we're a processor company, our legacy in the computing industry is really one of empowerment through technology," Barnett explained. "There's a history in the computing ecosystem of bringing capability that initially is limited to early adopters -- usually enterprises that can afford it -- and then democratizing it for the benefit of individuals."

Big data may very well follow a similar evolutionary path, Barnett added. But to get there, we'll need tools that empower people to function well in a data-driven world.

Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.

Why should big data be more difficult to secure? In a word, variety. But the business won’t wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more.