Big Data drives information tech careers into crazy new frontiers

Ever notice how something you post on Facebook brings a related ad to your page the next time you visit? Like you upload a new photo of your dog, then pet food ads appear. Or, you join a hot game like Cityville and suddenly its developer, Zynga, tries to hook you on a slew of its other games such as Farmville and Fat Cat.

With 794 million Facebook users and another 240 million players of Zynga’s games, we can’t begin to imagine how much data just these two social outlets collect on each one of us every day. Multiply that by all the other information trails we leave behind through smartphone apps, Google or visiting other websites and you start to understand why analysis of Big Data is creating so much tech career buzz.

Companies that research tech markets put Big Data near the top of their lists for careers that will need a huge influx of professionals trained in collecting and analyzing the trillions of data gigabits we create each year.

A recent news release from International Data Corporation (IDC) estimated the market for Big Data technology and services will grow from $3.2 billion in 2010 to $16.9 billion in 2015. That’s an annual growth rate of 40 percent which is seven times the growth for the whole information and communications technology (ICT) market! Already there’s a shortage of trained Big Data technology experts in addition to a shortage of analytics experts, the IDC release said.

The most obvious use of Big Data is on the consumer side, collecting information on our every online move so marketers can grab out attention and get us to buy their products. Those ads on Facebook and Google that match keywords to advertisers pop up through automated software programs. But Big Data analysis careers extend way past consumerism to saving the environment, mining medical research to find new cures or treatments as well as analyzing data to solve crimes and catch terrorists before they strike.

Crunching massive amounts of data often needs super powerful computing. Last week (April 10), seven Ontario universities joined a new $210 million super computer initiative funded by IBM as well as the federal and provincial government. According to a Globe and Mail story, “The initiative is designed to expand Canada’s existing high-performance computing infrastructure, which helps researchers crunch massive amounts of data that can be used to solve major challenges from urban infrastructure to health care, water and energy use.”

Even U.S. President Barack Obama thinks Big Data is big deal. Last month, his administration announced $200 million for the Big Data Research and Development Initiative, saying in a news release it would “ transform our ability to use Big Data for scientific discovery, environmental and biomedical research, education and national security.” For example, the U.S. Geological Survey is part of the initiative. It will use Big Data to research such issues as how species respond to climate change and track earthquake statistics to see if there are patterns that’ll help predict when they are most likely to happen next.

All kinds of new companies are being created just to help business, government, health care and every other sector collect the most relevant Big Data then analyze it for statistics and meaningful patterns.

You can even compete for prize money in data contests on a website called Kaggle where anyone can enter to solve real-world problems that business or government submit. Its users come from 100 countries and 200 universities around the world. Kaggle says it’s “making data science into a sport… Whether you're a world-class algorithm wizard competing for prize money or a novice looking to learn from the best, here's your chance to jump in and geek out, for fame, fortune, or fun.”

If you’re the type of person who sees patterns that others don’t and would love to discover meaningful new trends or solid statistics from mining data, this career is for you. Read CareerMash’s Real People profile of Sylvia Summers to learn how she uses data in her job as Manager of Analytics at Bell Business Markets, Bell Canada.