Search form

Courting Big Data Talent

Posted on Mon, Apr 01, 2013 by DC Denison

When does it make sense to start up a Big Data program? If your email marketing system isn't talking to your sales force automation system, and neither is synched up with your online purchase system, are you really ready to tackle a Big Data project? The answer may surprise you as we examine Big Data and its impact on the next generation digital experience in this fifth installment of our ongoing series "Are You Ready for Big Data?"

What skills do you need to implement a Big Data project? The field you want to explore is called "data science."

"Data scientist" is a relatively new job title, but thousands already have it on their business cards (700 at Google alone). Yet because the field is so new - university programs are rare - Big Data professionals are hard to find. McKinsey & Co. predicts that by 2018, the U.S. could face a shortage of more than 1.5 million specialists needed to capture, store, manage and analyze Big Data.

In Fall 2012, Thomas Davenport, visiting professor at Harvard Business School, and D.J. Patil, data scientist in residence at the venture capital firm Greylock Partners, wrote a cover story for the Harvard Business Review that outlined strategies for staffing Big Data projects.

Tom Davenport, Harvard Business School

One approach they recommended: Grow your own. Recruit and develop Big Data talent in house, or look for achievers in any field with a strong data and computational focus and grow with them. Experimental physics and systems biology, for example, are two fields that could generate promising data scientists, according to Davenport and Patil.

But Davenport and Patil warn that the search won't be easy.

What makes it particularly difficult, Davenport says, is that the best data scientists need a variety of technical, business, analytical and relationship skills.

According to Davenport, the best data scientists often have advanced computer science degrees, or advanced degrees in fields such as physics, biology or social sciences that require a lot of computer work. In addition they have to be familiar with a wide variety of disciplines such as Hadoop, MapReduce and related tools, programming languages like Python, and disciplines like natural language processing.

Also: "Nothing beats experience," adds Shawndra Hill, who works with and teaches about Big Data in the Operations and Information Management Department at The Wharton School of the University of Pennsylvania. She says that the best data scientists have loved data for a long time and have gained an intuition about what can and can’t be done. They also have a creative eye to think about how to use new data to solve old problems, and old data to solve new problems.

Shawndra Hill,The Wharton School, University of Pennsylvannia

"Paths to data science usually start with an interest in solving hard problems," Hill says. "The rest of the path is lined with exciting hard problems that have been solved successfully over time. The speed of computing makes so much more possible."

In addition to these technical skills, data scientists also need the attributes previously necessary for analytical professionals, including mathematical and statistical skills, business acumen, and the ability to communicate effectively with customers, product managers and decision makers.

The skills are so varied, Davenport reported, that some companies have decided to create data science teams that together embody this collection of skills.

The yearly salary for data scientists, according the online career site Glassdoor, ranges from $80K to $220K.

One encouraging sign for companies in search of expertise: many of the hottest, most lavishly funded start-ups in the Big Data arena are working on products that mix analytics with Big Data, often in a cloud-based service. Ultimately these products could lighten the load for companies hoping to get a Big Data project off the ground.

Until then, Davenport says, the goal is to find data talent that is "a hybrid of data hacker, analyst, communicator, and trusted adviser."

That combination, he admits, is "extremely powerful—and rare."

Editor’s Note: This is the fifth post in the ongoing series “Are You Ready for Big Data?” by DC Denison. Download the complete "Are You Ready for Big Data" ebook to learn more about Big Data, its applications in creating the next generatlon digital experience, and what it takes to get into the game.

Plain text

Filtered HTML

Use [acphone_sales], [acphone_sales_text], [acphone_support],
[acphone_international], [acphone_devcloud], [acphone_extra1] and
[acphone_extra2] as placeholders for Acquia phone numbers. Add class
"acquia-phones-link" to wrapper element to make number a link.

To post pieces of code, surround them with <code>...</code> tags. For PHP code, you can use <?php ... ?>, which will also colour it based on syntax.