Beyond the skill shortage: broadening access to data, and not just data

As news goes, the data scientist shortage isn’t really news. The rapid acceleration of data supply (and demand) has really strained the available pipeline of capable technologists all along the data supply chain, from statisticians to developers to data wranglers to you-name-it. We need to look beyond the skill shortage.

We’re fond of pointing out that the convergence of data supply and data demand should create more opportunity – but that there’s a legacy stack of tools and technologies in the way. Our solution is aimed squarely at lowering the barriers to access and utilization of Big Data and the opportunities it creates.

But when it comes to talent and opportunities, there’s a different supply-and-demand gap.

Look around (not least if you’re with us at the Hadoop Summit this week); you’ll see there is a huge shortage of women in tech. An MIT study out this month shows that only 20% of CS degrees are going to women, and 13% of engineers are women. The fact is that our industry faces a huge labor shortage, and we’re only drawing from half the pool. Whatever the explanation, the growth in technology – and along with it, the growth in data as a resource – hasn’t lowered the barriers to access for women to make the most of the opportunity.

That’s one of the reasons why Arcadia Data is sponsoring Night of the Living Data, a comedy benefit in San Jose, CA at the Hadoop Summit on behalf of Girls Who Code. This innovative non-profit takes the long view on closing the gender gap, and growing access to tech careers for girls. Their programs educate, equip, and inspire girls with the computing skills they’ll need to pursue 21st century opportunities. The problem is complex, but it starts with some necessary changes to basic perceptions of ‘how things have always worked.’