Big Data, Small Talent Stocks – The Big Data Talent Shortage and How to Overcome It

Thanks to the rise of Big Data and Data Analytics, Data Scientists and BI Specialists have become some of the most in-demand roles in the world. With such a desperate talent shortage, what can employers do to beat the drought and make their BI units as strong as possible?

Over the
past few years, we’ve seen an intensified focus on Business Intelligence (BI)
within many businesses. During the recession, being data-driven was not as much
of a choice for many companies as it was a necessity. Having the necessary
information to back up each and every business choice could often be the
difference between keeping the company’s head above water, or being in serious
trouble.

Now that
we’re in a period of growth again, we’re seeing more organisations looking to
take this data driven approach. Having a strong Business Intelligence unit is
crucial to implementing this effectively, and succeeding in today’s information
obsessed society. In the age of ‘Big Data’, the swathes of information that are
now relevant to business success are incredibly large. Although that new data
creates new opportunities for businesses to engage with their customers, it also
introduces new complexities and challenges. One of those challenges is selecting
the right tools for your business whilst trying to find the talent to resource
these new needs.

Supply & Demand

Big Data
alone is potentially revolutionary, but without the people to make sense of it
nobody will be able to make use of it, and that potential will remain
unrealised. Since the rise of Big Data has increased the need for people to
service it, the scarcity of those with the ability to engage with it has become
more apparent. This has resulted in a massive level of interest for those who
can fill the new "Data Scientist” role. Salaries have skyrocketed and publications have gone as far
as calling Data Scientist "the sexiest job of the 21st
century.”

However,
the role remains limited to those with an extremely specialist skillset.
Someone who is a great researcher and has a background in statistics, who is
also a good problem solver and is familiar with the technical side of data
science. Finding someone with just one or two of these traits might be
difficult, finding someone with all four is very hard unless you have an
extremely attractive business. Whilst perfect candidates do exist, they are
very rare and inevitably know their value, which therefore makes them very
selective about opportunities. In reality, most organisations are unlikely to
be able to attract these candidates, and need to consider looking further
afield.

Data
Science is such a new discipline that specialist courses for it are only just
starting to be rolled out, but help is on the way. Massey University is now offering a degree in Data
Science, but they are the only University in Australasia to currently offer
this. Although this is a huge step forward, there is no immediate solution to
the current shortage. However, that doesn’t mean that there will suddenly be
less demand for Data Scientists or BI professionals. Businesses are still
pushing forward with Big Data, especially in customer-driven industries like
retail and banking, and the public sector. This means that a strategic,
long-term approach needs to be taken when it comes to hiring. So with this in
mind, what can you do about the current situation?

Contingency Plans

At the
moment, most of the experienced BI specialists are based overseas as they are
exposed to big multi nationals and therefore exposed to more advanced
technologies and tools. There is always the possibility of hiring from
overseas, from other markets with more experienced BI candidates. However, this
can be expensive and troublesome, and there is also no guarantee that overseas
candidates will fit into your company culture. If you do pursue this route, for
instance by engaging a specialist recruitment agency, it does get you the
person with the best skills and experience as soon as possible. You can also
engage contractors or other contingent labour to fill your immediate need, however
this opens up questions about long term planning and, again, comes at a cost.

Another
option is to take someone from a different industry who is already applying the
necessary skills in a different context. This could be an individual with a
non-technical background, such the financial or banking sectors, who would be
able to run the same analytical, trend-aware, eye over Big Data. With the
dangling carrot of such a significant paycheque, moving into this kind of role
could be a tempting change of pace for many professionals in analysis based
roles in other industries.

You can
also take these people on and then surround them with a team of skilled
technicians. Data Science, especially when it comes to Big Data in these large,
complex organisations, is a function that is driven by business and made
possible by technology. This means that a balance is required in terms of the
expertise required of each team. If you already have a good team of Developers,
getting someone with strong commercial experience and business acumen to join
that technical team can be beneficial for both parties, and leave you with a
well-balanced Business Intelligence unit.

Advice for Candidates

For candidates, whether they’re on the business or
intelligence side of Business Intelligence, there is some general advice that
can help you become a more attractive candidate for a role dealing with Big
Data. Firstly, try to move into a large and complex organisation. Regardless of
if it’s public or private sector, having experience within a certain size of
organisation (and consequently having experience with a certain volume of data)
will help you eventually move into that Big Data BI role.

Whether you’re on the mathematics, computing or business
side of BI, make sure that you open yourself up to as many opportunities as
possible and get a variety of experience. As with our advice to employers,
being versatile enough to have an understanding of the technical side if you’re
mostly business focused, or the business drivers if you’re mostly technical,
can be invaluable. Experience across a
variety of tools will also take you to the forefront.

Summary

Business Intelligence has been around for a few years now but skills like Data
Analytics and Big Data are trending and aren’t going to be going away. They
won’t become less important. Although there is currently a shortage of skilled
candidates for those highly sought after roles, this puts the onus on companies
to think ahead, start identifying talent needs ahead of the curve and scale
right for their business. Make the right decisions in terms of the tools, and
the right decisions in terms of your people, and do it ahead of time. If you
need specialist help when recruiting for your Business Intelligence unit, feel
free to get in touch
with me or one of the other experts at OCG.