Big data analysis from online vacancies

Information on skills demanded from job-seekers by employers is very useful to inform career and continuing VET decisions of individuals. In the very dynamic labour market of today, understanding employers’ requirements is also crucial for employment services and guidance counsellors to assess individual training needs, as well as for training providers to revise curricula and training programmes and tune them to current labour market demand. Policy-makers also feel the need to obtain up-to-date information on skill needs, new jobs being created as well as emerging skills.

Although useful and relevant, comprehensive data on skills demand by employers across the EU does not exist. Methodological complexity and high costs are key obstacles to an EU-wide survey of employers on skill needs. In addition, survey data suffer from significant time lags (up to two years) between data collection and final results. Given the high and increasing dynamic of labour markets reducing time between data collection and the use of the data is critical.

Over the last decade, the usage of online platforms for posting job vacancies significantly increased, including for advertising jobs of different types and levels. The Internet has become a rich source of ‘real-time’ labour market information, which is now accessible for analysis thanks to technological advances in big data collection and analysis.

A feasibility study carried out by Cedefop has confirmed that a pan-EU analytical tool for collecting data on skills demand by employers using online job postings can be set up. The costs of expanding Cedefop’s system to the whole EU are proven to be much lower than the expected costs of alternative methods of collecting similar data from employers, while the data collected appears to provide significant value-added. Cedefop will therefore further develop the current prototype in order to collect data for the whole EU. The data collected within the feasibility study allow for identifying skills and job requirements typically requested across occupations as well as new and emerging jobs and skills. The results from fully fledged system will therefore fill an important gap in the EU evidence on skill needs, i.e. the skills demand of employers.

The project by Cedefop to develop an EU-wide system started in 2017. First results will be released end of 2018 and a fully-fledged and validated dataset for all EU Member States will be available in 2020. The project will benefit from collaboration with other EU institutions, in particular DG Employment and Social Affairs, DG Connect, Eurostat’s big data task force and network of European statistical systems (ESS – net). Some EC initiatives (such as ESCO, EURES) will strengthen the usability of the pan-EU tool for vacancy scrapping and analysis, while other initiatives – the new Europass and the Blueprint for sectoral collaboration - will benefit from the data collected. Joining forces between institutions will produce synergies, reduce costs and increase the value-added of the data.