Storage-based Convergence between HPC and Cloud to handle Big Data

BigStorage is an European Training Network (ETN) whose main goal is to train future data scientists in order to enable them and us to apply holistic and interdisciplinary approaches for taking advantage of a data-overwhelmed world, which requires HPC and Cloud infrastructures with a redefinition of storage architectures underpinning them – focusing on meeting highly ambitious performance and energy usage objectives.

Nowadays there is a lack of professionals who know how to deal with storage, management and analysis of Big Data. Indeed, there is a gap between infrastructures for dealing with Big Data and applications using these volumes of data. In 2011, the McKinsey Global Institute published a study that found that, by 2018, there could be a shortage of up to 190,000 data scientists in the United States, representing a 50 percent to 60 percent gap between supply and demand. Similarly, European officials estimate that 300,000 data scientists will be needed in Europe in the forthcoming years. Other reports, such as those from PRACE and ETP4HPC, have also emphasized the need of skills in HPC, Cloud, Storage, Energy, or Big Data to maintain Europe’s economy. In this context, a major goal of this project is to bring a substantial contribution to the training process of these future experts.