With the advent of faster area detectors for imaging, large arrays for spectroscopy and the prospect of high coherent flux from diffraction limited storage rings, planning for future data capture, analysis, collaboration and archival is becoming critical in X-ray microscopy.

International efforts to establish frameworks for Big Data are developing with the PaNdata and PNI-HDRI programs in Europe and the ‘Tao of Fusion’ initiative in the USA, and strategies for handling, sharing, modification during analysis and long-term accessibility and metadata are emerging (e.g. NeXus, Data Exchange).

The Big Data workshop will focus on issues and approaches towards handling the growing data flood facing future X-ray microscopy, including handling the raw data, data formats for collaboration, processing or data reduction, statistical classification or compression, archival and also exploration and discovery within large data-sets. It will have particular emphasis on growing X-ray microscopy issues, such as X-ray tomography, ptychography/CDI and 2D/3D fluorescence data and also include experience, ideas and input from other fields that are dealing with Big Data. The workshop will also encourage and facilitate demonstration of Big-Data software tools by attendees.

Big Data will be discussed around three themes:

Collecting & handling – How to handle the data ‘firehose’ from modern experiments.

Analysis – Exploring and analysing big data, how to get meaning from big data in a timely way.

Research Associate Professor Kevin Vinsen, International Centre for Radio Astronomy Research (ICRAR): Kevin’s focus is for the issues caused by the huge data sets that modern radio and optical astronomy generates. He is working with colleagues from around the world to design the science data pipe for the Square Kilometre Array radio telescope.