DataSpaces

An Extreme Scale Data Management Framework

What is DataSpaces?

DataSpaces is a programming system targeted at current large-scale systems and designed to support dynamic interaction and coordination patterns between scientific applications. DataSpaces essentially provides a semantically specialized shared-space abstraction using a set of staging nodes. This abstraction derives from the tuple-space model and can be associatively accessed by the interacting applications of a simulation workflow. DataSpaces also provides services including distributed in-memory associative object store, scalable messaging, as well as runtime mapping and scheduling of online data analysis operations.

DataSpaces is currently being used by production coupled scientific simulation workflow on large-scale supercomputers. For example, as part of the coupled fusion simulation workflow framework, DataSpaces enables memory-to-memory coupling between the gyrokinetic PIC edge simulation code XGC0, and the MHD code M3D-OMP. Similarly, as part of turbulent combustion workflow DataSpaces enables data coupling between the direct numerical simulations (DNS) code S3D and the data analytics pipeline. DataSpaces has been integrated with and deployed as part of the Adaptive IO System (ADIOS) framework distributed by Oak Ridge National Laboratories (http://www.olcf.ornl.gov/center-projects/adios/). ADIOS is an open source I/O middleware package that has been shown to scale to hundreds of thousands of cores and is being used by a very wide range of applications.

Here is a video of DataSpaces demo with ADIOS/Paraview/PIxel3D

Recent News

June 2016: DataSpaces 1.6.1 Release!

Dataspaces 1.6.1 has been released with new network support and various bug fixes. It is available in the Download section.

July 2015: DataSpaces tutorial at XSEDE'15!

We gave a DataSpaces tutorial at XSEDE'15 conference in St. Louis, MO - July 2015. The slides are available here (part1,part2)

July 2015: DataSpaces 1.6.0 Release!

Dataspaces 1.6.0 has been released with new features and various bug fixes. It is available in the Download section.

January 2015: DataSpaces 1.5.0 Release!

Dataspaces 1.5.0 has been released with various bug fixes. It is available in the Download section.

June 13th, 2014: DataSpaces 1.4.0 Release!

Dataspaces 1.4.0 has been released with new features and various bug fixes (along with a DataSpaces logo). It is available in the Download section.

His poster titled "Runtime Support for Dynamic Online Data Management and Insight Discovery in Large Scale Coupled Simulation Workflow" highlights his research work in HPC arena. You can see the poster here.

This work is supported by the National Science Foundation and the Deparment of Energy.