ADIOS grew out of a 2008 collaboration between the Oak Ridge Leadership Computing Facility (OLCF) and researchers from academia, industry, and national laboratories. Their goal was a system to get information in and out of a supercomputer efficiently.

“The measurement of success for us has always been ‘what percentage of time do you spend in I/O?’” said OLCF computational scientist Scott Klasky. ADIOS was inspired by Klasky’s work at the Princeton Plasma Physics Laboratory, where he noticed up to 30 percent of researchers’ computational time was spent reading and writing analysis files.

The open-source middleware is designed to help researchers maximize their allocations on leadership-class computing resources from wherever they may be. In essence, it creates more time for research by minimizing the time needed to read and write data to files, even if researchers are sending those files from thousands of miles away.

The previous release—version 1.2—improved usability by allowing users to construct new variables in their simulations while they run, simplifying the interface users work with and improving I/O performance. In fact, researchers using the middleware found the writing process substantially expedited, consuming only .06 percent of their computation time on average.

The biggest challenge for the newest release—version 1.3—was to improve reading efficiency, according to Qing Liu, a scientific researcher working in the Remote Data Analysis and Visualization branch at the National Institute for Computational Sciences and the leading developer of ADIOS.

“Our users were very happy that the writing was greatly improved [with version 1.2]. They could get 30 gigabytes per second for the writing performance, but when they tried to read, the performance was much lower,” he said. “There was a huge gap between writing and reading performance.”

Ray Grout, a researcher at the National Renewable Energy Laboratory, was one of the first people to test the latest update, using the S3D combustion code to study turbulent flows reacting to one another. Grout noted a huge increase in reading performance.

Klasky also added that ADIOS 1.4 will continue grass-root collaboration from computer science researchers to greatly reduce the problem of coping with large data on high-performance machines.