SEPTEMBER 21, 2017

Due to scheduling issues for the presenter, the webinar titled "Software Engineerng" has been rescheduled from Septermber 27 to November 29. We apologize for any inconvenience this might have caused you.

A new webinar track has been added, called "Workforce and Inclusion" and the first webinar in this new series is titled "Opportunities and challenges: diversifying your workforce." This webinar will be presented by Toni Collis, Co-Founder of Women in HPC, Applications Consultant in HPC Research & Industry for EPCC at the University of Edinburgh.

May 25, 2017

The webinar titled "SPIRAL FFT" has been canceled and will not be rescheduled. We apologize for any inconvenience this might have caused you.

MAY 2, 2017

Due to unexpected scheduling conflicts the "High-performance workflows with Swift/T" webinar has been canceled and is being rescheduled for Fall of this year. Those of you who registered for the webinar will be informed of the new date in a separate email. We apologize for any inconvenience this might have caused you. Please send any questions to bw-eot+webinars@ncsa.illinois.edu

April 18, 2017

Swift/T webinar

Swift/T is a workflow system that enables users to combine traditional workflow features like the execution of external programs with calls to in memory functions. Swift/T runs as an MPI program managed by a scalable load balancer, allowing it to handle trillions of tasks on large supercomputers. It offers a concise high-level language for describing data dependencies and iterations. It also allows workflows to call into embedded scripting interpreters, such as Python, R, and JVM scripting languages, allowing scripts and systems (e.g., Numpy) in these languages to be applied at large scale. In this webinar, participants will learn how to use Swift/T with Python functions on Blue Waters.

HDF5 Release Cycle and New Features webinar

The HDF Group will give an overview of the HDF5 development and release cycle in general and the new features in the current family of HDF5 releases. The HDF Group will describe a new storage layout called Virtual Dataset Layout, which allows one to access data stored in multiple HDF5 datasets across HDF5 files as a single (logical) HDF5 dataset. We will present a new way of reading data while it is being written to an HDF5 file ("Single Writer/ Multiple Reader" or SWMR feature). Both features were released in HDF5 version 1.10.0 in March 2016.

Applications' memory footprint and efficient I/O is the focus of the imminent HDF5 1.10.1 release (April 2017). We will explain how one can reduce application memory usage by taking control of the HDF5 metadata cache ("evict on close" feature) and how to accelerate I/O for applications that use HDF5 files as restart files by invoking the "cache image" feature.

Small-sized and random I/O accesses cause poor performance on many HPC systems. HDF5 release 1.10.1 introduces a new file space management strategy ("paged aggregation" feature) and an additional caching layer ("page buffering" feature) to mitigate the problem. If configured, the HDF5 library aggregates small metadata and raw data allocations into constant-sized well-aligned pages, which are suitable for page caching. Each page in memory corresponds to a page allocated in the file. Access to the file system is then performed on a single page or multiple of pages if they are contiguous. This ensures that small-sized accesses to the file system are avoided.

We will also give an overview of new features beyond the HDF5 1.10.1 release, including the forthcoming support for parallel compression and other I/O optimizations, which will be included in future releases.