The Illustris Explorer

This interface is an experiment in the exploration, visualization, and dissemination of
massive data sets -- in particular, those generated by large, astrophysical simulations
such as Illustris. With total data volume of the order ~0.5 PB, all three of these tasks
become increasingly difficult. A powerful approach is thin-client interaction with derived
data products (e.g. group catalogs, merger trees) with a relational database, combined with
on-demand, server side data processing of the raw data (e.g. full snapshots). Leveraging
many modern web technologies we can build this interface within the browser, separating the
user from the details of the data storage and processing, and making access instant and
platform agnostic.

Methods:

A full box slice of the simulation is shown in projection, with a depth of
15 Mpc/h, revealing a fifth of the total volume of Illustris at z=0. All the
imagery is pre-computed and saved as hierarchical image pyramids. Fast data search is
handled by a RDBMS, structured around the group catalogs. The results allow for
near instant data extraction from the entire dataset, which can then be visualized
within the browser or returned in native (HDF5) format.

The image layers are 2^17 pixels on a side, for a total of ~16 Gigapixels each. Gas
projections are ray traced through the volume with an adaptive stepsize, sampling
the N-nearest neighbors along each ray segment according to an inverse distance
weighting type scheme. X-ray luminosity is proportional to n^2 * sqrt(T). The SZ y-parameter
is proportional to n_e * T. Dark matter annihilation is proportional to rho_DM^2. The DM
density layer splats each dark matter particle with
a SPH-type kernel with radius scaling with the local DM point density. The stellar
composite layer splats individual star particles with a Gaussian kernel, with FWHM scaling
with the local baryon particle density. Stellar luminosities are calculated in several
Johnson/SDSS filters following BC03, and the RGB composite is formed from the (r,g,B) bands.