Probably this has been answered somewhere else already, but I could not find it.

Right now, the BigDataViewer h5/xml file format is great for data viewing, however I think one cannot do any computation on such a data set using standard IJ functionality, isn’t it? If that’s the case then using virtual stacks, e.g., via Bioformats Importer, currently seems the only way for dealing with huge data sets, or? In principle that’s actually OK, but I think one cannot import a BDV file as a virtual stack, can one?

What’s the perspective along those lines? Will imglib2 have an own file format or is the idea to use the BDV format (with lazy loading) for actual analysis? Or is it maybe already somehow possible to compute on BDV files? In fact, one quick fix right now could be to have a Bioformats virtual stack importer for BDV files, or?

In Fiji there is File > Import > BigDataViewer… which allows you to open individual views form a BDV dataset (you need to pick the timepoint, source, and resolution level). If desired it opens as a virtual stack. It is macro-recordable (at least it was, last time I checked), so if you want loop over timepoints etc, that should be possible.

Something more advanced, like opening everything in one virtual stack with time and channel dimensions (where channel dimension flattens angle, channel, illumination direction, …) should be possible in certain situations. It does require some coding, but in principle it should be easy. It will not work in general because every view in a BDV may have different dimensions. In practice, at least is likely that the dimensions of an angle, tile, etc don’t change over time, so maybe virtual stack of one source over time would be something useful. Depends on what you want to do exactly.

What’s the perspective along those lines? Will imglib2 have an own file format or is the idea to use the BDV format (with lazy loading) for actual analysis?

With SCIFIO, you can open files in any supported format (including Bio-Formats formats) as a “SCIFIO cell image” which reads planes on demand. This is an ImgLib2 data structure, so any ImgLib2 algorithm will work on it. And the data is paged to and from disk on demand, so changes are preserved, unlike ImageJ 1.x virtual stacks out of the box.

The ImageJ Legacy layer transparently translates between ImageJ1 and ImageJ2 (i.e., ImgLib2) data structures. The goal is to allow most existing IJ1 algorithms to operate on ImageJ2 data in this way. (There will always be IJ1 algorithms which cannot work with the more flexible ImgLib2 structures—e.g., ones that directly manipulate pixel arrays. But many IJ1 algorithms do not need to do this.)

Both ImageJ Legacy and SCIFIO still need additional development to become more robust. But this is the direction I want things to go.

My idea/preference here is that people can keep using their normal file formats. That said, if a custom file format helps for performance reasons here, we can certainly think about supporting it.

Thanks! I did not know about [File>Import>BigDataViewer…] for individual views this should already help somewhat. For “simple” xyzct files it would be nice though to be able to import the whole thing as a virtual hyperstack as well.

For “simple” xyzct files it would be nice though to be able to import the whole thing as a virtual hyperstack as well.

I’m reluctant to make a Fiji plugin for this, because it will not work in general for the reasons mentioned. But if you only need this for some datasets where you know they have matching dimensions, it is not difficult to do this in a few lines of java.

(Actually, for most BDV datasets, the SetupImgLoader will be a BasicMultiResolutionSetupImgLoader. This has an explicit getImage( final int timepointId, final int level, ImgLoaderHint... hints ) method, where you can specify the resolution level explicitly. But you still only get one resolution level at a time…)

You can however BdvFunctions.show the spimData directly, and this will then make use of the resolution pyramid.