OTB-Ice asynchronous loading

What changes will be made and why they would make a better Orfeo ToolBox?

This issue is about implementing asynchronous loading of tiles of image-data by the OTB-Ice rendering engine in order to improve responsiveness and interactivity of Monteverdi, such as for Google Maps.

Currently, the OTB-Ice rendering engine divides image-data of each layer into a grid of tiles, thus decreasing the disk-to-RAM loading when scrolling the image-view. Also, it provides a fast move feature which postpones loading of tiles after the scrolling has finished. The OTB-Ice rendering of the image-layer is blocked while new tiles are being loaded.

The idea is to delegate loading of image-data tiles to a background task which signals when each tile is loaded so that the OTB-Ice rendering is not blocked. Loading of tile image-data would start at lowest resolution first, be rendered then be updated to requested resolution and be rendered.

High level description

Loading of image-data tiles for each image-layer is currently done by the ::LoadTile() function of the otb::GlImageActor. It could be delegated to an otb::ImageTileLoader which manages a background loading thread:

the otb:GlImageActor enqueues a tile request into the otb::ImageTileLoader which asynchronously loads the lowest resolution first

The otb::ImageTileLoader calls back the otb::GlImageActor when image-data of the tile has been loaded

The otb::GlImageActor then renders the lowest resolution of image-data of the tile

When all lowest resolution tiles have been processed, the otb::ImageTileLoader automatically starts loading image-data of tile at requested region and calls back the otb::GlImageActor when ready.

Whenever all tiles have been processed, it sleeps.

Whenever the requested resolution has changed, already loaded tiles are kept and the otb::GlImageActor enques a new tile request. When image-data has been loaded, the highest resolution is released, thus freeing memory.

The ::CleanLoadedTiles() function of the otb::GlImageActor would need only minor changes to handle that. So does the ::UnloadTile() function.

The otb::GlImageActor::Tile structure contains tile regions and settings alongside with image-data. It should be split in two separate parts:

the former being used to enqueue tile requests and check tile state

the latter being used for image-data exchange between the otb::GlImageActor and the otb::ImageTileLoader.

The otb::ImageFileReader<> ownership would need to be transfered to the otb::ImageTileLoader.

A cancel tile request should be added to the otb::ImageTileLoader and appropriate driving code in the ::UpdateData() function of the otb::GlImageActor in order to handle changes of resolution and/or scrolling when image-data of tiles are still being loaded.

Note: A flag could be implemented to switch asynchronous loading ON/OFF.

An additional idea is to implement a thread-pool manager from which the otb::ImageTileLoader could borrow a thread resource. The number of thread managed could be limited or not. It the latter case, it is equivalent to the aforementioned implementation except that ownership of thread resources is granted to the thread-pool manager.

Risks and benefits

When importing image-data in Monteverdi, the rendering is blocked until all image-data for the viewport has been loaded. Asynchronous loading could help to reduce this delay since each tile would be rendered as fast as possible without waiting for all tiles to be loaded. Likewise, when scrolling or zooming.

The OTB-Ice renders layer back-to-front in order to handle translucency or view-through effects. So, when image-data of a tile has finished loading, a render call-back will cause the rendering of all the scene to be redone. But, this OpenGL-based hardware-accelerated rendering is faster than disk-to-RAM loading, especially when decompressing image-data.

One otb::ImageTileLoader would be associated to each otb::GlImageActor which means one background thread for each otb::GlImageActor running concurrent disk access (on different image-data files). Usually, the number of image-data layers is low. So, the number of threads is kept low but concurrent disk access could kill the performances and is hardware dependent, which should be benchmarked. Also, a system thread context increases RAM usage.

Note: Loading image-data from different disks would improve parallelization.

The use of a thread-pool manager could be useful to limit number of simultaneous threads, thus limiting resources usage, but will limit parallelization of image-data loading.

Alternatives for implementations

Not Applicable.

Who will be developing the proposed changes?

(TBD) Development of this feature needs good knowledge of the OTB-Ice rendering engine, graphics, OpenGL rendering and Monteverdi.