In our project we have developed new visualization technology for ultrasound imaging.
The biggest achievements are 3D high-quality ultrasound visualization, real-time image based ultrasound tracking, ultrasound feature segmentation and clipping using B-Mode and CEUS imaging, ultrasound smoothing filter, visual guidance in ultrasound sequences, and Couinaud segmentation overlay in live ultrasound stream. These results have been attractive for Norwegian and international ultrasound industries, who are starting two spin-off research projects through the BIA funding program based on the research done in the context of the MedViz project P11 - IllustraSound.
The project has further resulted into 2 finished PhDs, 1 PhD is finishing in Q1 2013. So far 16 research papers have been published at various prestigeous schientific venues, two popular-science reports were published one survey paper has been written to consolidate the research work in ultrasound visualization, and one patent was filed. We have gained a reputation of an expert group on ultrasound visualization worldwide and recently we have submitted collection of our work to Eurographics Visual Computing Medical Prize.
The research project is of interdisciplinary character: medicine researchers work together with technologists in imaging and visualization, and additional competence has been obtained by expert advisory in scientific illustration. The project has also various partners along the value creation chain: research partners, innovation partners, company partners, and clinical partners as the users of proposed technology. Throughout the project we have cooperated with various international partners from University of Magdeburg, Vienna University of Technology, n22 Competence Center, Austria, Technical University Munich.
The Helse Vest funding was primarily used for financing implementation of medical software prototypes, communication between ultrasound scanner and the medical workstation, research travels, research visits, Forskningsdagene in Bergen installations, clinical personnel, and scanner rental.
We have managed to establish a cooperation within two BIA projects with GE Vingmed Ultrasound which is an important medical company. And also we have managed to file a patent (DOFI).

2011

Paradigm Shift in 4D Ultrasound VisualizationIn the last years, all the major ultrasound vendors introduced to the markets new 4D (volumetric) transducers, targeting primarily at cardiac examinations. The next natural step is to deliver 4D technology for abdominal examinations and vendors are intensively competing to present the best possible 4D imaging quality with new abdominal transducers.The IllustraSound research project carried-out at the University of Bergen, Haukeland University Hospital and Christian Michelsen Research envisions entirely new utilization of 4D ultrasound in the abdominal context. In particular, the project aims at technology that would ultimately replace computed tomography (CT) used nowadays for liver examinations for treatment planning, and intraoperative guidance. With CT it is possible to extract the liver from imaging scans, compute its volume and segment the organ into separate blood-filtering compartments and demonstrate hepatic pathologies. However, CT is associated with ionizing radiation which may seriously harm the patient.
In the course of our project, so far we proposed parts of complementary technology utilizing traditional 2D ultrasound with magnetic position sensors. Although this technology looks promising, it is clear that it will never outperform CT in terms of precision. The reason is simple, from series of 2D scans, a precise 3D reconstruction is difficult to achieve when considering all the voluntary and involuntary patient’s movements such as respiratory movement.
In the last year IllustraSound project researchers have started systematic investigation to propose new visualization technology based on 4D ultrasound examination of the liver. The first exciting preliminary result is that we can construct a high-precision liver scan from several overlapping small-area liver sectors. These sectors are stitched together to constitute the entire organ. This stitching is known as 3D image registration. The requirement is that it has to be performed instantaneously on the fly. We boosted the performance of this computationally-intensive task by utilizing the latest 3D gaming graphics hardware and we currently achieve registration of two sectors per second. This is sufficient for our purposes and in future gaming hardware will become faster and allow for higher frame rates. It even seems that the additional positional sensors, necessary for previous 2D stitching techniques, can be removed and the sector stitching can be done entirely based on overlapping information between 3D ultrasound sectors. Preliminary results show that the instantaneous sector stitching delivers high precision. This needs to be clinically validated first. If it proves successful, we can proceed to extraction of the liver parenchyma, vascular structures, and the pathologies from the ultrasound volume. Here we plan to use sophisticated methods that from few sketches, done by the examiner, algorithms will complete the extraction of 3D anatomical information. Afterwards, on the top of anatomy, we can incorporate relevant physiological information and trace respiratory movement and deformation of the organ. To reach our goal, we are still on a long way, but the initial proof of concept seems that we are on the right path.

2010

IllustraSound Project Results for 2010The project started mid of September 2009. Our first initiative was an ultrasound installation at Forskningsdagene 2009 [1,2], which was very enjoyed by participants. Then we designed our first subproject to a) research new visualization technologies for ultrasound, b) translational research of existing basic research approaches.In the first part mainly the PI and the PhDs were contributing, together with PostDoc, and our clinical partners. Already in 2009, some illustrative approaches were summarized in a book [3] to which the PI was contributing. In the same year we have developed new illumination models for fast high-quality 3D ultrasound rendering. This has been very very much appreciated by GE Vingmed Ultrasound, when they recently visited us, and has indicated closer cooperation in future with this important industrial partner. This work resulted in publication on the biggest European publication venue in visualization, EuroVis 2010 [4]. We have been working on localization of structures in 3D which resulted into publication at the EG VCBM conference, an important venue for visual computing in medicine [5]. Novel illustrative compositing technique, also useful for ultrasound data, has been published in a C&G journal [6]. The last publication this year will be on automated segmentation of vascular structures from tracked ultrasound data [7]. So in general on the research side, very good success.
The second line of research was focusing on developing an application for Couinaud liver segmentation guidance in the ultrasound examination. Here a novel single-modality approach has been developed, which looks very promising. The only missing element is tracking of patient motions. This activity will be in the focus in the Q1 2011 and then the application will be subject of clinical testing.
The ultrasound data is very difficult to extract from the acquisition console, especially during the live examination. We have therefore discussed with GE Vingmed Ultrasound the possibility of having the data streamed from the ultrasound over the network. This feature is very important for development of real-time interaction methods. We have now a working framework which allows the real-time data streaming. This important enabling step has been achieved by the PostDoc in the team and the CMR team and we are very pleased by the increasing interest in the research from the side of the Norwegian medical industry.
We are disseminating the results of our research on various scientific conferences and universities. IllustraSound related talks were in the first project period held on EuroVis conference in Bordeaux (FR), VCBM conference in Leipzig (GE), VMV conference in Siegen (GE), as invited talk at the Vienna University of Technology (AT), at Technical University of Montreal (CA), Meeting of Norwegian Society of Ultrasound in Lillehammer and at the recent meeting of Society for Medical Imaging Technology in Trondheim.
[1] Forskningsdagene 2009 IllustraSound webpage: http://www.ii.uib.no/vis/events/FD09/
[2] Forskningsdagene 2009 webpage, http://www.forskningsdagenebergen.com/?page_id=1421
[3] Information Theory Tools for Computer Graphics, M. Sbert, M. Feixas, J. Rigau, M. Chover, I.
Viola, Morgan and Claypool Publishers Colorado, 2009
[4] A Multidirectional Occlusion Shading Model for Direct Volume Rendering, V. ?oltészová, D.
Patel, S. Bruckner, I. Viola, Computer Graphics Forum 29(3):883-891, 2010
[5] Guided Visualization of Ultrasound Image Sequences, P. Angelelli, I. Viola, K. Nylund, O. H.
Gilja, H. Hauser In Proceedings of Eurographics Workshop on Visual Computing for Biology and
Medicine, 2010
[6] Hybrid Visibility Compositing and Masking for Illustrative Rendering, S. Bruckner, P. Rautek, I.
Viola, M. Roberts, M. C. Sousa, M. E. Gröller, Computers & Graphics - Special Issue on Illustrative
Visualization, 34:361-369
[7] Ultrasound Painting of Vascular Tree, Å. Birkeland, I. Viola, In Proceedings of Vision, Modeling
and Visualization 2010