Imagine surgeons doing an operation in which they see through opaque areas of the body as if they had X-ray eyes. Now imagine that these X-ray eyes can see not only bone, but also soft tissue.

To some extent this happens already when antenatal doctors do an amniotic probe, watching the position of the needle in the womb via an ultrasound scan. But they are normally looking at a monitor. This involves looking up and down - or possibly sideways.

But what if, instead, they could keep their eyes on the patient while seeing a superimposed image of what is going on inside the patient’s body, using augmented reality glasses or visors?

This is precisely what surgeons at Imperial College, London have been experimenting with. Using the Microsoft HoloLens, they have been conducting trials in which surgeons doing reconstructive surgery on lower limbs are able to see the positions of bones and blood vessels via CT scans overlaid against the area that they were operating on.

Leg surgery

The team of surgeons were trying out the new technology at St Mary’s Hospital, which is attached to Imperial College. The most difficult part of reconstructive surgery on limbs is reconnecting blood vessels and sometimes bones. In many cases, the surgeons are effectively working blind, or at least with a highly restricted view. CT scans can give them the extra information they need, but the problem of looking in two directions makes the task difficult. The augmented overlay approach, solves this problem neatly and effectively.​​The method was tested on five patients, including a middle-aged man with leg injuries from a road accident and an 85-year-old woman with compound fractures of the ankle, including a protruding bone that had pierced the skin.​​The CT scans did not take place in real-time while the surgery was being performed. Instead, the patients went through CT scans beforehand to map out the positions of bones and blood vessels, as well as connective tissue, muscle and fat. The CT scans were then segmented, with the bones, blood vessels, muscle and fat converted into polygonal models and rendered as digital images in a format compatible with the HoloLens. These images were then projected onto the HoloLens visor, so that the surgeons could see them as an overlay against the actual injured limb.

CT Scans

The European radiology Experimental journal published the results of the study. In the abstract they started:

Intraoperatively, the models were registered manually to their respective subjects by the operating surgeon using a combination of tracked hand gestures and voice commands; AR was used to aid navigation and accurate dissection. Identification of the subsurface location of vascular perforators through AR overlay was compared to the positions obtained by audible Doppler ultrasound. Through a preliminary HoloLens-assisted case series, the operating surgeon was able to demonstrate precise and efficient localisation of perforating vessels.[https://eurradiolexp.springeropen.com/articles/10.1186/s41747-017-0033-2]