PhD Preliminary Research Examination: Nan Cui

DateMay 4, 2017

Start Time1:30 p.m.

LocationGreen Hall, Room 0120

Event Description

Abstract: Near-infrared (NIR) fluorescence image-guided surgery (IGS) is a medical technique that can provide critical
structural and functional tissue or organ information to the surgeon in clinical settings to successfully detect and
differentiate tumor tissue from surrounding healthy tissue. Current state of the art NIR imaging systems have two major
shortcomings: First, they cannot capture NIR fluorescence information under surgical light illumination due to the high
dynamic range requirements. Second, the combined NIR and color information is presented either on a monitor or videodisplay
goggles. The surgeon’s natural vision is replaced with virtual information comprised by the color and NIR sensor.
To address these shortcomings, I have successfully developed a multi-exposure image sensor with optical filters that
capable of capturing both color and NIR image at the same time under surgical light source. However, the result reveals
two major problems: 1) the dynamic range (DR) of the scene in the operating room (easily exceed 120 dB) cannot be
detected and the signal-to-noise ratio (SNR) of low intensity object is lowed, caused by the read-out scheme of traditional
voltage mode active pixel sensor (APS). 2) both the color and NIR fluorescence image are still displayed on a computer
monitor. To address these shortcomings, I propose to investigate the realization of a bio-inspired, multi-spectral imager
using time domain imaging. It will mimic the mechanism of butterfly’s eye and retina, producing uniformly high SNR and
high DR. I will also investigate augmented reality display system that can present the NIR information without blocking
the surgeon’s natural vision using Microsoft Hololens®, which allows the user to see the augmented image on the goggle
transparent display component. One of the challenges is to co-register the NIR fluorescence augmented image with the
actual target. I will develop an optical setup and algorithm to co-register the human natural vision, tumor and the
augmented NIR fluorescence image. Finally, I will demonstrate the use of the bio-inspired CMOS multi-spectral imaging
system and augmented reality display system in a small animal models and in clinical settings.