/CSE Ph.D. Student at GT

Interactive Classification for Deep Learning Interpretation

The modified image (left), originally classified as dock is misclassified as ocean liner when the masts of a couple boats are removed from the original image (right).
The top five classification scores are tabulated underneath each image.

Abstract

We present an interactive system enabling users to manipulate images to explore the robustness and sensitivity of deep learning image classifiers.
Using modern web technologies to run in-browser inference, users can remove image features using inpainting algorithms to obtain new classifications in real time.
This system allows users to compare and contrast what image regions humans and machine learning models use for classification.