Robust One-Shot Facial Expression Recognition with Sunglasses

H. Jiang, K. Huang, T. Mu, R. Zhang, T. O. Ting, and C. Wang

Abstract—Automatic facial expression recognition (FER) is both interesting and important in computer vision and machine intelligence. While previous FER systems often focus on learning a classifier in a controlled environment, a more practical and robust scenario is now under consideration. More specifically, traditional FER requires collecting as many as possible facial photos so that they will accurately recognize expressions no matter a particular person wears sunglasses, hats, and other accessories or not. Such requirement is however inconvenient and could impose practical difficulties for users. To alleviate this problem, a robust one-shot FER system that only requires taking one single facial photo for each expression of each user is proposed in this paper. When taking the single photo, the user is free to choose whether to wear sunglasses or not. The sunglasses can even be of different shape and of various luminous transmittance. Such one-shot recognition improves the user-friendliness of the FER system. Importantly, a novel and practical sunglasses detection and recovery approach is developed, which obtains an obvious accuracy improvement of 6.09%, 5.86% and 4.33% with state-of-the-art classifiers including Support Vector Machine (SVM), Linear Discriminate Analysis (LDA) and K-Nearest Neighbors (KNN) respectively on the modified Japanese Female Facial Expression (JAFFE) benchmark database.