Abstract

Most existing feature selection methods focus on ranking features
based on an information criterion to select the best K features.
However, several authors find that the optimal feature combinations
do not give the best classification performance. The reason for this
is that although individual features may have limited relevance to
a particular class, when taken in combination with other
features it can be strongly relevant to the class. In this paper, we
derive a new information theoretic criterion that called
multidimensional interaction information (MII) to perform feature
selection and apply it to gender determination. In contrast to
existing feature selection methods, it is sensitive to the relations
between feature combinations and can be used to seek third or even
higher order dependencies between the relevant features. We apply
the method to features delivered by principal geodesic analysis
(PGA) and use a variational EM (VBEM) algorithm to learn a Gaussian
mixture model for on the selected feature subset for gender
determination. We obtain a classification accuracy as high as 95\%
on 2.5D facial needle-maps, demonstrating the effectiveness of our
feature selection methods.