We propose a purely geometric approach to facial attribute recognition which has better cross-modal performance than a state-of-the-art appearance-based method. While labeled color imagery is plentiful for facial attribute learning, labeled imagery in other modalities such as infrared is comparatively rare. Because face appearance is significantly altered in infrared imagery, standard attribute recognition methods trained on color imagery may not transfer well. To address this problem, we propose attribute recognition based purely on geometric information, i.e. geometric relationships derived from a facial landmark detector. We show that our method outperforms a state-of-the-art appearance-based method in attribute recognition when both are trained on color images and tested on infrared images.