Numerous schemes are available for feature selection in a pattern recognition problem, but the feature extraction process is largely intuitive. A sequential feature extraction scheme is proposed for binary features. A decision function, which is linear and near optimal, is developed concurrently with each feature. Performance bounds are developed for several design strategies. Experimental results are given to illustrate the use of the scheme and the effectiveness of the performance bounds.