Abstract

In the last years of his life, Leo Breiman promoted random forests for use in classiﬁcation. He
suggested using averaging as a means of obtaining good discrimination rules. The base classiﬁers
used for averaging are simple and randomized, often based on random samples from the data. He
left a few questions unanswered regarding the consistency of such rules. In this paper, we give a
number of theorems that establish the universal consistency of averaging rules. We also show that
some popular classiﬁers, including one suggested by Breiman, are not universally consistent.