The proper analysis of complex and multi-modal imaging requires careful consideration of the nature and properties of the collected data. Often techniques that provide no information-theoretic guarantees as to their optimality in describing the given data set and are prone to overfitting are adopted. The analysis of spatially resolved discrete orientation measurements, as produced by EBSD, serves as a ubiquitous example. The estimation of orientation distributions via generalized spherical harmonic expansion relies on ad hoc methods for choosing parameters and enforcing constraints. Here we present an unsupervised learning approach for the estimation of orientation distributions as a finite mixture of Bingham distributions. The Bingham distribution is the maximum entropy distribution for the rotation group. The algorithm introduces a minimum message length criterion, to balance data likelihood against model complexity. This approach leads to ODFs which are less likely to overfit the data, eliminating the need for a priori parameter choices.