We compute the characters of the simple
$\text{GL}$
-equivariant holonomic
${\mathcal{D}}$
-modules on the vector spaces of general, symmetric, and skew-symmetric matrices. We realize some of these
${\mathcal{D}}$
-modules explicitly as subquotients in the pole order filtration associated to the
$\text{determinant}/\text{Pfaffian}$
of a generic matrix, and others as local cohomology modules. We give a direct proof of a conjecture of Levasseur in the case of general and skew-symmetric matrices, and provide counterexamples in the case of symmetric matrices. The character calculations are used in subsequent work with Weyman to describe the
${\mathcal{D}}$
-module composition factors of local cohomology modules with determinantal and Pfaffian support.