This paper is concerned with the problem of computing a discrete-coefficient approximation to a digital filter. In contrast to earlier works that have approached this problem using standard combinatorial optimization tools, we take a geometric approach. We define a Riemannian manifold, arising from the difference in frequency response between the two systems of interest, on which we design efficient algorithms for sampling and approximation. This additional structure enables us to tame the computational complexity of the native combinatorial optimization problem. We illustrate the benefits of this approach with design examples involving IIR and FIR filters.