Strategies for reducing the effect of cumulative rounding errors in geometric numerical integration are outlined. The focus is, in particular, on the solution of separable Hamiltonian systems using explicit symplectic integration methods and on solving orthogonal matrix differential systems using projection. Examples are given that demonstrate the advantages of an increment formulation over the standard implementation of conventional integrators. We describe how the aforementioned special purpose integration methods have been set up in a uniform, modular and extensible framework being developed in the problem solving environment Mathematica.