This paper considers Peirce's striking remarks about mathematics in a little-known review of Spinoza's Ethics within the larger context of his philosophy of mathematics. It argues that, for Peirce, true mathematical reasoning is always at the vanguard of thought, and resists logical demonstration. Through diagrammatic thought and her pre-theoretical innate faculty of logica utens, the great mathematician is able to see a theorem as true long before the logical apparatus necessary to demonstrate its truth exists. For Peirce, true mathematical thought is in some sense pre-logical, and thus, the logical demonstration of this thought in the form of mathematical proofs is in fact "merely a veil over the living thought."