In his academic and professional life, he followed in the footsteps of his father, Robert Patterson, who was also a professor at the University of Pennsylvania, and who served as director of the US Mint from 1805 to 1824. His mother was Amy Hunter Ewing and he was one of eight children of that marriage.[3] The younger Patterson was elected a member of the American Philosophical Society[1] in 1809 and eventually became its vice-president[4] (his father was elected in 1783) and (again, like his father) was interested mainly in astronomy and geophysics. He was asked by a committee of the APS in 1836 to write a brief report on recommendations for astronomical and physics observations to be carried out by the United States Exploring Expedition, or US Ex Ex, which sailed in 1838.

Patterson was interested in ciphers and regularly exchanged coded correspondence with Thomas Jefferson. One of Patterson's ciphers included in a December 19, 1801[5] dated letter to Jefferson was decoded in 2007 by Dr. Lawren Smithline.[1][6] The cipher consists of 7 digit pairs and is decoded by decrypting 7 blocks at a time.[7] The cipher was of the Declaration of Independence, of which Jefferson was the primary author.[1] Patterson called it his "perfect cipher" and Jefferson considered adopting it for government use.[1]

Frederik Nebeker, Astronomy and the Geophysical Tradition in the United States in the Nineteenth Century: A Guide to Manuscript Sources in the Library of the American Philosophical Society, APS Publication No. 16 (Philadelphia, 1991), p. 75–76.