He attended the U.S. Military Academy for three years and was first in his senior class, but was forced to resign on the sudden deaths of his father & older brother in order to take care of the family.

As a lawyer he was Chief Counsel for the Baltimore and Ohio Railroad. He was an architect in his own right. He was involved with colonization in Liberia, Africa and was President of the American Colonization Society in 1853. He was involved with litigation between Nicholas Roosevelt and Robert Fulton about the vertical wheel on steamboats. He was involved in the incorporation of the Magnetic Telegraph Company and the Western Telegraph Company to conduct and carry the electro-magnetic telegraph, invented by Morse, over lines alongside the B&O Railroad between Baltimore and Washington. He was Counsel for the Ross Winans Railroad interests in Russia. He was President of the Park Board of Baltimore and was involved with the purchase of land for Druid Hill Park. In 1846 he was granted a patent for improvement in stoves. This was known as the "Latrobe Stove". Although over 300,000 were in use, he never got any financial rewards for it. Parts were cast in the foundry and built at the Bartlett-Hayward Company in Baltimore until 1910. It was similar to the "Franklin Stove". He was the first president of the Maryland Historical Society. He was Aide-de-Camp to Gov. Thomas Swann during the Civil War.