According to most accounts, Henry Guest was born in Philadelphia in 1727. He was known as an ambitious and civic minded man who was a tanner by trade. He had a tannery at the far end of Livingston Avenue and served as an Alderman for the second Colonial Charter of the city in 1763. Henry Guest was an ardent revolutionary during the war for Independence and existing correspondence between Guest and Thomas Paine and John Adams confirms their close relationships.

Henry Guest had three sons from a second marriage that survived to adulthood, William, Moses and Henry, Jr. All served honorably with the Continental Army during the War of Independence, but it was Captain Moses Guest who won distinction for his capture of the notorious British raider, Lt. Col. John Simcoe. Captain Moses Guest is also credited with recapturing the house from the British after they had been using the home as their headquarters in 1776 and 1777.

Henry Guest died in 1815 and the house was then advertised for sale as "one of the best stone houses in the State of New Jersey," with six handsome rooms and a kitchen adjoining, with a double piazza or porch overlooking the Raritan River, in a respectable neighborhood. It was sold by the Guest family in 1817. Moses Guest, who was living in the house when his father died, moved to French Street with his family when the house was sold.