In the late nineteenth century another school, the Graham School, operated in a later building on the same site. That building's exterior was painted red, leading to the common reference to it as “The Red Academy.” Beginning in 1898 the building housed Warrenton High School, initially a private school and after 1923 a public school.

References:
Lizzie Wilson Montgomery, Sketches of Old Warrenton, North Carolina (1924)
Charles L. Coon, comp., North Carolina School and Academies: A Documentary History, 1790-1840 (1915)
Kenneth McFarland, The Architecture of Warren County, North Carolina, 1770s to 1860s (2001)