The limestonequarried in the area was soft and easily carved when first extracted, but became harder as it endured the elements, making it a long-lasting building material. The historic Hubbell House used the stone in 1856, and it was likewise used in 1865 for the Dodge County Courthouse, designed by E. Townsend Mix and now the oldest working courthouse in Minnesota.

Mantorville is named for Peter and Riley Mantor, brothers who came to the settlement in 1853. The Mantorville Historic District, which covers both sides of MN 57 and 5th St., was added as a historic district to the National Register of Historic Places in 1974.