Known for some years even in state records as "Bakersfield alias Knoultons Gore", the town had originally been granted to Luke Knowlton (Knoulton) of Newfane and sold in 1798 or 1799 to Joseph Baker of Westboro, Massachusetts. Baker had acquired title to the entire 10,000 acre town for £500. Several changes have taken place to bring the town to its current size, including the addition of parts of Fairfield and the former Smithfield in 1792, the entirety of Knights Gore in 1795, and Averys Gore in 1963.

While Baker and his family are remembered in the name of the town, it is Knowlton who was one of the state's distinguished early settlers. Born in Shrewsbury, Massachusetts in 1737, he came to Vermont about 1770. Probably because he held land in Newfane under title from New York, his political sentiments seem to have leaned in that direction: just before the Revolution, he and other York-minded people formed the Committee of Safety for Cumberland County (New York's designation for much of southern Vermont) and worked steadily to further his hope that Vermont would elect to be part of New York.

Toward the close of the Revolution, Knowlton was active in conducting the Haldimand negotiations with the British in Canada in an effort to keep British troops out of Vermont. For some time after the war, Knowlton acted as a representative of the Yorkist faction in Vermont, and spent considerable time trying to persuade the Continental Congress that Vermont should not be allowed to become an independent state. He also was active in promoting the so-called "Eastern and Western unions," by which some towns in New Hampshire and New York were for a time joined to Vermont. Eventually Knowlton withdrew his objections to the state's independent status, and from 1784 to 1801 served in either the Legislature or the Governor and Council. It was probably because of his office that his land was chartered fairly soon after it was granted, and why he was the sole grantee of Bakersfield's original 10,000 acres, an unusually large amount of land for one man.

Knowlton was also unusually favored in being one of the few resident Vermonters who received part of the $30,000 settlement that was paid to New York for the cancellation of that state's land claims in Vermont. One of his contemporaries described Knowlton as "a leading character, a man of great ambition and enterprise, of few words, but possessed of the keenest perception . . ." When he tendered his resignation to the Vermont legislature in 1801, it was received with very real regret. He died in Newfane in 1810.

This is a basic geographic reference, intended to show relative location of adjacent towns. Directional accuracy is limited to 16 compass points. There isn't even the slightest suggestion that one can necessarily travel directly from one town to the next (as in "You can't get there from here").