James Knox Park was once a part of the 40 acre complex of the St. Mary's School for Girls, which opened in 1868 and closed in the 1930s. At that time, it was known as St. Mary's Woods
After the school closed, St. Mary's Woods was generally neglected. The Knoxville school district bought the six and one half acres in 1956 as a possible site for a public school. The district abandoned the plan and auctioned the ground in 1961.
The City of Knoxville was interested in the land, but a referendum was needed to buy the property. Mayor Franklin Burgess and his wife purchased the land and held the title until a referendum could be placed on the ballot. The referendum passed and Burgess sold the land to the city for the same amount he had paid for it.

Much work was needed to clear the grounds of scrub trees and brush. Knoxville's civic groups pitched in their labor and have continued to help with the upkeep and improvements of the park over the years. The large picnic shelter was the first park structure erected by the city. Plans were obtained from the State of Illinois and the building is patterned after shelters in Delabar and Jubilee State Parks.

Excess funds from the shelter project were used to excavate the swampy ground of what was once Mirror Lake. The students at St. Mary's School for Girls had used the lake for boating, fishing and skating. Much of the lake had filled in with sediment over the years and is now a pond. The city recently improved the pond by reshaping the banks of the pond, dredging it and installing an aerator fountain.

James Knox Park is named for Knoxville resident James Knox (1807-1876), a prominent lawyer, banker, merchant and civic leader. On Sept. 21, 1963, an estimated crowd of more than 2,500 visited the grand opening of the park. Two 4-H calves were donated to be used for barbeque beef sandwiches and local media reported the food was depleted within four hours.

In 1976, the Knox County Retired Teachers Association moved Newman School to the southwest corner of the park. A smaller pavilion was built in 1996 near the school and the pond. Playground equipment and a skateboard facility have been added for the youth in the community and the park is the site of the city's annual fireworks and community band concert each year on July 4.