The first house on the site was built by Edmund Cooke between 1635 and 1646 and in 1664 was the fourth largest house in Twickenham.[1]

In 1709 the property was purchased by Sir Godfrey Kneller, court painter to British monarchs from Charles II to George I. He had the original house demolished and replaced by a new building, reputedly designed by Sir Christopher Wren. This second house was known as Whitton Hall, after the village, but was renamed by Kneller's widow, in memory of her husband. In 1757, the house was sold to Sir Samuel Prime, a prominent London lawyer, who, with his son of the same name, extended it significantly and landscaped the grounds. After Samuel Prime junior died in 1813, the hall was sold to Charles Calvert, WhigMP for Southwark from 1812–1832. He further expanded the house (to designs by Philip Hardwick), adding drawing rooms at the east and west ends of the building. Calvert died of cholera in 1832 and his widow inhabited until death about 1845. The house was then acquired by the government as a teacher training college but it needed substantial reconstruction.[1]

Around 1850 much of the Wren bulk of the house was dilapidated and so was demolished. The Hardwick additions formed the wings of the new house, the one still standing today, designed by George Mair. This third building on the site is stone-corniced, casemented and constructed as to its ground floor central range and otherwise built of red bricks, all in the neo-Jacobethan style suitable to 19th and 20th century mansions of its scale. It consists of 17 bays, the central of which is the widest and of stained glass and the two adjacent of which are arrow-slit windows. The bays are split into five recessed or projecting ranges. Two square, pavilion towers form the near left and right bays, each tower crowned by a stone and leaded ogee-shaped cupola. Its gatepiers and house are separately listed by Historic England in the initial, grade II category.[2]

From 1850 to 1856 Kneller Hall served as a teacher training college under principal Frederick Temple and vice-principal Francis Turner Palgrave, known for his poetry anthology Palgrave's Golden Treasury. In exchange for a two-year college education almost entirely subsidised by the state – one of the first forms of state-sponsored social mobility – graduates of the college were supposed to teach pauper and delinquent boys in specially built district schools. Though Kneller Hall itself was generously funded by the state, the larger scheme to build district schools never came to fruition owing to political and religious conflict. The college closed when it was unable to attract any more students, who were now expected to teach in miserable conditions in Britain's workhouses.[3]

On 18 January 2016 defence minister Mark Lancaster announced that Kneller Hall is one of 12 MOD sites proposed to be "released" (meaning sold) as part of the department’s drive for greater efficiency.[6] In November 2016 the Ministry of Defence announced that the site would close in 2020.[7]