Once a medieval estate adjacent to the hamlet of Upton, the extant Upton House was begun by Sir Rushout Cullen to the designs of an unknown architect. Surviving rainwater heads from Cullen's construction programme are dated 1695. In 1757 the estate was purchased as a hunting lodge by Francis Child (1735-63), the heir to the Child family banking dynasty, and Robert Adam’s first patron at Osterley Park from 1761. Francis Child died suddenly in 1763, and his entire fortune and property - including Upton - passed to his brother Robert.

Robert Child (1739-82) maintained Adam as architect at Osterley, and moreover, commissioned designs for his townhouse at 38 Berkeley Square, and for a new ceiling and frieze in the drawing room at Upton. Although Child did make improvements at Upton with the installation of items provided by John Linnell, Adam's scheme for the drawing room was not executed, and it is not known if he made any further designs for the house.

Following the death of Robert Child's widow, Sarah, in 1793, the Child fortune was then placed in trust for their eldest granddaughter, Lady Sarah Sophia Fane (1785-1867). In 1804 Sarah Sophia came of age, inherited the Child fortune, and married George Bussy Villiers, 5th Earl of Jersey. Upton then remained in the possession of the Earls of Jersey until 1894, when it was sold to Andrew Motion. Then in 1927 the estate was sold once more to Walter Samuel, 2nd Viscount Bearsted, the heir to his father's fortune as the founder of Shell. He extended the property, installed new interiors, and then gave it to the National Trust in 1948.