The Manor of Croydon was connected with the Archbishop of Canterbury from at least the late Saxon period, and records of buildings date back to before 960. The Palace as it now exists is a group of largely 15th and 16th century buildings, "an aggregate of buildings of different castes and ages", as Archbishop Herring found it in 1754.[1] The 15th-century Great Hall is thought to have been installed by Archbishop Stafford (d. 1452), with a late-14th-century two-storey porch and a vaulted ceiling to the lower chamber. The hall interior has a rich 16th-century timber roof and windows with interesting features such as the late Gothic interior porch. The Great Hall was partially remodelled in the 17th century by archbishops Laud and Juxon, who also rebuilt the chapel.

West of the Hall are the state apartments including the first-floor Guard Room, now the school library. The room is ascribed to Archbishop Arundel (Archbishop 1396–1414), and has an arch-braced roof with carved stone supports and an oriel window. Other rooms have later panelling and fireplaces. The chapel has fine 17th-century stalls and an elaborate corner gallery. The fine altar rails are now in the Guard Room. The exterior of the whole palace is of stone or red brick, with early stone windows or Georgiansash windows.

By the late 18th century, the Palace had become dilapidated and uncomfortable and the local area was squalid. An Act of Parliament enabled Croydon Palace to be sold and Addington Palace on the outskirts of Croydon to be bought in 1807. This became the new episcopal summer residence for much of the rest of the 19th century.