The site is enclosed by a boundary wall with a monument to John Playfair, president of the Edinburgh Astronomical Institution, in the southeast corner. The oldest part is the Gothic Tower in the southwest corner, facing Princes Street and Edinburgh Castle. It is also known as Observatory House, the Old Observatory, or after its designer James Craig House. The central building with the appearance of a Greek temple is the Playfair Building, named after the building's designer William Henry Playfair. This houses the 6-inch (15 cm) refractor in its dome and the 6.4-inch (16 cm) transit telescope in its eastern wing. The largest dome of the site is the City Dome in the northeast corner. During the early 20th century this contained a 22-inch (56 cm) refractor.

James Craig designed the observatory, which, under Robert Adam's influence, was to look like a fortification with a wall and Gothic towers at its corners. The city controlled the building project, but the money ran out after only the first of the towers was built. Short moved into this as residence and ran the observatory until his death in 1788. An actual observatory, smaller than originally planned, was also built where the Playfair Building is now. After Short's death the observatory was kept going by his family for a while, then leased to opticians and finally abandoned around 1807. The site reverted to the city.

Short's daughter Maria Theresa Short was to return to Edinburgh in 1827. She ran a second – a popular and commercial rather than scientific – observatory elsewhere on Calton Hill. In 1850 this was removed[1] and she moved to Castle Hill, where her enterprise eventually became today's Camera Obscura on the Royal Mile.

In 1812 the observatory was handed over to the Edinburgh Astronomical Institution, which opened its popular observatory in the Gothic Tower. In 1818 work began on the Playfair Building. Designed by William Henry Playfair this was to become the scientific observatory of the Institution. Following a loyal address to George IV in 1822 this became the Royal Observatory. Again the funds proved insufficient, so that the purchase of instruments and the employment of an observer depended on funding from the Government. After much delay the instrumentation was completed in 1831 with delivery of the transit telescope. Fraunhofer had made the lens, but after his death it fell to Repsold - and after his death to Repsold's son - to complete and install the instrument.

According to the English journalist William Jerdan, naturalist and oceanographer Edward Forbes, F.R.S. and his "The Red Lions", a dining club for younger members of the British Association, (named after the tavern where the first meeting was held), had occasion to run up to the observatory of Calton Hill for astronomical studies.[2]

The main purpose of the observatory was a time service. The transits of stars through the meridian were observed and used to keep the observatory clockaccurate. Accurate time was important for navigation, and mariners would bring ships' chronometers from the port of Leith up to Calton Hill for adjustment. In 1854 the time ball was installed on Nelson's Monument next to the observatory and visible from the port. This was controlled by electrical pulses from the observatory clock. A few years later the One O'Clock Gun on Edinburgh Castle was added. This was also controlled through an electrical wire, spanning the city from Calton Hill to the Castle. Today the time ball and One O' Clock Gun are tourist attractions. They are no longer controlled from a state of the art clock, but are triggered "by hand".

By 1888, when Smyth resigned, through underfunding the observatory's equipment had become largely obsolete. Also, the site had originally not been chosen through scientific investigation and its proximity to the city was causing problems. In 1896 the Royal Observatory moved to a new site on Blackford Hill. The Calton Hill Observatory, once again, reverted to the City of Edinburgh.

In the wake of the Royal Observatory moving to Blackford Hill, a new City Observatory was created on Calton Hill toward the end of the 19th century. The 6-inch (15 cm) Cooke refractor was donated by William McEwan and installed in the dome on the Playfair Building. Additional domes were built for a 13-inch (33 cm) reflector donated by Robert Cox and for a 13-inch (33 cm) refractor transferred from the Dunecht observatory. Only remnants of these two domes remain today. The City Dome was built to house a 22-inch (56 cm) refractor. This did not perform very well and was dismantled in 1926. The dome remained and was used as a lecture theatre.

The observatory opened in 1898 with William Peck as City Astronomer. After his death in 1925 his assistant John McDougal Field continued to run the City Observatory.

In 1924 the Astronomical Society of Edinburgh was formed with Field as first president. William Peck and Ralph Sampson, the then Astronomer Royal for Scotland, were honorary presidents. Field died in 1937; and in 1938 the observatory was leased to the Society, which ran the City Observatory until vandalism and theft of roofing material rendered the site unusable in 2009.[3]

In 2009 the Astronomical Society of Edinburgh moved out of the City Observatory and the buildings reverted to the City of Edinburgh Council.[4] The Council renovated the Observatory House building structure and the interior was restored by the Vivat Trust (a charity which aims to preserve old buildings) who let it as holiday accommodation.[5][6] After being unoccupied for several years, in 2012 the Council started a £1,000,000 program[4] to renovate the remaining buildings in partnership with the Collective Gallery.[7] In 2014 the Collective Gallery relocated from Cockburn Street, Edinburgh, opened an exhibition in the City Dome[8] and received an award of £900,000 from Creative Scotland to further develop the site.[9][8]