The building of the castle began in the spring of 1544, or possibly the year before.[3] The castle was constructed around a square keep;[4] to the south towards the sea the castle had an angled bastion with the same arrangement on the north side.[4] Square gun platforms made up the east and west sides.[4] This is believed to be the first use in England of the trace italienne style of fortification already in use in Continental Europe.[5] Henry's previous forts used semi-circular bastions which could not be fully covered by flanking fire from the supporting walls, and thus left an area of "dead ground" that could be exploited by enemy assault parties and miners.[6]

The building work was supervised by the Governor of Portsmouth, Sir Anthony Knyvett, who kept King Henry advised of the rapid progress of the construction with regular letters. By July 1544, the first two guns had been installed. Throughout the summer, Knyvett made repeated appeals for more funds, as there was often insufficient to pay the workforce. By October, more than £3,000 had been spent, £1,300 of which is known to have come from the Dissolution of the Monasteries. In the same month, Knyvett reported the completion of the castle to the Lord Chancellor, hoping the king would be pleased with the work "which was of His Majesty's own device".[7]

By chance, King Henry was in Portsmouth on 18 July 1545, when the arrival of a French invasion fleet resulted in the Battle of the Solent, in which the English flagship, the Mary Rose, was lost. Although the castle overlooked the action it was not directly involved. Henry ordered improvements to the castle including embrasures to protect the flanks of the bastions and internal traversing walls and guard houses to aid defence should the external walls be scaled.[8]

Edward VI spent a night at the castle in 1552 while inspecting the defenses of Portsmouth.[9]

The early part of the 17th century found the castle "verie ill prepared for defence",[10] and in March 1625 a fire started that caused significant damage to the fortification.[11] It took a further ten years for the damage to be repaired only for further damage to be caused by a fire in March 1640.[12]

During the English Civil War the castle was initially held by royalist forces under the command of one Captain Challoner.[13] They moved the castle's guns so they pointed inland leaving the seaward side undefended.[13] The castle was captured by the Parliamentarians as part of the Siege of Portsmouth in September 1642. The Royalist garrison consisted of only a dozen men,[13] opposed by infantry backed by cavalry of the Parliamentarians.[13] Equipped with ladders the Parliamentarians approached the fort by night and although spotted and fired on by the guns of Portsmouth were able to make to the seaward side of the castle.[13] At the same time a small party approached the main gate called on the castle to surrender.[13] Captain Challoner was at the time somewhat inebriated and asked them to come back in the morning.[13] Somewhere around this time the guns of Portsmouth once more opened fire on the assaulting force and the Parliamentarians responded by scaling the walls and captured the castle without further opposition.[13] A significant Parliamentarian garrison was then installed in the castle to keep it from being retaken.[13]

With the end of the civil war the castle became a prison for a time.[14] In 1665, King Charles II commissioned his Dutch-born master engineer, Sir Bernard de Gomme, to improve the defences of Portsmouth. A plan dating from around 1668 shows de Gomme's design for an earthen glacis to be constructed outside the moat for protection against direct artillery fire, and a new platform for 30 guns to be built between the moat and the sea shore.[15] The castle's gate was also reconstructed, and a large stone plaque bearing Charles's coat of arms was set above it, dated 1683.[16]

During the Glorious Revolution, the Catholic James FitzJames, 1st Duke of Berwick was pressured by George Legge, 1st Baron Dartmouth into handing control of the castle to the Protestant Captain Carter.[17] Control was transferred by 18 December 1688.[17] The castle was damaged in an accidental gunpowder explosion in August 1759 that killed 17 people.[14] Although the damage was patched up, by the 1770s the castle had become "a most shameful ruin" and in 1785, the Master-General of the Ordnance, the Duke of Richmond, declared that it was "of too bad a form to deserve the expense necessary to repair it". Nevertheless, when invasion threatened in 1797, the castle was urgently repaired, but only eight 32-pounder and five 6-pounder guns could be mounted, as de Gomme's gun platform had been eroded by the sea.[18]

Another rebuilding began in 1814 with the castle being extended north by 20–30 feet.[19] The moat was rebuilt and a Counterscarp gallery added.[19] In 1828 a lighthouse, commissioned by The Admiralty, was constructed on the western gun platform,[20] rising 34 feet above the its base atop the walls.[21] A few years later in the 1830s a proposal was made to build a ship canal to the castle from Langstone Harbour[22] The next alteration to the castle occurred in 1844 when it was modified so that it could serve as a gaol for 100 military prisoners.[23] This arrangement lasted six years before the castle lost its function as a prison and gained seven up-to-date gun emplacements.[23]

During World War I the castle was at first manned by Royal Garrison Artillery and No 4 company of Hampshire R.G.A territorials.[4] Later these units were transferred to France and were replaced by the Hampshire R.G.A. Volunteers.[4] Due to the threat from zeppelins a QF 3 inch 20 cwtAA gun was added to the castle.[25]

Even though it was still in active military use the castle had by 1929 become a tourist attraction.[26] Visitors to Southsea were able to watch the castle garrison carry out practice firings out to sea.[26]

During the World War II the castle was hit by a number of incendiary bombs which did little damage.[27] The castle was manned by a range of units including coastal artillery from the regular army and the home guard.[27]

On 23 June 1940 the castle became involved in an armed stand-off with French ships that had escaped the fall of France.[27] The garrison was ordered to prepare to fire on the French craft while the French destroyer Léopard responded by aiming its guns at the castle.[27] The standoff came to end on 3 July when British forces boarded the ships as part of Operation Catapult.[27]

In 1960 Portsmouth city council took control of the castle and in 1967 it was opened as a museum.[28] By 1975 the castle was able to attract 99,704 visitors over the course of a year.[29]

The castle is operated as a tourist attraction by Portsmouth City Council. It can also be hired as a venue for weddings and parties. The area next to the castle known as Castle Field forms a natural amphitheater and is used for public events including fairs, music concerts and festivals. Among other exhibits the castle houses a collection of cannon. Two of these are outside the castle, a 68-pounder and an RML 9 inch 12 ton gun.[30] Within the castle the collection includes a 24-pounder from HMS Royal George,[30] an RML 9 pounder 8 cwt[30] and two hexagonally rifled Witworth 3-pounder breech-loaders dating from 1876 [30]

Prior to the 2011 reopening the castle underwent an extensive cleaning carried out by Portsmouth City Council.[31]

The castle is reportedly haunted by the ghost of a young girl in scarlet, who, according to local legend, was the daughter of one of the castle's 19th century lighthouse keepers claimed by scarlet fever.[32]