Somerset was blamed by the press and the government for the sufferings of the British soldiers in the terrible Crimean winter during the Siege of Sevastopol owing to shortages of food and clothing[25] although this, in part, was the fault of the home authorities who failed to provide adequate logistical support.[4] A piecemeal allied assault on Sevastopol on 18 June 1855 was a complete failure.[4] The anxieties of the siege began to seriously undermine Somerset's health and he died from a mixture of dysentery and clinical depression on 29 June 1855.[4] His body was brought home and interred at St Michael and All Angels Church, Badminton.[8]

"This house with 238 acres of land was purchased by 1623 of the friends, admirers and comrades in arms of the late Field Marshal Lord Raglan GCB and presented by them to his son and his heirs for ever in a lasting memorial of affectionate regard and respect."

A blue plaque was erected outside Somerset's house at Stanhope Gate in London in 1911.[29]