When Tony Blair, on his first day as premier in 1997, created a new super-ministry to replace the previous Conservative administration's Department of National Heritage, he called it Culture, Media and Sport. But, although the c-word was given first place on the letterhead, it has been the other two responsibilities that have most preoccupied DCMS secretaries of state, for good or ill.

Sport, in the form of the securing and then execution of the 2012 Olympics, provided the post's crowning glory so far – while the media have proved to be its crown of thorns. Maria Miller, who resigned yesterday after facing sustained pressure over an excessive expenses claim, is the second holder of the position in a row to depart with their head caught in journalistic cross-hairs. Just 18 months ago, Jeremy Hunt was moved to health following controversy over his alleged over-closeness to the Murdoch empire, in an industry it was his responsibility to regulate.

Because both newspapers and games are generally noisier creatures than culture, an incumbent's efforts on behalf of the first word in their brief are often overlooked. But, even so, Miller has made notably little impact on the arts. Whereas the first secretary of state at the DCMS, Chris Smith, was (like the first heritage secretary, David Mellor) someone frequently to be seen at theatres or in CD shops, Miller always felt more of a media figure.

Most cultural initiatives during her time at the DCMS – appeals to keep rare paintings or books in Britain, grants to museums and galleries, reform of the heritage rules – have been presented to the public by Ed Vaizey, her very able minister for culture. Miller herself led the press releases and conferences relating to a new cultural agreement with China and the nomination of the Forth bridge for World Heritage Site status, but both those campaigns felt more driven by wider politics – David Cameron's desire to keep China and Scotland, for different reasons, sweet – than by any personal artistic vision.

To be fair to her, the aspect of the cultural portfolio that attracts most attention, the provision of funds to the Arts Council, is an almost impossible game to win. If the figure is lower than theatres and museums hoped for, the DCMS takes the blame; but if it is higher, then the prime minister and chancellor of the exchequer are prone to claim the credit. Serving in a time of cuts, Miller became the target for angry rhetoric from National Theatre boss Nicholas Hytner and other cultural leaders about what they saw as her refusal to understand the economic value of the arts to Britain.

But if Miller appears to have achieved little at the DCMS, she might argue that she had no time to complete, or even really start, her work. Her departure solidifies the department's reputation for having an ejector seat behind its biggest desk. Although it is the Home Office that traditionally has the reputation as the riskiest cabinet position, the DCMS has now had eight bosses since 1997. And, as Tessa Jowell accounted for six of those 17 years, none of Miller's five other Labour or coalition predecessors completed two years in the post. Continuing this trend, even if he avoids trouble, her successor, Sajid Javid, will have barely 12 months before parliament is dissolved ahead of the May 2015 election.

One brief incumbent – Labour's Andy Burnham – left in 2009 because he was fast-tracked for promotion, but the majority have gone unwillingly and/or unpleasantly. You don't have to believe in post-Leveson "media witch hunts", as some of Miller's supporters apparently do, to see that the holder of a cabinet position that includes oversight of the press may invite particularly brutal scrutiny of their conduct. Even the department's great survivor, Tessa Jowell (2001-07), experienced a lengthy scandal over her husband's business affairs.

But the mayfly careers that have become common in this position have also had consequences for the department's political responsibilities. Given the general estimate that it takes at least a year for a fresh secretary of state to impose their will, the DCMS has been under full control for less than half of this parliament and, by the time Javid gets his feet properly under the desk, he may also be out. It is little wonder that some of the ministry's clients, especially in the arts, complain of a lack of continuity and achievement.

If the DCMS is retained by the next government, the PM may find it tough to select secretaries of state who are sufficiently newspaper-proof to survive. But in the cause of improving relations with the cultural community, they might also be wise to choose someone who, like Mellor and Smith, gives the impression of having art in their heart. Miller always felt more at home with the media, even though they eventually made her ministerially homeless.