Churchill at the siege of Sidney Street or cigar grasped in one hand, V for victory the other. Lloyd George with his flowing locks. Wilson with his pipe and cardigan, Attlee with his homburg hat. Thatcher's white scarf on top of a tank.

Nothing about photographing politicians is non-political. Politicians have used the photograph to try to promote their image for years. Ever since Margaret Thatcher allowed Herbie Knott to photograph her applying her make-up at her dressing table and the media fascination with the private lives of politicians intensified, the demand for "behind the scenes" material has increased.

Photograph: Andrew Parsons

Tony Blair and Gordon Brown kept their family life as private as possible but David Cameron has allowed the cameras into his home and the Conservative party employed a photographer, Andrew Parsons, during the election to take "candid" pictures of him on the campaign trail. The pictures were quality material but I thought the results were mixed. In a sense they rode on the back of a tradition of photojournalistic impartiality – until you realised how carefully selected they were. It is to the media's shame that many were used without accurate attribution to Conservative Central Office.

The pictures that politicians fear – and let's remember that they warrant barely a footnote in most political biographies – are the offbeat ones or the ones where photocalls go wrong. Major and Brown with head in hands as though in despair. David Miliband with his banana. Thatcher walking through an industrial wasteland on Teesside. Kinnock slipping on the Brighton pebbles while trying to rescue his wife from the surf.

Blair's image was very carefully managed throughout – unlike Thatcher – and to the assembled photographer fury, he refused to get on top of a tank (this was in 1996). Later he did allow me and others limited access to No 10 and his life on the road as a prime minister – access unprecedented and gratefully accepted as a result. Like Cameron's, the pictures – man working in his office, on the phone, in a corridor, on a train etc etc – were not actually that interesting and Blair's biography showed a more intriguing man than any of his photographs betrayed.

It seems to be towards the end of their careers that prime ministers, with power slipping away, begin to reveal something of their true natures: Ken Lennox's Thatcher in tears leaving No 10 for the last time, Dan Chung's picture of a strained Blair riding in a helicopter in southern Iraq, and my own photographs of Gordon Brown with his children inside No 10 in the last few hours before going to the palace – an unprecedented, and incidentally, totally unplanned, opportunity.

Upfront and personal, however, I've often come across a certain odd fragility in prime ministers. Something to do with the demand to relate and behave like an "ordinary" person when a photographer shows up for that very intimate moment which is the taking of a picture. Obsequiousness tends not to make good pictures of politicians – unless you happen to be Thomas Gainsborough or George Romney – and in a sense photographers are that unusual thing for them, a person just getting on with doing their job just as they might with anybody else.

I guess if you are accustomed to being surrounded by the sycophancy of power that can be unsettling. "I wore it especially for you," said Blair about the rather nice blue jumper he was wearing in the Downing Street garden one day and I thought, "Well, you're the prime minister, you can wear what you like surely – but thanks for the thought anyway!"

Photograph: Andrew Parsons

Cameron's non-stick image has, as yet, no rough edges. None of his photographs show a man under the stress of power or indeed the stress of fatherhood. Perhaps this is something to do with an assurance imposed by a public school education, perhaps by the division of responsibilities in a coalition, but as the premiership moves on, the stress will increase and the pictures will be more interesting as a result. The Conservatives' photographer is now a civil servant – the first ever Downing Street permanent photographer – and as I understand it is to photograph the official occasions, the "grip and grins", the line-ups, the photo calls. The "fly on the wall" stuff is no more for the moment but, Andy, grab the opportunities when you can – a few years down the line when Cameron is on the lecture circuit and the rest of us are hanging up our cameras for good, you should have an unprecedented photographic record of a seat of power.