Hacking book: from the Wild West of the 1980s to voicemail interception

Here is the concluding part of my afterword in the second and updated edition of The phone-hacking scandal: journalism on trial,* in which I seek to place phone hacking in some sort of historical perspective.

Yesterday's extract dealt with the methods employed by journalists from the 1950s onwards. Today, I pick up the story in the 1980s...

In the immediate aftermath of the 1986 Wapping revolution there was a period of what can only be described as Wild West behaviour. It was led by the News of the World's daily sister, The Sun.

There was the 1987 libel of the singer Elton John and the false story suggesting that Liverpool football fans had been responsible for the deaths of 96 people in the 1989 Hillsborough tragedy.

A Sun feature by Piers Morgan about the sexual proclivities of various male singers was headlined "The poofs of pop."

There was an even further push into darker territory, exemplified by the treatment of the dying television personality, Russell Harty, in 1988. Photographers desperate to snatch pictures of him in his hospital bed trained long lenses on his room after renting a flat across the road.

In the weeks beforehand, the News of the World had sought to "out" Harty as a gay man with Aids. In his address at Harty's memorial service, the playwright Alan Bennett referred to "the gutter press".

More examples of bad and ugly journalism in this era were detailed in a book by one of the leading commentators of the period, Ray Snoddy, who also hosted a Channel 4 television series, Hard News, which turned the tables on newspapers by investigating their investigations. It ran for two years from 1987.

Many thousands of words have been written about the press pursuit of Diana, Princess of Wales, throughout her marriage to Prince Charles until her death in 1997 in a Paris underpass with paparazzi following her car.

I witnessed two occasions towards the end of her life when packs of photographers goaded her in order to obtain saleable "candid" shots. It is fair to say that she was guilty of intruding into her own privacy on occasion, but it is blindingly obvious why she felt it necessary to make a pact with the devil.

Brian MacArthur, the founding editor of the short-lived middle market tabloid Today who went on to be a leading commentator on the press, summed up the history of royal reporting:

"The story of the tabloids and the royal family has been one in which new depths have been plumbed every few years – from photographs of a holidaying and pregnant Diana to Squidgygate and Camillagate – both of which involved intercepted phone calls, but in which newspapers were not themselves agents of the interception."

He told of rival royal reporters bugging each other and their paying of people in order to discover the family's travelling arrangements. MacArthur, who was noted for his fairness and level head, concluded:

"This is a murky area of Fleet Street pond life, where the rule is not to get caught and where editors plead public interest when they are only feeding the interest of the public."

The Diana years were also marked by a significant editorial crossover as the methods and ethos of celebrity journalism started to infect the reporting of other spheres. And the reverse happened too.

Celebrities, a term used to include almost anyone with a public profile, were subjected to the kinds of well-resourced investigative journalism usually reserved for matters of significant public interest.

In very different ways, these practices were revealed, and extolled, by three former News of the World journalists – the investigative reporter, Gerry Brown; the TV editor Sharon Marshall; and the newspaper's editor for a year in the mid-1990s, Piers Morgan.

Marshall's anecdotal book should not be taken too seriously, but, in evidence to the Leveson inquiry she explained that she quit the newspaper after refusing to carry out a story that would have breached the editors' code of practice. The executive who made the request was not disciplined.

A single sentence by Marshall during her appearance spoke volumes about the pressures on all tabloid reporters: "You're only as good as your next byline." It was a reminder of the hierarchical nature of popular papers. They are not democracies. In newsrooms, the editor's word is law.

The middle 1990s marked the beginning of a much more pro-active form of tabloid investigative journalism, notably at the News of the World. That newspaper had used covert tape-recording and filming as far back as the 1960s. Its reporters, usually a man and woman working together, had also grown accustomed to going undercover to expose sleazy stories of allegedly illicit sex.

But improvements in technology, enabling the concealment of pinhole cameras, combined with a much more aggressive use of subterfuge, gave rise to a plethora of so-called sting operations.

The master of this art was Mazher Mahmood, who was appointed as the News of the World's investigations editor. His penchant for dressing up in Arab robes gave him a nickname he was happy to adopt, the Fake Sheikh.

His entrapments of celebrities, minor members of the royal family and sportsmen won him awards. He also used his skills to expose people traffickers, illegal immigrants, drug-dealers and a variety of low-level criminals.

But his methods were hugely controversial and led to bitter criticism from his victims, several other journalists, lawyers and, occasionally, judges. There were suspicions - strenuously denied - that he sometimes used agents provocateurs to set up crimes that resulted in his exclusive stories.

And then came hacking. Seen in the context of a 50-year history of escalating misconduct, it can now be seen as a natural development rather than a one-off. It was not an aberration, as popular paper editors wish Leveson to believe. It was the culmination of a long-run process.

Since 2006, when the News of the World's royal editor, Clive Goodman, and its contracted private investigator, Glenn Mulcaire, were arrested, everyone has sought to distance themselves from any involvement in hacking.

Now that people have been charged I cannot write more on the details of the affair at this stage. But I think it's entirely fair to say, on the basis of my researches and conversations, that hacking was not viewed as an extraordinary activity.

It was seen as just another way of confirming story tips or obtaining stories. In the vernacular, it was just another part of "the game," not a matter of genuine internal concern.

So let me finish by considering the overarching reason for a half century of increasingly aggressive news-gathering by British popular newspapers. Ownership, when reduced to its essentials, is all about profits or propaganda, and usually both.

All editorial content, whether serving one or the other, derives from thoseobjectives. That is just one reason why the argument by proprietors that their private ownership of newspapers is the underpinning of press freedom is so hugely problematic.

There are two inescapable facts. First, the London-based national press is the world's most competitive. It has been marked throughout its history by intense rivalry between the publishers and editors of the popular papers, whose major preoccupation has been to build circulations in the belief that the size of their audiences provide them with a mixture of political, social, cultural and commercial influence. And, of course, profits.

Second, for the best part of 30 years, all the 20 or so titles that compose that national press have been suffering from declining sales, a decline that has accelerated in the past 10 years. Meanwhile, advertising revenue has fallen off a cliff due to the current recession and an insistent migration to the internet. Profits have therefore been much harder to come by.

It would be overly simplistic to say that this dire commercial reality is the sole reason for editorial misbehaviour on the scale I have described. But it undoubtedly played a role in the way in which publishers, notably Rupert Murdoch, were prepared to turn a blind eye to ethical standards in order to maximise the sales – and profits – of their newspapers.

That's why I conclude, just as I concluded my evidence to the Levesoninquiry, by quoting an observation in an essay by one of my City University London students: "Most ethical dilemmas in the media are a struggle between conscience and revenue."

Tomorrow:Harry Evans explores "the relationship of mutual antagonism" between journalists and politicians, one that is "moderated by mutual dependency"

*The phone hacking scandal: journalism on trial, second and updated edition, edited by Richard Lance Keeble and John Mair, will be published by Abramis on 17 September