Unhealed wounds

David Lean is worshipped for his exotic epics - Lawrence of Arabia, Doctor Zhivago, A Passage to India - but it was with intimate English stories of love destroyed by duty and shame that he was in his element. A century after the director's birth, David Thomson pays tribute to a master of the stiff upper lip

It is late afternoon in the jungle that once was Burma. A wisp of a man, so starved that the bones stick out of his body, is contemplating the grandest work of his life. Perhaps it is even his great love, though "love" is not one of his words. We don't know much about the private life of Colonel Nicholson (Alec Guinness). We're not sure it exists. He could be like Fred, the husband in Brief Encounter, who never quite notices the storm that has swept over his wife Laura.

Nicholson is a prisoner now, yet he tells the warm evening air, and the Japanese commandant who hardly understands him, that he's given the army 28 years. And he'd do it again. He pats the crafted woodwork of the bridge as if stroking a favourite Golden Labrador. It is a moment of exquisite, perilous happiness. And it is just before the climax to David Lean's The Bridge on the River Kwai, the night before the grand inauguration of that bridge - and its necessary destruction.

When The Bridge on the River Kwai opened in 1957 and won seven Oscars, it was described as a film about the madness of war. I'm not sure that was accurate. In 1957, there were at least three other films that spoke more drastically to the disasters of war - Anthony Mann's Men in War, Stanley Kubrick's Paths of Glory and Nicholas Ray's Bitter Victory. Kwai is about something else: the odd yet plausible way in which some ordinary, decent fellows are fulfilled by war. Some men cherish duty and are lost without it. I think that's what really makes Kwai a tragedy - the misguided urge in this colonel to maintain the morale of his wretched men by taking on the task of building a beautiful bridge in the jungle (which will help the enemy). It's a curious, Lean-like love story: next morning, in a matter of minutes, Nicholson learns that he must kill the thing he loves. He must blow up the bridge as the commando team sent on that mission are eliminated.

Lean would have been 100 this year, and he is still (if we accept that Alfred Hitchcock went to America and absorbed a lot of Hollywood in his character) probably the most celebrated English movie director. Lean was a confined Englander for many years: he made small, thoroughly tidy pictures from English classics or from stories that addressed the tragicomic loneliness of the English romantic strain. Kwai was his fourth nomination as best director. And then, on the wings of astonishing success and an unlikely partnership with Sam Spiegel, he became an international figure with films including Lawrence of Arabia (1962), Doctor Zhivago (1965), Ryan's Daughter (1970) and A Passage to India (1984).

During that period, he went on location more and more (250 days in the jungle for Kwai) - yet he clung to English types, and so he cast Guinness as Nicholson, as an Arab prince (Lawrence of Arabia), as a Soviet commissar (Doctor Zhivago) and as an Indian (A Passage to India) painfully close to the comic figures created by Peter Sellers. In America, in 1948, Guinness's startling Fagin had drawn some criticism as being antisemitic. In fact, the performance was true to Dickens and the novel's illustrations. But in Lawrence and A Passage to India, there were more grounds for the charge that Lean had reduced exotic foreign characters to a rather superficial, if assured, gloss of English acting. So it comes as a great shock, after facing the silken vagueness of Guinness in those big films, to look again at the tragic, ravaged close-ups of Celia Johnson in Brief Encounter (1945), or even Ann Todd in The Passionate Friends (1949) or Madeleine (1950). There is no doubt that the films from the 40s now seem the most modern, the most personal and the most complicated.

This may not seem likely, or proper, but for me, Brief Encounter, Oliver Twist and The Passionate Friends are richer than Lawrence of Arabia, Doctor Zhivago or Ryan's Daughter. Those who once worshipped those epics (including Steven Spielberg) seem to see Lean escaping from a kind of cramped Englishness and a limitation of resources that had become claustrophobic and intimidating. It was as if Lean had proved himself as an adventurer film-maker, an epic traveller full of the spirit of exploration and military prowess, so that a film like Lawrence triumphed by making an unknown and mysterious world tractable and film-friendly.

Lean was from Croydon, south London, raised as a Quaker, the son of an accountant, who was bowled over in his late teens by a first exposure to movies. He went on to become the bearer of an un-English tradition of spectacle and epic, as borne out in the stunning 70mm landscapes of

Lawrence, themselves portended in that great cut where a blown-out match ushers in the vast desert horizon with just a rim of sunlight to it. At the same time, as you come away from Lawrence, you feel the odd hints of reticence, such as in the memorial service where Allenby (Jack Hawkins) admits the wonder and absurdity of TE Lawrence. Lawrence of Arabia gives no hint of how a modern nation (it could be Iraq today) emerged from the postwar untidiness in the area, and it has as little real feeling for Arab sensitivity as a film that breezily cast Anthony Quinn and Guinness as men of the desert. So it takes a little time for the viewer to realise that Lawrence is just another Brief Encounter: how TE had a short time in the desert (a passion, or a madness), how he found love and hatred, and how resolutely he reassembled the skeleton of duty so that he could turn his back on the empty place.

As centenary tributes get under way (on both sides of the Atlantic) and as the British Film Institute rereleases one of Lean's least-known or appreciated films, The Passionate Friends, there is much to wonder about. How good was David Lean? What happened as he made his "big" pictures? Doctor Zhivago and Ryan's Daughter feel pretty hollow and clumsy now. They supposedly encapsulate the grandeur of Lean, and it's not that they don't have amazing spectacle - the house overtaken by winter in Russia, the storm in Ireland - but it takes a generous viewer to feel the same sense of disastrous love in the close-ups of Omar Sharif, Julie Christie or Sarah Miles as throbs still in the deepening shots of Celia Johnson in Brief Encounter or Ann Todd in a couple of the films she made with Lean.

There were those who rejoiced to see Kwai and Lawrence because they were British picture-making sweeping, or leading, the world. But if you compare those later films with the merciless tightness of the British pictures of the 1940s, you have to admit the slackening in the art and the process that has led a real film artist to become a clichéd showman. The love story in Zhivago is tepid; that in Ryan's Daughter oddly lurid. Lean had lived past the age of censorship, and he was drawn into nudity and movie "sex", but at heart he was a repressed, puritanical director - more an accountant than a poet - far more comfortable to have sex personified by the express train that doesn't stop.

Lean was in his element not just with love stories (as opposed to epics), but in stories of love being shut down by duty, shame and a fear of fuss. Lean's Lawrence is a man disconcerted by his personal discoveries in the desert: his boyish taste for dressing up and his instinct for sadomasochistic punishment. We can only wonder how far suburban respectability and the Quaker reluctance to see movies had built up that repressed code in Lean, along with a hatred of conformity that aches for Rachmaninov and the surging express trains to smash the timetable world of Brief Encounter. But whereas, in so many US films of the same era, the force of desire and its emotional energy are urging characters and audiences to break out, so Lean is bringing restraint and self-effacement into his films. It's hard to decide what Zhivago really thinks about the Soviet revolution, or even the desirability of "poetry", whereas films such as Brief Encounter, The Passionate Friends, Madeleine and the Dickens adaptations are unhealed wounds, which repeatedly ask the awkward question: what is the place of damaged feeling in this England?

Lean was both an apprentice in the picture business and a craftsman editor. He was a dud at school, and far from the brilliant university men who strolled into movie-making in the 60s. Instead, Lean was the chosen aide to no less than Noël Coward, and it's important to see how deeply he felt himself to be protege to the man who best represented Britain's self-mocking stiff upper lip in the moment of war. Working for and with Coward, Lean made his first four films: In Which We Serve (co-directed with Coward); This Happy Breed and Blithe Spirit (both Coward plays); and Brief Encounter (a Coward one-act play).

It's hard to believe that Lean never suspected Coward's homosexuality. It's possible (though I know of no evidence for this) that Lean actually challenged Coward: write a story, very simple and controlled in its elements, but with English emotions being self-corrective and self-denying. Brief Encounter is a story about a woman, yet there are reports that Coward had first conceived it as a gay allegory. I doubt that Lean took that on, because the filming of Brief Encounter is infatuated with Celia Johnson. Just as, in the US the same year, a romance like Meet Me in St Louis (1944-45) told the war-torn public, you can go home again, home is still there; so Brief Encounter admits, well, yes, there may have been love affairs, but they will be set aside in favour of loyalty, habit and the Freds of the old.

As you start to watch Brief Encounter nowadays, you anticipate a surfeit of English indirectness and euphemism, an onslaught of restraint - this is a love story in which the lovers don't do it (and first preview audiences were indignant about that). And the merit of the film 60 years later are the questions it still raises: is it best for a nation that Fred's limited sentimental education be honoured, or does Laura deserve her wild escape, even if it ends in misery or the kind of Anna Karenina-like death that train stations hint at? Read the text of the film (the Coward stuff), and you gather that marriage is an estate or a regiment "in which we serve". But see the film (watch and suffer the ravaged close-ups, without make-up, with the piano and Johnson's voice in beautiful counterpart), and you understand how far Lean has dug up his own buried soul in the name of a woman's picture.

Did he know it? Not necessarily - I see him as one of those directors who needed to be free from too much introspection, and I suspect that England in the late 40s insisted on action. So Lean made his two Dickens films - Great Expectations and Oliver Twist - with the team of cinematographer Guy Green and designer John Bryan. They are superb Dickens, in that they grasp the need to depict the world from the perspective of a frightened child. Does Oliver really want the secure, safe, upper-middle-class life he is heir to, or isn't there something "more" in the dark life that Fagin offers him? There's no doubt which is most vivid on screen, and Lean's film is enough to leave you wishing that Dickens had written a novel in which Oliver became the Artful Dodger, and thereafter a leading London criminal who then moves up in the world of business, society and politics. Lean knows that it is excitement or madness that makes life - not order.

And yet he never could throw off order. The real discovery of this centenary is to celebrate two Lean films that have stayed oddly hidden - The Passionate Friends and Madeleine. The first is from a story by HG Wells and describes a love triangle in which a woman (Ann Todd married to Claude Rains) cannot quite forget or give up her affair with another man (Trevor Howard). It's as if the game posed in Brief Encounter now goes on in life, but over a long period. Once more, Lean settles on the pressing need for propriety, but not before the film has put its characters and the audience through a wringer of contradictory feelings. Indeed, it's a film that leaves you rather longing never to be in love again - until you see the curiously cold, pinched beauty of Ann Todd anew (she was Lean's wife at the time).

And we see her again as Madeleine - a Glaswegian, 19th-century, very proper young lady who has an ideal gentleman caller (Norman Wooland), as well as a flagrant cad (Ivan Desny) who knocks on her window with his cocky walking stick. Madeleine is based on a real case in which the Scottish verdict of "not proven" was returned. It is a staggering period film noir, and another study of a lover (probably) killing her beloved.

These two films impress something else upon us, something that was there in the David Lean obituaries, begging for explanation. After all the glory of a career full of plaudits, the obituarist has to add: "Sir David was married six times." All of a sudden, Croydon, accountancy, Quakerism and Fred-ism are out the window. It is said - and the loyal biographies are not good on this reality - that Lean did not just drop wives and lovers, he forgot them, he erased them. Not orderly? Not English? Maybe, but I suspect this is the clue to Lean's artistic personality, and I would just point out that in his masterly views of Claude Rains's owl-like composure (the cuckold, yet a lover, too) in The Passionate Friends, we have two upright Englishmen surveying one another, with an eventual combined total of 12 marriages between them. David Lean's The Passionate Friends is in cinemas from June 6. Rediscover David Lean, a retrospective, is at BFI Southbank, London SE1, through June and July.

· This article was amended on Saturday May 17 2008. David Lean's film The Passionate Friends was not based on a novel by Eric Ambler, although he wrote the screenplay. It was adapted from a story by HG Wells. This has been corrected.