The tall, lean, ascetic looking Roy Scheider, who has died aged 75 of multiple myeloma, a form of blood cancer, had to work long and hard before making it as a movie star in his own right - emerging from the shadow of Gene Hackman (The French Connection, 1971), Dustin Hoffman (Marathon Man, 1976) and Bruce, the white shark (Jaws, 1975), having played second fiddle to all of them.

But Scheider, whose usually tough screen characterisations benefited from a broken nose sustained during a high-school boxing match, brought a calm, discreet professionalism to every role he played that mocked the pretensions of flashier, more theatrical film stars. Unfortunately, his talents, which blossomed in the 1970s, were less well exploited from the 1990s, when he appeared in films that were unworthy of him. In a career spanning four decades, he made more than 60 films, as well as playing numerous roles on stage and television.

Born in Orange, New Jersey, the son of an Irish Catholic mother and a German Protestant father, Scheider was a keen sportsman from a young age, competing in baseball and boxing tournaments. He graduated as a history major with the intention of going to law school. But after serving for three years in the US air force, he decided to become an actor, and studied drama at both Rutgers University, New Jersey, and Franklin & Marshall College, Pennsylvania. He made his professional debut in 1961 as Mercutio in Romeo and Juliet at Joseph Papp's New York Shakespeare Festival. He spent the next seven years in theatre, during which he played Face in Ben Jonson's The Alchemist (1964), Private Hurst in John Arden's Sergeant Musgrave's Dance (1966) and the title role in Hugh Leonard's James Joyce adaptation Stephen D (1967), for which he won an Obie award (Off-Broadway).

In the meantime, he had made one film, billed inconspicuously as Roy R Sheider, The Curse of the Living Corpse (1964), a low-budget horror movie in which he played an asthmatic alcoholic among the murder victims. But he started his movie career proper in the role of a mafia heavy in the lame thriller Stiletto (1969). More rewarding was his New York businessman involved with fashion model Faye Dunaway in Jerry Schatzberg's Puzzle of a Downfall Child (1970) and prostitute Jane Fonda's former lover and pimp in Klute (1971).

His first big role, which gained him an Oscar nomination, was as Hackman's low-key cop partner, Buddy Russo, cracking a drug ring in William Friedkin's The French Connection. Cashing in on the success of this exciting movie, 20th Century Fox then trusted Scheider with the lead in The Seven-Ups (1973), in which he again played a New York cop.

Temporarily getting away from the hardman image, he was next cast in the comedy Sheila Levine is Dead and Living in New York (1975), as a doctor with whom the frumpish girl of the title (Jeannie Berlin) becomes smitten. Unfortunately, the film died in New York and elsewhere but, that same year Scheider was back playing a cop in Steven Spielberg's mega hit Jaws, doing sterling work as Police Chief Martin Brody at the holiday resort of Amity.

His subtle performance as the dedicated family man who disliked the water was the sane fulcrum around which the plot turned. His suggestion to the shark hunter, after seeing the size of the threatening fish - "You're gonna need a bigger boat" - was voted No 35 on the American Film Institute's list of best quotes from US movies. Scheider played the same role with little variation in Jaws 2 (1978).

He portrayed the first of his CIA types in John Schlesinger's The Marathon Man, getting killed halfway through the movie - before his brother (Dustin Hoffman) goes on to find his murderer. But it was in 1979, as the government agent recovering from a nervous breakdown and trying to find the origin of a death threat against him, in Jonathan Demme's The Last Embrace - and in Bob Fosse's autobiographical All That Jazz - that Scheider demonstrated a depth of feeling and a more complex persona than hitherto. In the latter, which earned him the Golden Palm at Cannes, and another Oscar nomination, he also sang and danced as a director-choreographer fantasising while undergoing open heart surgery. It was, he said, his favourite role.

In 1980, Scheider returned to the stage in Peter Hall's Broadway production of Harold Pinter's Betrayal, opposite Blythe Danner and Raul Julia. Financial rewards notwithstanding, it might have been better if he had stuck to the theatre given some of the mediocre film vehicles that followed. The thriller Still of the Night (1982) did nothing for his career, nor did his flying a souped-up helicopter in Blue Thunder (1983) or being a scientist on a mission to Jupiter in 2010 (1984).

Back on terra firma, he was excellent as Cohen, the jaded, taciturn hitman partnered by the younger, unstable and violent Adam Baldwin in Cohen and Tate (1988), and as a hard-bitten American colonel on the Czech-West German frontier in John Frankenheimer's plodding and dated The Fourth War (1990). There was not much difference between that role and his CIA man in The Russia House (also 1990). One of his last good pictures was David Cronenberg's The Naked Lunch (1991), in which he played the sinister quack and drug dealer Dr Benway.

Scheider, who lived in Long Island, had been politically active since his days protesting against the Vietnam war. In 2003, he was among a group of demonstrators who lay down in the road in a symbolic reference to Iraq war casualties. He is survived by his second wife, the documentary filmmaker Brenda King, their children Molly and Christian, and Maximillia, the daughter of his first marriage.

· Roy Richard Scheider, actor, born November 10 1932; died February 10 2008