For a reminder of the best and worst of Frank Sinatra, look no further than the recording of a concert he gave at Carnegie Hall in the spring of 1974, shortly after emerging, at the age of 58, from a brief “retirement”. A medley of three ballads – “Last Night When We Were Young”, “Violets for Your Furs” and “Here’s That Rainy Day” – is prefaced by a clumsy, cheesy, self-regarding monologue drawing the sort of sycophantic laughter and applause to which he had long become accustomed. Then he gets down to the business of bringing a great seriousness to bear on the trilogy of peerless songs, each an established part of his repertoire, reaffirming all the qualities of technique and interpretation that had made him the greatest male interpreter of Broadway melodies.

This dissonance between the life and the art, a permanent feature of the singer’s career, is a biographer’s dream and the inevitable preoccupation of the concluding volume of James Kaplan’s massive two-part study of Sinatra. The music’s sublime artistry provides a counterpoint to the lurid details of four marriages, countless affairs and fist fights and a web of connections with America’s ruling elites, from the back rooms of mob-run casinos to the White House, all spanning a period of great cultural upheaval.

At the end of Frank: The Voice, published five years ago, the author left the skinny crooner from Hoboken clutching an Academy Award statuette for his performance in From Here to Eternity, the role that, in 1953, lifted his career out of the slump which had followed his widespread success in the 1940s and had left him – in Kaplan’s words from the earlier volume – “smelling like a loser”. Sinatra: The Chairman begins with a triumphant restoration of his fortunes on almost all fronts. By 1956 he is earning $250,000 for each film, a guaranteed $200,000 a year from his record company, and a great deal more for his various television appearances.

The year of his album Songs for Swingin’ Lovers was, of course, also the year of “Heartbreak Hotel”, putting Sinatra in the interesting position of becoming the dominant symbol of a set of seemingly modern but strongly conservative values at precisely the time they came under attack from the early exponents of rock’n’roll. He rode out that storm, only to be capsized in the middle of the following decade by the arrival of the Beatles, whose music he could never truly comprehend. From that moment he began to live on past glories, boosted only by the occasional chart success with songs he despised for the very good reason that they lacked the combination of melodic ingenuity and verbal sophistication necessary to bring the best out of his own talent. (“Every time I get up to sing that song,” he said of “My Way”, “I grit my teeth because no matter what the image may be, I hate boastfulness in others. I hate immodesty, and that’s how I feel every time I sing the song.”) Having tried and discarded the beard, turtleneck and medallion look and divorced a much younger third wife, Mia Farrow, he dusted off his tuxedo, selected a more age-appropriate toupee, married his fourth wife, and settled into a long final act which, between 1974 and his death in 1998 at the age of 82, included more than a thousand concerts.

Kaplan’s two volumes, running to total of 1,600 pages, represent an attempt to construct a synoptic biography in large part through the exploitation of earlier attempts, from which he quotes repeatedly and at length as he assembles his detailed 360-degree portrait. Hitherto, if you wanted the gossip, you went to Kitty Kelley, J Randy Taraborrelli or Seymour Hersh. If you wanted to learn about the music, you consulted Will Friedwald or Charles Granata. For a balanced account, you read Arnold Shaw or Donald Clarke. If you wanted an extended fan letter, Pete Hamill. For a snapshot of the singer at the height of his fame, Gay Talese. For a reconstruction of the Rat Pack years, Shawn Levy. For the movies, Tom Santopietro. For the valet’s view, George Jacobs. And so on. Kaplan draws heavily on every one of them, as well as on dozens of memoirs from the supporting cast, including three of Sinatra’s wives and his two daughters, and on the clippings files of many syndicated gossip columnists, including those with whom the singer fell out.

Much of the author’s work, then, is necessarily interpretive, and sometimes he is reduced to presenting two versions of the same incident (such as the conflicting accounts of his near-drowning in Hawaii in 1964) and inviting the reader to decide between them. Coming late to the task, he is able to present original research only in the closing stages of the story, from witnesses including Quincy Jones, Raquel Welch and Larry King. Here, however, he occasionally strikes gold. The actor Tony Bill, Sinatra’s costar in Come Blow Your Horn, describes a trip to the singer’s Palm Springs house as being “like visiting a very, very posh hotel, the owner of which was home but busy”. The comedian Shecky Greene, once Sinatra’s warm-up man, remembers: “The air was volatile and violent around him all the time.” Tiffany Bolling, a 20-year-old good-time girl when they began an affair just as his marriage to Farrow was disintegrating, recalls the pain he endured from his hair transplants: “I know he was conscious of his looks – Mia was very young, I was very young – and he was trying to hold on to his youth.” He treated her perfectly, she says, until he hired a prostitute for a threesome, at which point she walked.

If the first volume was dominated by the star-crossed marriage of Sinatra and Ava Gardner, the defining affair of its successor is the one linking the singer with John F Kennedy. Both relationships were asymmetrical, ending unhappily for Sinatra. At least his distress over losing Gardner, as the great arranger Nelson Riddle said, taught him how to sing a torch song. Rejection by Kennedy, followed by a snub from Hubert Humphrey, turned a lifelong Democrat into an active supporter of Ronald Reagan.

In a book studded with brassy crescendos, the long climax occurs roughly between 1963 and 1966, when Sinatra finds himself confronted by crises on all sides. The tide that carried the Rat Pack – that celebration of male entitlement – has ebbed. His prized friendship with Kennedy, based on his ability to procure votes and women, dissolves on the advice of the new president’s brother Bobby, the US attorney general, who is going after organised crime and humiliatingly vetoes a stay at the Palm Springs house. The FBI’s discovery of Sinatra’s links to the Chicago mobster Sam Giancana costs him the gaming licence that allowed him to operate his own Lake Tahoe resort casino, the Cal-Neva Lodge, and to own a piece of the Sands, the Las Vegas hotel whose 350-seat Copa Room provided his most fitting stage. His son, Frank Jr, is kidnapped. And the arrival of Farrow represents not just a genuine mutual attraction of two seemingly disparate human beings but, to his friends and the public, a symbol of his sudden identity crisis.

The man who would really do justice to this electrifying story is the Don DeLillo of Libra and Underworld. Kaplan is not in that league, but he is certainly more than a hack. Occasionally, while briskly marshalling such a huge volume of material, he finds a memorable phrase. “He moved on a bubble of agitation, always with a pack, searching for the next amusement,” he writes. In Kennedy, he observes, Sinatra knew he had met “the one other man in America whose connection with the nation’s dream life was as deep and powerful as his own”. The clash between the singer’s first and second wives – Nancy, the mother of his three children, and Ava, the love of his life – “was the birthplace and the essence of his bad conscience”. But a weakness for portentous single-line payoff paragraphs (example: “How could he be so big and feel so small?”), while giving a zing to the narrative, lets him down badly on the last page.