Writing for The New York Times in 1967, just weeks after the album's arrival on June 1, Goldstein actually praised the Beatles' closing epic "A Day in the Life," calling it "one of the most important [John] Lennon-[Paul] McCartney compositions" and "a historic pop event." The rest of Sgt. Pepper, however, left him cold.

"The sound is a pastiche of dissonance and lushness," Goldstein said back then. "The mood is mellow, even nostalgic. But, like the cover, the overall effect is busy, hip and cluttered. Like an over-attended child, Sgt. Pepper is spoiled."

As critical momentum definitively swung the other way, Goldstein's broad side became one of the most notorious in all of rock criticism – an example, it has long seemed, of one guy getting it all wrong. But there were also others – including Nik Cohn. "It wasn’t fast, flash, sexual, loud, vulgar, monstrous or violent," he wrote in a contemporary piece for Pop From the Beginning. "It made no myths."

Even some critics who praised it, like Rolling Stone's Greil Marcus, were apt to add backhanded compliment along the way. "Sgt. Pepper, as the most brilliantly orchestrated manipulation of a cultural audience in pop history, was nothing less than a small pop explosion in and of itself. The music was not great art; the event, in its intensification of the ability to respond, was.”

Other outliers include Jim DeRogatis, pop music critic for the Chicago Sun-Times: "The Beatles [had] given us 39 minutes and 52 seconds of rather unremarkable, uninspired music," he wrote in 2004, "with a central theme that’s conservative, reactionary and retrogressive." The Guardian's Richard Smith says Sgt. Pepper is "an excruciating lesson in orientalism, why music hall died out, why making records on drugs isn't always a good idea, and why you shouldn't let Ringo [Starr] sing a number." Keith Richards, both a contemporary and longtime Beatles rival, memorably called it a "mishmash of rubbish."

Yet Goldstein's review remains the most cited pan of an otherwise broadly celebrated album. Given an opportunity to recant in the pages of the The Village Voice a few weeks later in 1967, Goldstein held his ground.

"When the slicks and tricks of production on this album no longer seem unusual, and the compositions are stripped to their musical and lyrical essentials, Sgt. Pepper will be Beatles baroque – an elaboration without improvement," he said. "In Revolver, I found a complexity that was staggering in its poignancy, its innovation and its empathy. I called it a complicated masterpiece. But in Sgt. Pepper, I sense a new distance, a sarcasm masqueraded as hip."

Listen to the Beatles Perform 'A Day in the Life'

Robert Christgau, in a December 1967 piece for Esquire, came to his fellow critic's defense – and he too offered some reservations about the album.

"Goldstein was disappointed with Sgt. Pepper," Christgau said. "After an initial moment of panic, I wasn't. In fact, I was exalted by it, although a little of that has worn off. Which is just the point. Goldstein may have been wrong, but he wasn't that wrong. Sgt. Pepper is not the world's most perfect work of art. But that is what the Beatles' fans have come to assume their idols must produce."

Robert Hilburn, looking back at the album in the Los Angeles Timesin 1987, stood by Goldstein's 20-year-old review. "Sgt. Pepper was a monumental moment in pop culture, the zenith of a generation's absorption with the realigning of the social order," he wrote. "As such, it remains a landmark work that is an essential part of any rock library. On a strictly musical level, however, most of the songs in Sgt. Pepper are just what Goldstein said: 'undistinguished.'"

Only later, as part of his 2015 memoir Another Little Piece of My Heart, did Goldstein reveal that the stereo used for that long-ago review had a busted left speaker. “That’s f---ed up,” Christgau shot back in a 2017 talk with the Washington Post. “You don’t review a record on a stereo that isn’t working, certainly not a record of that consequence.”

Goldstein refused, however, to blame the equipment for his negative reaction to Sgt. Pepper's Lonely Hearts Club Band. "I didn't get it because the record wasn't really rock. It didn't belong in any genre that I could think of, so I didn't know how to receive it. I think I was quite a puritan when I reviewed that album. I'm much less puritanical about music now, and I think so are most music listeners," Goldstein told Rolling Stone in 2015. "It felt chaotic and cluttered, too showy and all of that. There are people who think I was right, and maybe I was. I don’t know. There’s no right and wrong about a piece of music. All there is, is the consensus."

The consensus is that Sgt. Pepper is the greatest rock album ever made. Still, Hilburn isn't so sure in the decades that have passed.

"Where the album once seemed to define a culture, it now stands as a curio from a past age," he said. "Equally important, the innovation – which inspired hundreds of other bands to think in more artful terms (a mixed blessing indeed considering the pretentiousness of the early-'70s progressive rock movement) – no longer camouflages the weakness in material."

He feels the album is bookended with strong material, but is particularly critical of the seven tracks between "Fixing a Hole" and "Good Morning Good Morning," writing that they "easily represent the longest stretch of mediocre material that the Beatles committed to vinyl."

Hilburn then drills deeper into one of the project's most controversial legacies. "Sgt. Pepper, for all its spectacle, sounds too artificial and forced, where the best of the Beatles' brilliant earlier work had seemed so spontaneous and alive," he added. "Except in the highlights cited, there's too little sense of real people and real feelings, too much preoccupation with technique – which may explain why so much of the progressive rock movement that was influenced by this album was itself so sterile."