One of the glories of the American Century was that phenomenon known as American popular song. The songs now known as standards – which are still performed, not as museum pieces but as a living part of American popular culture, eighty or ninety years after they were composed – were not just incidental ornaments: they helped to shape the sensibility that made America great.

Before the Golden Age of American Song, American popular music was defined by folk songs of the frontier like “I've Been Working on the Railroad” and by homespun tunes such as Stephen Foster's (1826-64) “Oh! Susanna” and “Jeanie with the Light Brown Hair.” These songs still have their charms, but they evoke ways of life long since gone with the wind. Today they're played, and appreciated, as period pieces.

But beginning around the time of the First World War, and during the transformational decade of the 1920s, a new breed of American songwriters – most of them immigrants or the children of immigrants, almost all of them New Yorkers, and the majority of them Jewish – effected a sea change in American song. Drawing on such diverse sources as black American folk songs and the music of the shtetls, they created a new kind of popular song that was marked by an urban energy and edge and that sought to be the music of the new, modern America – a country that was suddenly the leading world power and, thanks to Hollywood, the leading edge of international popular culture. While never coarse or clinically explicit, these songs were far more open about sex than their predecessors – and far wittier. Over the course of the 1920s, they became increasingly sophisticated. They reached a peak in the 1930s and 40s. And then, beginning in the 1950s, they became overshadowed, and eventually crushed, by rock-and-roll.

This story has been told before – in fact, I've written about it myself. But what hasn't been emphasized is the degree to which this is a story not just about music but about American society and culture. It's no coincidence that the Golden Age of American Song was also, in many ways, the Golden Age of America itself. For the songs of that era embodied the values that made America great. They reflected those values, but also helped inculcate them. In a time of new freedoms and mobility and urbanization, these songs taught young people to enjoy their freedoms – but to exercise them in civilized ways. These songs, most of them love songs, gave young people permission to feel deeply, and did so by the very act of giving them the words for those feelings. They taught them, moreover, to laugh at those feelings. And they taught them to control those feelings: for while they often hinted at something more than chaste courtship, they depicted a world in which such matters, by their very nature, involved courtesy, decency, sympathy, and self-discipline.

In a word, American pop music was, especially for young men, civilizing: it taught them, without them even realizing it, how to channel their sexual drives and feelings into appropriate means of expression and appropriate conduct; it taught them boundaries, in both word and deed. It told them that it was perfectly acceptable for grown men to weep into their beer over a lost love (“One for My Baby”) but, by sheer omission of reference to certain kinds of language and behavior, made it clear to them that there are certain ways in which a gentleman simply does not treat a lady and certain ways in which he does not address her. (Consider, by contrast, such hits of today as the rap song “Blow My Whistle, Baby.”) Those songs also presented young women with a view of the world – and, specifically, of love and sex – that was radically different from the view presented by songs that tell today's young women that, for example, “they only want you when you're seventeen; when you're twenty-one, you're no fun.”

It wasn't just the words. The music of nineteenth-century American popular music, following familiar melodic, harmonic, and rhythmic patterns, was essentially free of surprise. The popular music of the 60s and afterwards would be increasingly harsh, dissonant, and aggressive. Both pre- and post-Golden Age popular song was generally simplistic in its melodies and harmonies, as well as in its lyrics. The Golden Age, by contrast to the periods that preceded and followed it, brought out a melodic invention, harmonic richness, and lyrical creativity that would not be seen before or afterwards. This, too, had its psychological effects.

One remarkable thing about the songs of the Golden Age is that they worked for both native Americans and immigrants alike. Take my mother, whose parents, born in a South Carolina burg at the turn of the twentieth century, were steeped in Methodist and Baptist hymns and in the quaint-sounding popular songs of the day (many of them barely distinguishable from hymns). For them, the songs of Rudy Vallee, Bing Crosby, and Showboat were a revelation. My mother (born in 1927) has talked about the revolution that the music of black performers effected in her Jim Crow town when she was a kid. Decades later, she would remember bonding with the family's African-American housekeeper over their shared fondness for Sarah Vaughan's recording of “Black Coffee.” This was new. When my mother, by then a Sinatra-worshipping bobby-soxer, left home for Los Angeles, and then, again, for New York, shaking off of her feet the dust of small-town prejudice, it was largely because she was drawn to these places by the Golden Age songs she loved.

Meanwhile, for my father (born in 1920), who was the son of Polish immigrants and who grew up on the Lower East Side of New York, the Golden Age songs served as a means of assimilation. Offering sentiments he could identify with and images he recognized, these songs (just like his favorite novelists, Sinclair Lewis, Booth Tarkington, and John O'Hara) helped introduce him to the America beyond his household and his immigrant neighborhood. Part of what made these songs seem familiar to him, even as they helped bring him more closely into the American family, was that many of the people who'd written the songs had, in fact, grown up in the same streets in which he'd grown up and were, like him, the children of immigrants.

The typical songs of the Golden Age affirmed, on the one hand, a healthy American patriotism (Irving Berlin's “God Bless America”) and, on the other, a genuine acceptance of cultural diversity and a friendly fascination (albeit often superficial and exoticizing) with other countries and cultures (just off the top of my head: “Moon over Burma,” “It Happened in Monterey,” “Flying Down to Rio”). The lyrics of Lorenz Hart (“Manhattan”), Cole Porter (“Down in the Depths on the Ninetieth Floor”), and others made every middle American a New Yorker; the lyrics of Johnny Mercer (“Travelin' Light”) and Hoagy Carmichael (“Georgia on My Mind,” “Up a Lazy River”) made every Brooklyn kid a country boy.

Never had so many popular songs been so sophisticated – and they encouraged young people to want to be as sophisticated as the songs themselves. Songs like “Penthouse Serenade” and “These Foolish Things (Remind Me of You)” – with its references to long-distance journeys by air and pianos being played in the adjoining flat – painted pictures of a well-heeled urban life worth striving for. Songs like “A Foggy Day in London Town” and “April in Paris” – a far cry from Foster's “Old Folks at Home” and “Old Kentucky Home” – made at least some young people want to be globe-trotters, citizens of the world, familiar with glamorous foreign capitals. These songs instilled ambition. They fostered dreams. (Consider, by contrast, a rap song of the present day, such as “I Wanna Be a Millionaire So Fuckin' Bad,” which also fosters dreams – but not necessarily socially positive ones.)

Songs taught young people that certain things should be taken seriously, but that it was also permissible to doubt, to criticize – and to make fun. Johnny Mercer, a lyricist who could write a serious, even solemn, song like “Dearly Beloved,” which essentially celebrated love as a sacrament and came to be a staple of wedding ceremonies, also wrote plenty of songs that were downright cynical about love and rich in Schadenfreude over the heartbreak of ex-lovers (“Goody Goody,” “I Want to Be Around to Pick Up the Pieces when Somebody Breaks Your Heart”).

Meanwhile Berlin's “White Christmas” and “Easter Parade,” and other now-standard Golden Age songs – “Have Yourself a Merry Little Christmas,” “The Christmas Song (Chestnuts Roasting on an Open Fire),” “I'll Be Home for Christmas” – slyly tweaked Christianity's holy days, turning them into national holidays for a secular society populated by Americans from a wide variety of faith backgrounds. A song like Sinatra's “The House I Live In,” moreover, explicitly equated American patriotism with religious, ethnic, and racial tolerance, while Rodgers and Hammerstein, the century's most successful Broadway songwriters, made a special cause of teaching racial tolerance, doing so very effectively – and without ever seeming intolerably preachy – in songs like “You've Got to Be Taught.”

Golden Age songs also helped Americans, as the years went by, to deal with their dramatically changing circumstances – helped cue, that is, healthy and responsible ways of dealing with the Roaring Twenties, the Great Depression, and World War II, and with the severe transitions from one of these periods to another. During the Depression, songs like “Brother, Can You Spare a Dime?” didn't shy away from the despair of those who had been thrown out of work; more common, however, were songs like “Life Is Just a Bowl of Cherries,” “We're in the Money,” “With Plenty of Money and You,” and “Happy Days Are Here Again,” which sought to cheer everybody up by reminding them that they were all in it together, by arguing that life is too mysterious to be taken too seriously, by claiming that the Depression was already as good as over, and/or by suggesting that prosperity was just around the corner.

Then, in the Forties, songs like “Don't Sit under the Apple Tree,” “I'll Walk Alone,” and “It's Been a Long, Long Time” directly addressed the long periods of separation caused by the war and helped shape a whole wartime sensibility – decent, dedicated, dignified, determined – for parted lovers and married couples, and indeed for the country as a whole. Never had so many romantic ballads been so heartbreaking: “My Heart Tells Me,” “The More I See You,” “You'll Never Know,” “My Shining Hour.” Meanwhile songs like “Boogie Woogie Bugle Boy,” “Praise the Lord and Pass the Ammunition,” and “Comin' in on a Wing and a Prayer” made it clear that there was room in the war effort for lighthearted optimism, while songs like “G.I. Jive” and “This Is the Army, Mr. Jones” made it clear that there was even room for outright humor.

After the Golden Age came the music of protest, of Vietnam and hippiedom and the civil-rights movement and beyond. Once rock-and-roll had knocked the Golden Age songs of Hollywood and Broadway off their perch, innumerable other musical subgenres crawled out of the shadows or came into being, among them bluegrass, disco, electronica, synthpop, ska, zydeco, rockabilly, techno, hip hop, house, trance, garage, funk, R&B, soul, gangsta rip, heavy metal, punk rock, and emo. These developments were hardly without a plus side: all kinds of good music was born. But the merit was all too often outweighed by the sheer decibel level and, in many cases, by vulgarity, incoherence, and brutality. In any event, as American musical tastes fragmented, so did American society and culture. Songs had once bound Americans together – and encouraged them to bind together. Now, all too frequently, they defined Americans in opposition to one another.

And yet in recent years, stars like Rod Stewart, Bette Midler, and Cyndi Lauper – all of whom became famous singing various types of contemporary pop or rock music – have recorded albums of songs drawn from the Great American Songbook. Why? What has motivated them to reach back into a treasury of music from before their own time? What have they sought there, and found there? Aside from the sheer excellence of the songs, which (unlike many modern hits) don't depend on a certain voice or arrangement to make them sound terrific, I suspect that as they grew older they were drawn increasingly to the values conveyed by these songs that were written before they were born – values that are, in many cases, very far indeed from those communicated by the music with which these entertainers made their names and fortunes.