Here's the reason for the inflection point (circa 1975). Several publishers decided to do the unthinkable - give best seller treatment (hardbacks, displays, advertising, promotional tours, ect.) to top genre writers.

And that would be because of . . . Star Wars (1977). Prior to SW, SF was a backwater in the movie industry, with "b movie" pretty much defining the genre (with the occasiaonal "arty" film like 2001, which didn't have the kind of marketing appeal of SW).

The publishing industry has followed trends in movies for a long, long time. It's good business.

Put me down as another who's not seeing a clear cut-off. There's plenty of popular fiction with at-best middling literary value throughout the list.

When I read the OP I thought, "What a cool idea!" When I looked at the actual list I thought, "I'm glad he's doing it so I don't have to." There's only so much Grisham I can read and Jonathan Livingston Seagull again? No thanks.

I'm sure somebody accessing the US Copyright office can come up with more detailed records to document the trend but it is clear that:

1- Sometime past 1950 (probably around 1960) the number of books published in the US exploded by an order of magnitude (paperback originals being the likely driver).

2- Some time past 2003 (probably around 2009) the number of books published in the US exploded by *another* order of magnitude (the mainstreaming of ebooks being the likely driver).

3- The number of litfic writers in the US has *not* exploded by an order of magnitude, much less two, in the past 8 decades.

4- The power of literary establishment gatekeepers and endorsers is much diminished since their heyday in the 50's.

I would suggest that the growth in published fiction titles over the past 8 decades has come almost exclusively in the mass-market genres and not in the litfic genre so that the number of "literary" titles published per year are more of less constant while the number of titles in the other genres has grown with the country's population, literacy rates, and market forces.
I would note that a "paperback original" "literary masterpiece" has always been pretty much an oxymoron and that a self-published "literary masterpiece" may exist but its chances of selling in 50 shades or Wool numbers are pretty low.

Bottom line: in the days of hardcover-only originals and 200 new titles a week (average) it would be a lot easier for an establishment-endorsed litfic title to become a bestseller than it would be today when we're looking at something like 10,000 new titles a *day*. (And note that the total number of books sold was much, much lower in the pre-paperback era.)

LitFic has always been championed by the traditional gatekeepers but when the gatekeepers lose their clout, their power to champion anything other than lowest-common denominator cash-cows (charity starts at home) is pretty much history.

So in that respect, yes, the literary establishment is in decline. Not because their grade of product isn't being produced and consumed in traditional numbers, but because everything else is smothering its visibility. Look around and you'll fine as many literary masterpieces coming out as in any decade past and their sales numbers will likewise be about the same. Was isn't the same is everything else.

Get used to it: we are now in an publishing economy of abundance and the challenge is no llonger getting published but getting noticed. And in the visibility game not even the gatekeepers of yore have much influence.

SteveEisenberg: Whoops, hoist by my own petard. Thank you for pointing out my error.

BelleZora, latepaul, HarryT: I don't dispute there are plenty of forgettable authors on the list prior to the 1970s, but there are also a surprising number of Nobel and Pulitzer prize winners whose works are regarded as classics, whereas for more than thirty years afterwards, the top spots are dominated by mass-market thrillers. Without even entering into judgments of quality, I think it's difficult to maintain that no change occurred at all.

BWinmill: Maybe, but I have never heard of a massive shift in American educational attainment or literacy rates occurring in the mid-to-late 20th century. Seems to me that market forces are more likely to have shifted than demographics (see below).

Ralph Sir Edward: Interesting. Do you have any insight into the change of policy with regards to Dune? Was it a happy accident or a deliberate manoeuvre? And why in 1976, ten years after the original Dune?

taustin: I was thinking this, as well.

fjtorres: Thank you for those figures. Do you happen to know more about why there was a massive explosion in published titles after 1950? I guess what I want to know is: how did American publishing transition from a relatively small (and perhaps more curated) industry to a mass market free-for-all? I also think that your observations about numbers need to be paired with those above them about taste-making. In an embarrassment of riches, promotion is king.

Incidentally, your post reminded me of this passage by Aldous Huxley, which I cannot refrain myself from posting here:

"Advances in technology have led ... to vulgarity.... Process reproduction and the rotary press have made possible the indefinite multiplication of writing and pictures. Universal education and relatively high wages have created an enormous public who know how to read and can afford to buy reading and pictorial matter. A great industry has been called into existence in order to supply these commodities. Now, artistic talent is a very rare phenomenon; whence it follows ... that, at every epoch and in all countries, most art has been bad. But the proportion of trash in the total artistic output is greater now than at any other period. That it must be so is a matter of simple arithmetic. The population of Western Europe has a little more than doubled during the last century. But the amount of reading—and seeing—matter has increased, I should imagine, at least twenty and possibly fifty or even a hundred times. If there were n men of talent in a population of x millions, there will presumably be 2n men of talent among 2X millions. The situation may be summed up thus. For every page of print and pictures published a century ago, twenty or perhaps even a hundred pages are published today. But for every man of talent then living, there are now only two men of talent. It may be of course that, thanks to universal education, many potential talents which in the past would have been stillborn are now enabled to realize themselves. Let us assume, then, that there are now three or even four men of talent to every one of earlier times. It still remains true to say that the consumption of reading—and seeing—matter has far outstripped the natural production of gifted writers and draughtsmen. It is the same with hearing-matter. Prosperity, the gramophone and the radio have created an audience of hearers who consume an amount of hearing-matter that has increased out of all proportion to the increase of population and the consequent natural increase of talented musicians. It follows from all this that in all the arts the output of trash is both absolutely and relatively greater than it was in the past; and that it must remain greater for just so long as the world continues to consume the present inordinate quantities of reading-matter, seeing-matter, and hearing-matter."

Write a book that people want to read instead of a book that you think people ought to read, and you just might sell a lot of books. I don't see anything shocking about the Silmarillion being the #1 book of 1978. Lord of the Rings sold more than 150,000,000 copies and the Hobbit sold more than 100,000,000 copies. LotR has been sneered at for quite a long time, but in recent years has gained grudging acceptance as being a worthwhile book. If there were a turning point, it might have been Airport in 1968, a fairly standard thriller. It's also not terribly surprising that the novelization of Return of the Jedi was the best-selling book in 1983, that's when the movie came out, and it was an enormous blockbuster.

Write a book that people want to read instead of a book that you think people ought to read, and you just might sell a lot of books. I don't see anything shocking about the Silmarillion being the #1 book of 1978.

I do, actually, although I can see how it came about. A significant number of people who read LOTR will have bought it sight unseen, thinking it was 'more LOTR'. Well, it was, in a sense, but it sure wasn't the same...

fjtorres: Thank you for those figures. Do you happen to know more about why there was a massive explosion in published titles after 1950? I guess what I want to know is: how did American publishing transition from a relatively small (and perhaps more curated) industry to a mass market free-for-all? I also think that your observations about numbers need to be paired with those above them about taste-making. In an embarrassment of riches, promotion is king.

Toward the end of the 1940s, with so many new entrants in the booming paperback business, magazine and comic book publisher Fawcett Publications gave the industry a new idea to mock: paperback originals. Up to that point, paperback publishers had limited themselves to reprinting hardcover titles or publishing quick, timely original nonfiction such as the wartime bestseller What’s That Plane, a guide to identifying American and Japanese aircraft.

Fawcett was saddled with a distribution agreement that prevented it from publishing and distributing its own reprints of hardcover titles. Seeking to exploit a loophole, editor in chief Ralph Daigh announced that Fawcett would begin publishing original fiction in paperback form beginning in February 1950.

“Successful authors are not interested in original publishing at 25 cents,” Freeman Lewis, executive vice-president of Pocket Books said. Hardcover publisher Doubleday’s LeBaron R. Barker claimed that the concept could “undermine the whole structure of publishing.” Hardcover publishers, of course, had a vested interest in maintaining the status quo. They were still receiving 50 percent of the royalties by selling reprint rights.

Fawcett silenced the skeptics by selling more than nine million copies within six months. Authors did the math, and writers of genre fiction—thrillers, Westerns, and romance especially—jumped at the opportunity to write paperback originals. Still, “serious” literary writers insisted on staying in the hardcover market for the prestige, and critics in turn declined to review paperback originals. Clearly, the stigma was still there.

Boldface mine.

Start with that.
Add in the baby boom generation (1946+) coming of age for genre reading (~14 for SF) and 1960 is (yet again) a watershed year.

Note, btw, that there was (effectively) no market for SF novels prior to 1950 other that serialization in the magazines. Other genres had abundant outlets in the pulps.

Which brings up the point that there has *always* been a market for popular fiction outside the traditional publishing establishment. Starting with the penny dreadfuls and the pulps there is a long tradition of what Huxley and co would dismiss as vulgar fiction.

The first "pulp" was Frank Munsey's revamped Argosy Magazine of 1896, about 135,000 words (192 pages) per issue on pulp paper with untrimmed edges and no illustrations, not even on the cover. While the steam-powered printing press had been in widespread use for some time, enabling the boom in dime novels, prior to Munsey, no one had combined cheap printing, cheap paper and cheap authors in a package that provided affordable entertainment to working-class people. In six years Argosy went from a few thousand copies per month to over half a million.[1]

Street & Smith were next on the market. A dime novel and boys' weekly publisher, they saw Argosy's success, and in 1903 launched The Popular Magazine, billed as the "biggest magazine in the world" by virtue of being two pages longer than Argosy. Due to differences in page layout, the magazine had substantially less text than Argosy. The Popular Magazine introduced color covers to pulp publishing. The magazine began to take off when, in 1905, the publishers acquired the rights to serialize Ayesha, by H. Rider Haggard, a sequel to his popular novel She. Haggard's Lost World genre influenced several key pulp writers, including Edgar Rice Burroughs, Robert E. Howard, Talbot Mundy and Abraham Merritt.[2] In 1907, the cover price rose to 15 cents and 30 pages were added to each issue; along with establishing a stable of authors for each magazine, this change proved successful and circulation began to approach that of Argosy. Street and Smith's next innovation was the introduction of specialized genre pulps, each magazine focusing on a genre such as detective stories, romance, etc.[3]

At their peak of popularity in the 1920s and 1930s, the most successful pulps could sell up to one million copies per issue.

Quote:

The pulp format declined from rising expenses, but even more due to the heavy competition from comic books, television, and the paperback novel. In a more affluent post-war America, the price gap compared to slick magazines was far less significant. In the 1950s, men's adventure magazines began to replace the pulp.

Quote:

The collapse of the pulp industry changed the landscape of publishing because pulps were the single largest sales outlet for short stories. Combined with the decrease in slick magazine fiction markets, writers attempting to support themselves by creating fiction switched to novels and book-length anthologies of shorter pieces.

What the paperback originals revolution did was replace the economics (and limited shelf-life) of magazines that the pulps followed with the economics of books, allowing for longer shelf-life. It also replaced the work-for-hire model of many (most?) of the pulps for the book industry's licensed freelance content acquisition model.

To a large extent what happened in 1950-60 and is happening again *now* is a technology-driven change in distribution model, with a parallel change in economic model, that redefines exactly what a book is. Once upon a time it was a hardcover cultural item. Two paradigm shifts later, we are seeing ebooks redefine what a book is.

And as usually happens, the prevailing orthodoxy runs two paradigms behind reality.

And that would be because of . . . Star Wars (1977). Prior to SW, SF was a backwater in the movie industry, with "b movie" pretty much defining the genre (with the occasiaonal "arty" film like 2001, which didn't have the kind of marketing appeal of SW).

The publishing industry has followed trends in movies for a long, long time. It's good business.

Star Wars was first published six months before the movie was released. The author was listed as George Lucas but, was actually written by Alan Dean Foster. I bought it because it sounded like a nice read. It said soon to be a major motion picture but, at the time the book did not sell well. After the movie release they released it again and it hit number one on the NY Times Bestsellers Chart. The book also differed in some respects to the movie. I still have my copy.
Apache

And that would be because of . . . Star Wars (1977). Prior to SW, SF was a backwater in the movie industry, with "b movie" pretty much defining the genre (with the occasiaonal "arty" film like 2001, which didn't have the kind of marketing appeal of SW).

The publishing industry has followed trends in movies for a long, long time. It's good business.

Not entirely true... SF authors were doing publisher sponsored signing sessions prior to SW, I have many signed books from such sessions done locally prior to this date. It's actually the later SF genre readers that tend to think SF began with SW being the be-all and end-all of defining SF but SW just widened the audience... publishers were even sponsoring authors and arranging releases to coincide with SF conventions prior to SW but SF was never the total backwater that many believed otherwise there wouldn't have been such a continuous line of SF publications in the post-war years...

Star Wars was first published six months before the movie was released. The author was listed as George Lucas but, was actually written by Alan Dean Foster. I bought it because it sounded like a nice read. It said soon to be a major motion picture but, at the time the book did not sell well. After the movie release they released it again and it hit number one on the NY Times Bestsellers Chart. The book also differed in some respects to the movie. I still have my copy.
Apache

If your copy is still in good condition then you might want to know that it is now a serious collectible and can get up to a three figure price tag especially if signed by Foster and even more if it has a Lucas sig as well...

I also have an original Starship Troopers. When that stupid movie came out people were offering ridiculous amounts for them. I will keep mine and if my son wishes to sell them after I am gone that is OK with me.
Apache

Ok all, here we go. If my memory is bad and some of this is garbled, please correct me, but please don't be snarky about it. I'm doing the best I can...

First, prior to the early 1970's, genre fiction was <not> long. Publishers were tied to the 170-300 page length, or else it just didn't get published. The only S/F writer who had done anything at greater length was Heinlein, with Stranger. And if you read his letters (Grumbles From the Grave), he sweated bullets worrying that the publisher wouldn't get their money back. (He even agreed to a S/F bookclub arrangement, just to help the publisher recoup their initial costs. He (like all other S/F writers) got paid peanuts on the bookclub edition.) And Stranger was only 407 pages, and Heinlein was hands-down the best selling S/F writer at the time.

Dune couldn't get published at all in the early 60's, That's the gospel truth! The only reason anybody ever heard of it was that 1. John W. Campbell liked it, and ran it as a serial for 12 months(!). and 2. an editor at a car repair manual was a huge S/F fan, and managed to sweet-talk the firm into publishing it (Chilton - 1966, more commonly know for such scintillating titles as Fix Your Chevrolet or Fix Your Volkswagon) (Sidebar - early paperbacks had trouble holding together if they were too long - glue limitations, which nobody wanted fixed (throwaways, remember?))

Still, in paperback, it sold and sold, and sold. Add in LOTR in fantasy (at long length), and Stranger, there seem to be a market for longer length paperback books. So various authors, wrote BIG books, and most of them sold. (Examples - Brunner's Stand on Zanzibar 1968 and The Sheep Look Up 1970, Heinlein's Moon 1965, I Will Fear No Evil 1970, and Time Enough For Love 1973, Niven and Pournelle's A Mote In God's Eye 1974, and Delaney's Dhalgren 1975)

But these were still treated as second class citizens, like all genre fiction. They made the publishers money, but they weren't literature...

In and around 1975, some publisher wondered what would happen if they gave one of these long books (which seemed to sell, if they were any good at all) the "Blockbuster" treatment, just like they did for the general bestsellers. Why? Because, somebody said, if they could get people to buy them as first run hardcovers, they'd made a <lot> more money. So the green-light was given to do one book that way. If it failed, they could survive the write-off, and if it succeeded, they would be head-and-shoulder ahead of their competition.

Well, as they said in Blazing Saddles - "Who'se it gonna be?" First choice would have been Heinlein. He had the best commercial track record. But Heinlein was gravely ill, and couldn't write. Asimov was doing non-fiction, mostly, and his style made a poor bet at the time. Same for Clarke. Brunner was too British for blockbuster American tastes. What were you left with? Hmmm... Herbert had done well with Dune, and was writing sequels, how about one of those? It would have the advantage of a built-in audience (just like Hollywood sequels), and if non-fans got lost, shrug, they had bought the book...

(Please note - this was done a couple of years before Star Wars came out...)

The one-armed bandit paid off - big time! And ever big name science fiction writer started getting the "Best Seller" treatment. Some sold, some didn't, but the publishers kept at it for decades.

And that's how genre became big time. Even Louis L'Amour got to do blockbuster sized books at the end of his career...

Star Wars was first published six months before the movie was released. The author was listed as George Lucas but, was actually written by Alan Dean Foster. I bought it because it sounded like a nice read. It said soon to be a major motion picture but, at the time the book did not sell well. After the movie release they released it again and it hit number one on the NY Times Bestsellers Chart. The book also differed in some respects to the movie. I still have my copy.

There was a rumor that wouldn't die for a long time that Star Wars the movie existed because Star Wars the novel hadn't sold very well.

I wish I had an original, but the one I read was in a classroom library and was in very bad shape.

Quote:

Originally Posted by Apache

I also have an original Starship Troopers. When that stupid movie came out people were offering ridiculous amounts for them. I will keep mine and if my son wishes to sell them after I am gone that is OK with me.

I read somewhere (tvtropes?) that the Starship Troopers movie was actually an attempt to remake Gung Ho! for the '90s. Since they couldn't use the original film's plot, they went with something splatter-heavy and a lot more socially acceptable (bugs instead of Japanese).