SIAI benefactor and VC Peter Thiel has an excellent article at National Review about the stagnating progress of science and technology, which he attributes to poorly-grounded political opposition, widespread scientific illiteracy, and overspecialized, insular scientific fields. He warns that this stagnation will undermine the growth that past policies have relied on.

When any given field takes half a lifetime of study to master, who can compare and contrast and properly weight the rate of progress in nanotechnology and cryptography and superstring theory and 610 other disciplines? Indeed, how do we even know whether the so-called scientists are not just lawmakers and politicians in disguise, as some conservatives suspect in fields as disparate as climate change, evolutionary biology, and embryonic-stem-cell research, and as I have come to suspect in almost all fields? [!!! -- SB]

Grave indictors:

Looking forward, we see far fewer blockbuster drugs in the pipeline — perhaps because of the intransigence of the FDA, perhaps because of the fecklessness of today’s biological scientists, and perhaps because of the incredible complexity of human biology. In the next three years, the large pharmaceutical companies will lose approximately one-third of their current revenue stream as patents expire, so, in a perverse yet understandable response, they have begun the wholesale liquidation of the research departments that have borne so little fruit in the last decade and a half. [...]

The single most important economic development in recent times has been the broad stagnation of real wages and incomes since 1973, the year when oil prices quadrupled. To a first approximation, the progress in computers and the failure in energy appear to have roughly canceled each other out. Like Alice in the Red Queen’s race, we (and our computers) have been forced to run faster and faster to stay in the same place.

Taken at face value, the economic numbers suggest that the notion of breathtaking and across-the-board progress is far from the mark. If one believes the economic data, then one must reject the optimism of the scientific establishment. Indeed, if one shares the widely held view that the U.S. government may have understated the true rate of inflation — perhaps by ignoring the runaway inflation in government itself, notably in education and health care (where much higher spending has yielded no improvement in the former and only modest improvement in the latter) — then one may be inclined to take gold prices seriously and conclude that real incomes have fared even worse than the official data indicate. [...]

College graduates did better, and high-school graduates did worse. But both became worse off in the years after 2000, especially when one includes the rapidly escalating costs of college.[...]

The current crisis of housing and financial leverage contains many hidden links to broader questions concerning long-term progress in science and technology. On one hand, the lack of easy progress makes leverage more dangerous, because when something goes wrong, macroeconomic growth cannot offer a salve; time will not cure liquidity or solvency problems in a world where little grows or improves with time.

"The speedup in information technology contrasts dramatically with the slowdown everywhere else."

I think this is the key sentence in Thiel's article. The issue is then to what degree the application of information technology to the sciences can alleviate the problem. Interestingly, Kurzweil's thesis is essentially that he sees general speed-up as a product of the other sciences becoming information sciences, but he is clearly much more optimistic about the degree to which this is happening and has already happened.

This seems to be an interesting idea but seems to be off. A lot of historical bubbles have occurred. Many historical bubbles like the infamous Tulip bubble were by many metrics a lot larger than the modern bubbles.

A lot of bubbles over many centuries, perhaps, but without a data series how can one say it's off? After all, we've hit 2 acknowledged bubbles in as many decades (notice that I don't say 'centuries' or 'half-centuries'), and there are more controversial recent ones (commodity bubble with oil or gold? 'Green' stocks? China?) which would push the count up even higher.

I'm not sure that one per decade is unusually many. The US had a lot of bubbles in the 19th century -- canal bubbles, real estate bubbles, railroad bubbles, etc. In all these cases (as with today), there was a real innovation underneath, that bedazzled investors into funding instances of it that didn't make sense.

Tyler Cowen's The Great Stagnation is worth a mention. (It is a fantastic, short, read. I highly recommend it to all Kindle owners on LW). One of TC's points is that we went through several decades (the 1930s excepted) of extraordinary growth, and made plans (revenue projections, budget forecasts, etc.) based on that growth continuing at the same high rate forever. This fits the broad pattern of the Planning Fallacy.

But the 19th-20th century boom had to end eventually, and we still have institutions (public and private) that work on the assumption that the growth will continue. So we'll have a painful period in which we all gradually catch up to the fact that "we aren't as rich as we thought we would be" as Cowen puts it.

Everyone brings up the Internet. But the internet has not changed people's lives as significantly as railroads, telegraphs, plumbing, electricity, radio, automobiles, central heating, air conditioning, telephone, or television did. I don't mean it hasn't changed it as much as those things combined; I mean each one of those had more impact on daily life than did the internet.

Here's the test I use: look at fictional stories set and written in the time before the technologies. How many of the plot elements have become quaint and obsolete due to the technology?

By my reckoning, for the other technologies you listed, my metric only turns up a few inconveniences here and there which are still present on some level today. In contrast, many stories (even as recently as the early 90s) become laughably quaint given the introduction of a well-organized many-to-many communication/information network.

Knowledge is power, and the internet provides it to people in droves. (cheesy, I know, but fair)

That metric's likely to be biased in favour of information & communication technologies, because fiction disproportionately relies on characters having limited information. I doubt there are many 19th century stories hinging on the humble electric washing machine not existing, but it's freed up billions of hours for everyday people regardless.

It's certainly evidence, but it's evidence based on fiction, which favours the sensational, dramatic, and unusual. One way for a TV drama or action film to advance its plot is to have a character get shot. This is far more popular than advancing the plot by having a character spend too long manually washing clothes with a scrub board. But I don't infer that bulletproof vests are a more significant technology than the washing machine! Just that shooting someone's more cinematic than scrubbing clothes.

"Fictional evidence" is, indeed, not evidence of the dangers presented in the fiction. But, in the aggregate, it is evidence of the common fears of the population -- of getting shot, of living a life of drudgery (by presenting a hero who doesn't), and of being hindered by information cut-off.

I would say, then, that the proper way to interpret the washing machine by my heuristic is to look at it as something that takes people slightly closer to living the heroic/fun lives of the protagonist. The internet, by contrast, obviates the entire problem class of information deficiency.

So you're right that there are no books whose plot elements revolve around "washing clothes time-sucks", but most stories casually assume away the slings and arrows of everyday drudgery (home upkeep, childcare). The extent to which a technology makes these dreams a reality is a measure of the relative significance (or, for an inversion, evidence of dystopia as in Brave New World.)

"Fictional evidence" is, indeed, not evidence of the dangers presented in the fiction. But, in the aggregate, it is evidence of the common fears of the population

Yes, and I reckon it's biased evidence of those fears (and the fears in turn are biased indicators of real world importance).

I would say, then, that the proper way to interpret the washing machine by my heuristic is to look at it as something that takes people slightly closer to living the heroic/fun lives of the protagonist. The internet, by contrast, obviates the entire problem class of information deficiency.

Putting aside my quibbles with the heuristic itself, I think this still overrates the Internet, which fails to fill in information gaps as modest as "where did I put my keys?", and underrates the washing machine, which allows us to assume away an entire afternoon of drudgery each week or so. (In fact, household appliances in general can account for most of the 20th century rise in American married women who work, a truly massive social change.)

I've seen those studies and his discussion of them. They all show that people would have to be paid a lot to stop using the internet (and they'd still be benefiting from its side effects), and they'd pay a lot if that were the only option. That indicates a huge consumer surplus, since you don't have to pay a lot for a base level of internet access.

Cowen's only response to any of that is to look at measures that gauge the marginal value of the last unit of internet usage (which of course nets out to zero) is low, when we really care about the consumer surplus it generates, which is completely different from (and reflected in different measures than) marginal value.

I've seen those studies and his discussion of them. They all show that people would have to be paid a lot to stop using the internet (and they'd still be benefiting from its side effects), and they'd pay a lot if that were the only option. That indicates a huge consumer surplus, since you don't have to pay a lot for a base level of internet access.

Do these studies take into account the collective action problem? It's clearly extremely costly to forsake the internet now that everyone expects you to use it, both privately and professionally, so that it would render you an antisocial weirdo and likely unemployable. I certainly wouldn't accept it for anything less than a fortune, but on the other hand, I could definitely see some serious upsides if the state of communication technology were magically reverted twenty years back -- so much that I would have to think carefully about the answer if I were given this offer by some supernatural force.

Also, of course, preferences that are stated rather than revealed should always be taken with extreme skepticism, since they're likely to be heavy on signaling.

Yes, this is the point Robin Hanson made in a blog post on the matter. Still, one can reasonably expect that the results would be unchanged if you permitted e.g. direct communication with friends through the internet but prohibited informational lookups. That would not entail taking on weirdness, and yet people would still have to be paid a lot for it.

Still, one can reasonably expect that the results would be unchanged if you permitted e.g. direct communication with friends through the internet but prohibited informational lookups. That would not entail taking on weirdness, and yet people would still have to be paid a lot for it.

I'd say it would still entail weirdness, unless the exemption is made broad enough to cover anything that would risk weirdness or unemployability -- however then we're back to something resembling the regular internet usage of people who aren't in the habit of wasting time on the web.

RK: Computers are getting faster.
PT: True, but new drugs are coming out slower.
RK: True, but computers are getting cheaper.
PT: True, but travel is getting slower.
RK: True, but the best medical technologies can do more.
PT: True, but the price of access to them is ballooning.
RK: True, but this can be overcome with sufficiently fast growth in a few critical sectors.
PT: True, but it won't be if the current political and economic environment persists.
RK: True, but it won't persist.
PT: Yes it will, unless we do something to reverse it.
RK: Yes, and we will.
PT: ...
RK: I think we still have 55 minutes left.

Today a letter from Einstein would get lost in the White House mail room, and the Manhattan Project would not even get started;

Does he know that Einstein's letter was not mailed? Alexander Sachs was given the letter, setup a personal audience with the President and read it to him.

There are other instances where I am less certain he is wrong but this idea that our present culture is so much worse for scientific research than our culture in the earlier part of the century seems weakly supported to me.

After Roosevelt got the Einstein letter, very little happened. The White House did not actually jump into action or start spending money. The thing was referred to a committee that moved very lethargically. Progress only picked up several years later, once the British, independently, had figured out that a bomb was practical, and had started making noise about it.

That's interesting, but I didn't know that. In any case, that detail doesn't matter because he was obviously being rhetorical - there are no studies showing the White House mail room has unusually high loss-rates. :)

It was rhetorical but he was still wrong - he seems to think science was taken more seriously at that time. It wasn't. Asr already pointed this out - but to add to it even before the "do nothing commission" was started the scientists wrote another letter to the President because he had taken no action on it at all after several months. Then he appointed a commission to do nothing for a good while longer. Then the bureaucracy got started on the org charts and Powerpoints. Er...you get the idea.

In June of 1905, when he submitted "The electrodynamics of moving bodies", he had a PhD in physics and had already published several papers in the same journal. He didn't hold a university post, but he very much a member in good standing of the physics community. I don't see why somebody in an analogous post today would have trouble publishing papers.

The lack of citations is interesting, but I think you're reading too much into it. It shows that scientific publication norms have changed since 1905, but it's not as though Einstein would have been unable to add the appropriate references if the journal had expected it. You might equally well say "the paper couldn't be published today because it's in German, not English".

Einstein's world changing paper, "The electrodynamics of moving bodies" would not be publishable today. He was not an academic, his paper lacked citations, and so on and so forth

In June of 1905, when he submitted "The electrodynamics of moving bodies", he had a PhD in physics and had already published several papers in the same journal

Many of which would also be unpublishable today.

it's not as though Einstein would have been unable to add the appropriate references if the journal had expected it.

The references are so that the editor knows where to send it for peer review and to reference the existing consensus to demonstrate orthodoxy.

Galileo ridiculed what we now call peer review as a team of carthorses.

The effect of peer review is to discover truth through authority, orthodoxy and consensus, without the slow, tedious, and inconvenient necessity of examining the world - and that is why "The Electrodynamics of Moving Bodies" would today be unpublishable.

Given that productivity growth was slow for most of human history, it seems more appropriate to ask why it was so fast for a while. I keep thinking it may have been that manufacturing precision reached a point where we could drink a couple centuries of scientific milkshake.

I'm surprised that Thiel doesn't engage more directly the hypothesis that for the last century (or perhaps century and a half), the living standards in the Western world have been determined by the race between rapid technological progress and somewhat less rapid degradation in the quality of government. To me this hypothesis seems very plausible, even highly probable. Assuming it is true, the interesting question is whether the recent stagnation is due to technological progress slowing down, the quality of government deteriorating faster, or perhaps the quality of government reaching some low point at which even extremely rapid technological progress can't save the day.

Also, these two variables are by no means independent. At the very least, bad government creates perverse incentives that draw smart and entrepreneurial people away from pushing real technical progress and towards rent-seeking. And as another interesting question, could it actually be that technological progress somehow inherently exacerbates the trends towards bad government, thus planting the seeds of its own future doom? I can think of some scary hypotheses along these lines, although they are highly speculative.

I'm surprised that Thiel doesn't engage more directly the hypothesis that for the last century (or perhaps century and a half), the living standards in the Western world have been determined by the race between rapid technological progress and somewhat less rapid degradation in the quality of government.

I think this is intentional.

Your speech in Aspen last month argued that technological change was stagnating, especially in energy. What's the solution? Is it that progress in energy is more difficult than progress in, say, chip design, so we have to be more patient?
Thiel: It goes back to the why-things-have-slowed-down question, which is the one that I've tried to avoid.

Unfairly tried to avoid?
Thiel: I'm not sure it's unfair. Because as soon as you get to the "why" question, it gets much more controversial. It makes people lose sight of the what, which is the thing I want people to pay attention to. I think that if we could get people to agree that there's a really big problem in innovation, then we can have a constructive conversation on what to do about it. As soon as we fixate people on why there's been an innovation slowdown, if you're on the right you can blame environmentalism, if you're on the left you can blame Reagan and financial engineering. And we don't even get to the question of whether a slowdown has happened. That's why I'm somewhat resistant to going straight to the ideological or political question.

Clearly, a proper justification for that claim would have to be given at enormous length, and it would have to involve all sorts of ideologically charged issues. But to try to put it in a nutshell, the government has been meddling in an ever increasing range of things, and this meddling is mostly guided by a mix of bureaucratic perverse incentives and straight-out ideological delusions, each operating in a frightful positive feedback loop with the other. (Mind you, this isn't a statement of some libertarian principle that government meddling is always bad, but purely a factual statement that destructive meddling has been increasing.) At the same time, even the basic and uncontroversial functions of government have been increasingly corrupted by these ever worsening perverse incentives and ideological delusions.

Or to put it another way, what we've been seeing for several generations is a move from government based on custom and precedent to an ideological bureaucratic state. The former was indeed bad in all sorts of ways, but at least it had a decent grip on reality in essential things, as long-standing customs and precedents usually do. The latter, however, is driven by ideology and bureaucratic incentives into constant and ever more severe clashes with reality.

I wouldn't call that first paragraph a degradation in quality. I'd call that too much government. I thought you meant politicians were getting dumber or more corrupt.

The point is that we're having too much (and increasingly more) bad government, this aside from the question of whether one accepts the libertarian principle that more government is always worse. As for politicians (in the sense of people running for elective offices), they are only one relatively small element within the government, and nowadays certainly not the most significant one.

I don't understand the second paragraph.

Try to imagine some real-life examples of getting things done in a straightforward, commonsensical, and customary way, where people's personal incentives and personal responsibilities are more or less aligned with the desired goal -- the way things are normally done in private life and well-run business. This may turn out to be inefficient, unfair, wasteful, irrational, etc. in many ways if scrutinized rigorously from an idealistic perspective, but it's still fundamentally sound and effective. Then try to imagine attempts to get things done by large faceless bureaucracies lacking any locus of personal responsibility, or by people driven by ideology that for them trumps reality whenever there is any conflict. (Or by some diabolical mixture of the two.)

Basically, my thesis is roughly that the way government works in the English-speaking world has been steadily drifting from the former to the latter way of doing things for a very long time. (Which then influences both the scope of things it attempts to do and the quality of work at whatever it undertakes.)

US tax revenues as a percentage of GDP have been fairly stable over the long term (IIRC), so if government's a problem it's not in the sense of government capturing all the resources. Regulation could be an issue.

The most recent big innovation in energy is fracking. Clearly this has experienced regulatory headwinds, and a pile of environmentalists are looking for rationalizations to stop it, but on the whole, looks to me like a reasonable rate of innovation in energy, which is in substantial part compensating for the exhaustion of hydrocarbon resources. Doubtless we could do better, but it is not obvious that to me that energy innovation is slower than would reasonably be expected in a mature field.

A good measure of innovation is new therapeutic molecular entities, because one can simply count them. By that measure, innovation in the west and Japan has slowed down drastically and alarmingly, though China and Taiwan are speeding up.

I observe ads by Singapore promising a more favorable regulatory environment for drug development. They don't have a drug research industry and are hoping for one. This suggests the existence of an unfavorable regulatory environment for drug development. It is clear that western development of new therapeutic molecular entities has slowed alarmingly.

Nobody engages the topic seriously besides perhaps a handful of bloggers on the internet. To talk about the topic intelligently requires rare traits. An extremely broad swath of knowledge about econ and history, and the ability to not be mind-killed by politics chief among them.

With regard to great achievements enable bad government, yes you find strains of this thought in most of the "cyclical" types of historical analysis, most famous being Spengler.

Nobody engages the topic seriously besides perhaps a handful of bloggers on the internet. To talk about the topic intelligently requires rare traits. An extremely broad swath of knowledge about econ and history, and the ability to not be mind-killed by politics chief among them.

It's not that surprising when you consider that the English-speaking societies have been getting increasingly ideologized for over a century. By this I mean both the increase in ideological uniformity and the tendency to see an increasingly narrow range of topics as non-ideological and thus freely debatable. From the official intellectual institutions, we still get interesting ideas about strictly technical topics that don't involve any ideological issues, but otherwise, these institutions are nowadays hostile to anything but conformist intellectual cant. So it's no wonder that insofar as some real intellectual life persists, it subsists only in obscure underground venues.

(The situation is in fact similar to the former Soviet Union, where the hard sciences worked reasonably well, but one would be foolish to look for much beyond official ideological cant from, say, historians or sociologists, and real intellectual life was happening underground. Of course, the ideological uniformity in the U.S.S.R. was enforced in much cruder and harsher ways, but the more subtle carrots and sticks that the present system offers to intellectuals are, if anything, only more effective.)

What I have in mind is not how successful the hard sciences were in the U.S.S.R., but that the environment for them was qualitatively and fundamentally different from softer fields, in which there was little ground (except perhaps for occasional highly specialized and abstruse topics) where one could discuss things freely without running into the rigid constraints of state ideology. Even if the output of Soviet math or physics was comparably poor, their mathematicians and physicists still had the freedom to do the same rigorous and sound math and physics as their Western colleagues, and many of them indeed did so, though perhaps there were fewer than could have been under a better system.

In contrast, it was outright impossible for a Soviet intellectual to discuss freely and publicly interesting and novel ideas about history, economics, political theory, etc., let alone more general ideas about humanity and human society that draw on all these fields. This would immediately run afoul of the all-pervasive official ideology and its points of disconnect with reality. My thesis is that the Western societies nowadays are in fact in a similar situation, although the ideological enforcement mechanisms are more complex, subtle, and non-obvious. (But they're nevertheless utterly effective in producing ideological uniformity among those whose opinion actually matters, even if the dissenters are typically just marginalized rather than jailed.)

My thesis is that the Western societies nowadays are in fact in a similar situation, although the ideological enforcement mechanisms are more complex, subtle, and non-obvious. (But they're nevertheless utterly effective in producing ideological uniformity among those whose opinion actually matters, even if the dissenters are typically just marginalized rather than jailed.)

Hmm, this is an interesting claim. Can you give concrete examples of fields where you think this is occurring and specific ideas that are being ignored by academia?

The point I'm trying to make is not that there are particular topics where the respectable opinion is so mendacious and delusional that they're impossible to discuss rationally in public, or even in any respectable company. (Although that is clearly true.) I am actually making a stronger claim.

Namely, some such topics exist in every human society, but their scope does vary a lot -- and if their scope is sufficiently wide, an upfront and rational public discussion of some very general and fundamental questions about government, society, and human affairs becomes impossible, since such a discussion would necessarily involve countering some of these established delusions and mendacity. My claim is that at some point during the last few generations, the English-speaking societies have in fact gone past that point. (This also applies to other Western societies, since there are no other ones whose intellectual institutions and traditions have maintained independent continuity through the 1914-1945 cataclysm.)

Again to use the Soviet analogy, imagine having a discussion about general and fundamental trends in modern history in which it would be an unimaginable heresy to open the question of whether the Russian Revolution in 1917 was perhaps not such a great and fortunate event after all, or that maybe Soviet socialism in fact wasn't such a great and liberating improvement on the Czarist government, or that (gasp!) workers are actually better off under capitalism. Clearly, it would be impossible to have a sensible discussion that maintains a solid grip on reality under such constraints, and the result would be a mere rehearsal in ideological cant. What I find is that the effective constraints on respectable public discourse nowadays are, for all practical purposes, equally suffocating -- although of course they are enforced by much less crude, explicit, and violent means, which rarely go beyond shunning and marginalization of dissenters.

(Also, here I don't have in mind just topics that are so incendiary that dissenting opinions are likely to raise firestorms of outrage and condemnation, but also other ones where the uniformity of respectable opinion is so well-entrenched and secure that someone opposing it will be affectionately and self-assuredly laughed off as an annoying but harmless crackpot, with comparably small repercussions.)

One obvious example is decolonization, which killed more people than the nazis. No postcolonial government was as good as the colonial government it replaced. Even the very worst colonial governments, for example the Congo, were followed by even worse postcolonial governments. In some cases, such as India, they very slowly got better over time. In other cases, like the Congo, they just keep on getting worse with no bottom in sight. They are back to eating each other in the Congo - of course they were back to eating each other as soon as the whites abandoned them, but the number of people being eaten is increasing.

Decolonization denial is arguably worse than holocaust denial - it killed more people, and continues to kill more people, but decolonization denial is mandatory, while holocaust denial is forbidden.

Then there is evolutionary psychology, which tells us that Archie Bunker was right about nearly everything,

For a thousand years before the mid nineteenth century, pretty much everyone agreed that equality between husbands and wives would destroy marriage and fatherhood. Then in the nineteenth century, they introduced marital equality. Observe the result.

Whenever you make such comments, you are making it impossible for me to upvote you, because if you end up upvoted, that'll by itself show the falseness of your claim, and therefore it would be a post unworthy of an upvote.

I promise an upvote for your comment however, if you edit to remove this sentence, because the remaining points you make are very interesting and worthy of such. (edited to add: Now upvoted, as per promise.)

Now to the content-relevant bits:

One obvious example is decolonization, which killed more people than the nazis. No postcolonial government was as good as the colonial government it replaced.

You are making true claims as far as it goes, but I don't think you're seeing the bigger picture in regards to colonialism and decolonization both. For starters it's my impression that the primary stated objections to colonization are deontological ("They don't have the right to rule us/We don't have the right to rule them") as opposed to utilitarian ("We're better off without them"/"They're better off without us")

Now I'm not a deontologist, but I wouldn't mind calling myself a rule utilitarian. And what I see is that claiming the right to control another nation is still used by even Nazi-sympathisers to excuse Hitler's policies (The argument is "If Britain had the right to rule over the hundreds millions in India, why couldn't the Nazi be allowed to rule over Poland and Czechoslovakia"). It was also used by the Soviets and the Americans to excuse their own interference (in a different way).

In this manner, European colonialism served to justify both German and Japanese imperialism in the eyes of their citizens; thus contributing to World War II -- it possibly had an even more direct effect on World War I, which may have been motivated by a German desire to take some colonies. And conversely after European Colonialism was defeated, and the rule "We must not control other nations by force" firmly established, one gradually sees peace descend on the European continent itself, and eventually a decrease in the amount of interference that the Soviets and Americans applied on their own spheres of influence too (Eastern Europe and South America respectively).

In short: In a world where European colonialism still thrived, would we have seen the Eastern European communist governments collapse? European decolonization had a primarily positive effect on Europe, the way I see it. Perhaps some Africans would want to be controlled by Europe again, I don't see Europeans as willing to accept the offer though.

Lastly in your condemnation of decolonization, i'm not clear if you're truly arguing:
a) It was a wrong choice for the colonized people to seek independence
or
b) it was a wrong choice for the European nations to grant it to them

I think you'll find it hard to argue for (b) -- effectively that France should still be wasting lives and money fighting wars in Algeria and Vietnam, or that the UK should be trying to crush the Indians violently. It's easier to argue for (a) -- especially if you limit your argument to sub-Saharan Africa, where the borders were artificial, the national identity often nil, etc,etc.

For a thousand years before the mid nineteenth century, pretty much everyone agreed that equality between husbands and wives would destroy marriage and fatherhood.

Do you have a citation for that? It would surprise me to learn that such equality was even discussed about a thousand years ago.

And conversely after European Colonialism was defeated, and the rule "We must not control other nations by force" firmly established, one gradually sees peace descend on the European continent itself,

Are you sure that's not just war fatigue following the world wars? Notable is that Europe experienced another period of peace of comparable length in the 19th century following the Napoleonic wars and the congress of Vienna. This despite that period being the height of colonialism.

and a decrease in the amount of interference in the Soviet and American spheres of influence too (Eastern Europe and South America respectively).

And what I see is that claiming the right to control another nation is still used by even Nazi-sympathisers to excuse Hitler's policies (The argument is "If Britain had the right to rule over the hundreds millions in India, why couldn't the Nazi be allowed to rule over Poland and Czechoslovakia").

That anti colonialism is objecting to "the right to control another nation" fails to describe the situation with Vietnam, Rhodesia and South Africa, where anti colonialism was used as justification to interfere with foreigners.

To describe anti colonialism as opposition to the right of one nation to control another is as misleading as describing PC speech controls as courtesy.

Anti imperialism is opposition to the right of one country to control another. Anti colonialism was typically the objection of the metropolitan elite to the colonial elite. Anti colonialism was those whose power and wealth derived from the capital, objecting to those whose power and wealth derived from the colonies

South Vietnam, Cambodia, and Laos, were supposedly not anti colonialist enough and therefore needed to be ruled by adequately anti colonial outsiders (members of the North Vietnamese communist party who spent most of their lives in Moscow) for their own good. Rhodesia was not under attack from Rhodesian blacks.

The Indochinese wars were imperial wars more than they were anticolonialist wars, and the imperialist side (communist) was the anticolonialist side. The same is true of the wars against Rhodesia. Mugabe's powerbase was in London and the UN, not in Rhodesia, his powerbase was the traditional imperial powers.

The most celebrated anti colonial conflicts had imperialists fighting colonialism.

Many "anti colonial" conflicts seem to involve "anti colonialists" with connections to ruling class universities in the metropolitan country, fighting "colonialists" without such connection,Mugabe comes from London School of Economics: Bishop Muzorewa went to a no name US university. And so the world removed Muzorewa and installed Mugabe, murdering as many Rhodesian blacks as necessary to ensure the right "democratic" outcome.

That really does not look much like Rhodesian blacks ruling themselves.

I think you'll find it hard to argue for (b) -- effectively that France should still be wasting lives and money fighting wars in Algeria and Vietnam

France was fighting a colonial war in Vietnam? Surely the events that followed the French fleeing suggest that this was a war with the Soviet Union, not a war with France's Vietnamese subjects, that the anti colonialist side was also the imperialist side. And if you find those events unconvincing, the events that followed the Americans fleeing should have convinced you.

The war with Algeria was indeed a colonial war, with colonialists and imperialists on the same side, and anti colonialists and Islamists on the other side. Let us recall, however, how and why the French got into what is now Algeria in the first place.

Europe had, for several hundred years, suffered Islamic terrorism. Punitive raids against the terrorists, for example the American Barbary wars, failed to deter them. So the French occupied the lands from which the terrorists attacked the most, and settled those lands with non Muslim settlers.

Algeria was a message to Muslims: Attack Christians, lose your land. Very eleventh century, and as in the eleventh century, it worked.

This successfully ended Islamic terrorism. From 1830 to 1960, the west had no problem with Islamic terrorism. When the French fled Algeria in 1960, Islamic terrorism resumed.

Europeans are considerably worse off for forcing European settlers out of Algeria, just as Jews are considerably worse off for forcing Jewish settlers out of the Gaza strip.

that the UK should be trying to crush the Indians violently

The UK was trying to crush the Indians violently?

The Indian independence movement was as much sponsored by the LSE and other British elite universities as the Rhodesian "independence" movement. Ghandi was a mascot. India continued to stagnate in poverty and communal violence until it finally got leaders from less "anti colonialist" (but suspiciously imperial) sources.

For a thousand years before the mid nineteenth century, pretty much everyone agreed that equality between husbands and wives would destroy marriage and fatherhood.

Do you have a citation for that? It would surprise me to learn that such equality was even discussed about a thousand years ago.

The status of women was not a government issue until the nineteenth century, but a private issue for families. In this sense, it was discussed, but not as a political question. Consider, for example the Paxton family quarrel over the right of daughters to choose their own husbands. The Paxtons tended to use their daughters as poker chips in the long struggle over the Falstoff inheritance, and the Bishop of Norwich was drawn into this drama to arbitrate between mother and daughter. While the Bishop correctly upheld the Christian doctrine of marriage by consent, he was arguably disturbed by the potential threat to the institution of marriage.

"The taming of the shrew" also addresses this issue, in this sense. Note that Petruchio has to tame Katherina without giving her the thrashing she so richly deserves, whereas Margaret could and did beat her daughter Margary in an manner alarming and scandalous, even though according to Christian doctrine (that marriage is by consent) Margary was completely in the right, and Margaret completely in the wrong.

There's been a lot of discussion in this thread about whether or not your examples are valid. But there seems to be a more substantial problem: Your primary example, of whether or not decolonization was a good thing or was handled well is extensively discussed in academia. Similarly, most of the people doing ev psych are tenured professors and the like. So whether or not the views expressed are accurate, the claim that they are ignored by academia seems to be false.

Now, moving on from that, let's look at your claims.

Then there is evolutionary psychology, which tells us that Archie Bunker was right about nearly everything,

Is this a statement about racial groups or a statement about gender relations? I can't quite tell which was intended. Note that ev psych doesn't really say that much that Archie Bunker would actually agree with. Is this intended for rhetorical effect? If so, can you please state this more explicitly.

For a thousand years before the mid nineteenth century, pretty much everyone agreed that equality between husbands and wives would destroy marriage and fatherhood. Then in the nineteenth century, they introduced marital equality. Observe the result.

The request for citations below for the first sentence has already been asked and hasn't been really answered. (I will note that the Talmud which is a series of texts known for debating almost everything about its own legal system has nothing at all saying that equality between the sexes would destroy marriage). But aside from that matter, there's really a pair of pretty easy explanations for the "breakdown" of marriage.

First, in many places, common law marriage (really sui juris marriage), which really wasn't much more than acknowledged long-term cohabitation, was considered marriage. As that became less acceptable in the 20th century, people who if they had lived a century before would have been considered married were no longer counted as married.

Second, lifespans went up. In the 19th century, many marriages ended at an early age with the death of a spouse. See here. This data isn't ideal for this purpose because they are calculating life-expectancy of everyone which means that the decline in infant mortality also comes through. But in general, life expectancy has gone up. Divorces have become more common as a means of ending marriage, but the actual length of the average marriage is not that different from what it was in the 19th century.

There is an argument that equality between males and females has also lead to higher divorce rates. And it probably helps that females can initiate divorce (something that was difficult to do in some places in previous centuries) . It also obviously didn't hurt matters that rising equality made it easier for women to live on their own, which made getting a divorce have fewer downsides. But the claim that equality in marriage itself led to a decline in marriage seems to be extremely weak to the point where the correlation so obviously doesn't imply causation that it is almost a textbook example of that problem.

Your primary example, of whether or not decolonization was a good thing or was handled well is extensively discussed in academia.

Really? Could you refer me to an academic paper that has a perspective on decolonization similar to the one sam presents? Near as I can tell "post-colonial studies" are all about blaming Europeans and their descendents for all the world's problems.

Another exercise, since I believe you're currently in academia your self, bring up the perspective on decolonization with fellow academics in a way that implies it has merit. Let me know if you still have an academic career by the time the resulting firestorm blows over.

As far as I can tell, the only demographic sector of my society in which fatherhood can be said to be "destroyed", is specifically that sector that has been targeted by government policies that systematically place young adult men in prison for long terms — and largely for ideological, pseudoscientific reasons.

Drug laws should be considerably relaxed (I actually favour full legalization of basically everything) but there is pretty strong evidence however you splice it (or rather whichever of the oh top three or four most likley social groups you may have in mind) that they would still be incarcerated more. I'm willing to go for 10 to 1 odds that this would be so even in a perfectly fair (whatever that is) system.

Upvoted for the shocking but intriguingly plausible comparison of the disultility from decolonization to that of the Holocaust. In additon I would agree that social equality between the average man and the average woman does hurt the traditional insitutions of marriage and fatherhood.

Actually, the more I read over Thiel's essay, the more I question our ideas about the future at mid century. For example:

--How many of those predictions were based on the erroneous belief that communist states were also valiantly leading the charge of progress? We now know that half the world essentially squandered five decades of human progress.

--To what extent were they underestimating the technical challenges involved?

--How many misplaced ideas about society were there? Many ideas, like interplanetary colonization, seemed to spring from a belief that people would want to play Daniel Boone in space suits.

--How much of it was driven by the now commonly understood ability of media to twist scientific findings? The world seems very different when you think that physics just created infinite energy or psychologists had just discovered how to completely unravel the mind.

And, to me, most importantly:

--What was the cost of overactive imagination? I feel like there is a lot of cynicism driven by failed futures which were wildly implausible either technically or socially. I remember that old science fiction liked to speculate about interplanetary travel and living in spaceships in the 1990s. Rotting magazines laying around the garages of grandparents seemed to present these fantasies as a fact about the near future. The sort of cynicism bred by none of this happening because none of it was really plausible could be fatal. (I for one fell into cyberpunk, so I've been getting a good serving of future. :P)

--How many of those predictions were based on the erroneous belief that communist states were also valiantly leading the charge of progress? We now know that half the world essentially squandered five decades of human progress.

This seems like an interesting idea but it exaggerates things. First of all, I think that everyone thought that there was a large amount of overlap between research being done in the both the US and the USSR (look at the space programs for example). Second, the USSR did do a lot of very good research on their own (look for example how many Nobel and Fields medal they won as a very rough metric).

To what extent were they underestimating the technical challenges involved?

This seems to be the biggest issue. There seems to have been a massive underestimate of just how tough space travel and associated technologies are. People expected the expense to go down at a far faster rate than it did. On the other hand, some technologies that almost no one saw coming, such as personal computers and the internet have if anything become extremely ubiquitous.

How many misplaced ideas about society were there? Many ideas, like interplanetary colonization, seemed to spring from a belief that people would want to play Daniel Boone in space suits.

I think that this belief isn't misplaced. The cost and tech issues seem to be more relevant. Being Daniel Boone in space is really expensive.

How much of it was driven by the now commonly understood ability of media to twist scientific findings?

Answering this is probably going to be difficult. But I think that even the people who were scientists were extremely optimistic. I don't have the precise year but in I think either the late 1960s or early 1970s, Asimov wrote an essay for the World Book Encyclopedia in which he laid out what he thought was going to happen. There would be colonies on Mars and other planets, and humans engaging in missions to the upper atmosphere of Venus, and about nowish we were supposed to start preparing a generation ship. The optimism was not the fault of the media simply twisting the science.

So, overall, I think for many of these technologies the main issues were not realizing how tough they were along with general unjustified optimism.

This seems like an interesting idea but it exaggerates things. First of all, I think that everyone thought that there was a large amount of overlap between research being done in the both the US and the USSR (look at the space programs for example). Second, the USSR did do a lot of very good research on their own (look for example how many Nobel and Fields medal they won as a very rough metric).

A quick and dirty estimation: the US shows 331 Nobel Prizes, France shows 58, Germany shows 102; Russia shows 27. Russia is also notably larger than either France or Germany, which it trails badly, and would have been the great majority of people in the Soviet Union.

I think that this belief isn't misplaced. The cost and tech issues seem to be more relevant. Being Daniel Boone in space is really expensive.

The locus here is not "being" but "playing"; "playing Daniel Boone in space" conveys the notion of an unserious, wasteful endeavor. It turns out that people are not actually all that eager to waste those resources for the "thrill of adventure" or "to be pioneers" or the like; yet much of the popular science and science fiction seems to assume this motive, possibly because it was first vigorously marketed to young males at a time when Westerns were a dominant narrative mode.

Asimov wrote an essay for the World Book Encyclopedia in which he laid out what he thought was going to happen.

Asimov was, however, a biochemist and a writer. He wasn't an aerospace engineer or an important physicist; he certainly wasn't someone in a position or with expertise to actually know the feasibility of what he was discussing. In most respects, he was more a member of the media than a member of the sciences; he is certainly more remembered that way, no?

A quick and dirty estimation: the US shows 331 Nobel Prizes, France shows 58, Germany shows 102; Russia shows 27. Russia is also notably larger than either France or Germany, which it trails badly, and would have been the great majority of people in the Soviet Union

Ok. Wow. I knew there was a discrepancy. I didn't realize that the differences were that massive, That's um... wow. Phrased that way I now have to wonder how more people didn't during the Cold War realize how much the USSR was being hobbled by its own problems. This undermines my claim quite a lot.

The locus here is not "being" but "playing"; "playing Daniel Boone in space" conveys the notion of an unserious, wasteful endeavor.

This still seems to be a function of the resource level involved and how much technology is required. If playing Daniel Boone in space took only a few hundred thousand dollars I suspect that a lot of people would jump at it.

As to Asimov, yes that's a valid point. He was writing far outside his field. He's not however the only example of this, merely the most prominent. Sagan was also in that hybrid zone of science and media but a bit closer (having actually worked on probes and aerospace ideas including a military proposal to detonate a nuke on the moon) and he made similar comments.But, you make a good point. It seems that the rank and file engineers were not nearly as optimistic. So the media point seems stronger than I stated.

Ok. Wow. I knew there was a discrepancy. I didn't realize that the differences were that massive, That's um... wow. Phrased that way I now have to wonder how more people didn't during the Cold War realize how much the USSR was being hobbled by its own problems.

Do we have reason to think the Nobel process was really non-political enough to take those numbers at face value?

As an example, I've read that in the early 20th century the Nobel physics committee deliberately decided to focus on awarding prizes for developments in atomic & nuclear physics instead of other fields like astrophysics or atmospheric physics. I can't remember why; I want to say it's because Sweden happened to be particularly strong in atomic physics at the time, but I'm much less sure of that. (I'm annoyed that I can't find a reference to double check this now.)

It's hard to make a comprehensive selection of relevant extracts, but here are some bits & pieces, since the article's paywalled:

I shall argue that the subsequent restriction of the scope of the physics prize stemmed from a determined effort within the Swedish physics discipline. Not surprisingly, Nobel Prize deliberations became enmeshed in the process by which various factions within the Swedish physical science community attempted to define and to legitimize their particular notions of physics: its objects, methods and goals.

[During the debate over awarding Einstein the Physics Prize], three of the committee's five members belonged to the Uppsala tradition of experimental physics — B. Hasselberg, G. Granqvist, and A. Gullstrand. The others were S. Arrhenius (physical chemistry) and V. Carlheim-Gyllensköld (mathematical and cosmical physics). The committee's strong experimentalist bias proved significant from the start in determining Nobel Prize decisions and for interpreting the statutes.

Debates on the role of mathematics and theory in physics that had begun in the 1890s carried over into deliberations on the prize. Hence, even in the case of a world-wide campaign for Henri Poincaré, in which his mathematical physics qua physics was delineated from his purely mathematical accomplishments, the committee did not see fit to recommend him for the prize⁸. In a protest note, Carlheim-Gyllensköld pointed out the committee's general unwillingness and inability during the first ten years to evaluate nominations in mathematical physics such as those of Boltzmann, Heaviside, Kelvin and Poynting⁹. Arrhenius had commented that Uppsala physicists consider "spectral analysis...the only part of physics worth pursuing"¹⁰. Indeed, Hasselberg did "all in my power to procure the prize" (1907) for A. A. Michelson, who had received but two nominations, not for his role in the aether-drift experiment but, rather, primarily for his work in precision spectroscopy and metrology¹¹.

Other examples abound. Thus the proposal for the 1908 prize was sabotaged partly by those who believed it "unjust" to award a prize to a theoretician, M. Planck, without dividing the honour with an experimentalist.¹³

The Einstein deliberation came at a turning point in the development of Swedish physics and of the committee. By 1923, after the deaths of Hasselberg and Granqvist, Oseen was elected to the committee as a regular member and was joined by his Uppsala colleague, atomic physicist Manne Siegbahn. Together with Oseen's friend, Gullstrand, the three Uppsala physicists commanded a majority on the committee [...] By promoting atomic physics, in which experiment and theory intimately progress together, he [Oseen] hoped to overcome traditional prejudices and to institutionalize a theoretical physics standing apart from abstract mathematical physics and mechanics. Yet, to be successful [...] they not only had to promote and legitimize their preferred research programmes, but would also have to eliminate what they considered insignificant specialties. Starting in the 1920s, the Uppsala group began a determined campaign to restrict the definition of physics within the Academy and with regard to the Nobel Prize.

in 1923, the new Uppsala group and Arrhenius attempted to eliminate astrophysics from the scope of the prize in physics. Arrhenius, who wrote the proposal, seems to have feared that the establishment of new Nobel institutions would threaten his own Nobel Institute for Physical Chemistry. [...] Previously, only Arrhenius's institute had been supported in this way, but during the war Carlheim-Gyllensköld suggested a department for cosmical physics that was later followed by detailed plans for an astrophysics centre²⁹'³⁰.

It goes on. More briefly, Swedish physicists' attempts to boost their own fields led to politicking within the Nobel Prize in Physics committee that shortchanged astrophysics, atmospheric physics, and physics that was too theoretical.

As to Asimov, yes that's a valid point. He was writing far outside his field.

When Asimov wrote a book on anything, be it Shakespeare, the Bible, or physics, I expect him to have more of substance to say than most experts in the field. He wrote a series of books on physics that are still used today, and he was a biochemist.

I think I agree with you about Daniel Boone in space, that if the (personal) resource costs were more tolerable we'd start seeing it. What I'm missing originally is the number of people willing to pay even millions of dollars to simply skim the surface of our atmosphere, a healthy portion of which don't seem primarily motivated by status. So yes, you're coorect that we'd probably have had a lunar colony if it were feasible to deliver people at fairly low cost; in fact, if the cost is low enough, I can see this as highly likely both for Daniel Boones and high risk research.

It would not so much matter if an accident released a virus into the lunar vacuum or obliterated several square miles of lunar surface, but we might have found it useful to station researchers or at least maintenance there to carry this out without communications lag or other issues.

Hmm, that essay is interesting. But it has clear problems. One part that jumped out at me:

But to grasp just how far our current mindset is from being able to attempt innovation on such a scale, consider the fate of the space shuttle’s external tanks [ETs]. Dwarfing the vehicle itself, the ET was the largest and most prominent feature of the space shuttle as it stood on the pad. It remained attached to the shuttle—or perhaps it makes as much sense to say that the shuttle remained attached to it—long after the two strap-on boosters had fallen away. The ET and the shuttle remained connected all the way out of the atmosphere and into space. Only after the system had attained orbital velocity was the tank jettisoned and allowed to fall into the atmosphere, where it was destroyed on re-entry.

At a modest marginal cost, the ETs could have been kept in orbit indefinitely. The mass of the ET at separation, including residual propellants, was about twice that of the largest possible Shuttle payload. Not destroying them would have roughly tripled the total mass launched into orbit by the Shuttle. ETs could have been connected to build units that would have humbled today’s International Space Station. The residual oxygen and hydrogen sloshing around in them could have been combined to generate electricity and produce tons of water, a commodity that is vastly expensive and desirable in space. But in spite of hard work and passionate advocacy by space experts who wished to see the tanks put to use, NASA—for reasons both technical and political—sent each of them to fiery destruction in the atmosphere. Viewed as a parable, it has much to tell us about the difficulties of innovating in other spheres.

The idea here is interesting. But by the time the shuttle was already being designed there was some understanding of the dangers of space debris (although Kessler's seminal work would not have been done for a few years). The problem of having very large objects in near optimal orbits would have been obvious. And since the shuttle flew at a fairly low orbit, one would actually have these deorbit and renter soon after launch anyways without giving them a massive boost. Spent fuel containers are also not optimal for later storage. They don't have the same characteristics that one generally wants unless one modifies them massively. So even if one could boost them even higher for free, it isn't at all clear you'd want them for construction without massive amounts of changes. Sure it might have been nice to have it as an option, but the main reason the shuttle didn't succeed was that it had so many different jobs it had to do. For example, the military wanted it to be able to launch into a polar orbit and come back down after a single orbit.

This also doesn't seem to appreciate at the time how incredibly innovative an actually reusable space plane was. It was perfectly reasonable in 1970 for this to be innovative. The fact that all the shuttle replacement proposals are basically copies of the shuttle or minor variants seems to be a much stronger argument that there's a real problem.

The fact that all the shuttle replacement proposals are basically copies of the shuttle or minor variants seems to be a much stronger argument that there's a real problem.

The space shuttle did not actually work - hence a new version that actually does work is the correct thing to do.

An actually useful space shuttle would be capable of frequent flights, say once a day, would not need crew to push the big button, and would land like the rocket it actually is instead of justifying NASA's air force affiliation with a few seconds of normal flight like a plane.

Since it would fly once a day, it would necessarily transport smaller cargoes to space: There just is not enough demand yet. So it would be capable of carrying one reasonably slim passenger plus his life support. Larger objects would have to be taken up in bits an assembled in space by a robot.

The current proposals aren't anything like this. They won't be anything that could fly once a day. They aren't proposing anything like that. The current proposed replacement will be able to launch if everything goes well slightly more frequently than the shuttle did. It won't be nearly as replaceable (crew launch will be an essentially Apollo-style system). The total lift mass will be higher than the shuttle eventually but not for the early versions.

The main systems that are coming from the shuttle are the shuttle booster rockets, and it would have a similar external fuel tank. There's no engineering reason for doing this. The primary reason is that certain contractors lobbied Congress so that they could keep their contracts for the parts they get to build. There's a proposal to eventually give the SLS a new set of booster rockets that use more advanced technology and are built to actually optimize the new SLS requirements, but I'm skeptical that this will happen. And if it does happen, there's only one guess about what company will make the new booster rockets.

This isn't about taking a flawed plan, learning from it, and making a new version that doesn't suffer from the old flaws. This is mainly about keeping the same small number of big aerospace companies happy.

I have. There are still people publishing books that go in the SF section of the bookstore, but I could count off on one hand the number of authors I can remember who are feeding people hope instead of stylish postmodern cynicism.

"SF don't inspire hope anymore" does not imply "SF has collapsed as a literary genre"; I'm not aware of "inspiring hope" being a factor in judging the existence of a literary genre.

However, I think a relevant data point in favor of Thiel's claim is this: where is our 2001: A Space Odyssey? That is, a work of hard-core sci-fi (10 on the "Mohs scale" of sci-fi) that's achieved mainstream success and entered the broader culture.

Set in crapsack world of overpopulation, famine and imminent nuclear war, where human race was from the beginning a toy of omnipotent aliens. What hope?
Our world in 2001 was not like in "2001", it was much better.

This could be a dispute about the definition of SF: if you consider the core of SF to be something antithetical to "stylish postmodern cynicism," then maybe it has come on rough times. If you use the same definition the bookstores do, on the other hand, it's doing fine.

The lack of such in the last few years may be more of an indication that that niche has simply already been filled? If someone did a hard scifi space story how inevitable is it that it will be compared to 2001?

Does it matter whether there is hope or not in the stories? We are less hopeful about this precisely because things haven't been nearly as impressive in the last fifty years as we thought they would be. A story set fifty years in the future with big Mars and Moon colones won't look hopeful, it will look at best overly optimistic, and possibly naive.

Moreover, this may actually be a signal that the genre has reached real maturity, that it isn't just gee-whiz look at that stories.

The claim that the genre has collapsed simply isn't born out when one looks at how popular Greg Egan, Charlie Stross, Alastair Reynolds, and Cory Doctorow are among others. The genre is in fact much more successful now. Fifty years ago, book shops didn't have separate sections for science fiction. Now they have large sections. Science fiction also is one of the most popular genres judging by the sales of used books. Science fiction is also far more accepted as a literary genre by academia, to the point where there's a separate area of study devoted to it. I have to wonder how much of what you are seeing is just the decline in average quality that occurs when a small thing becomes popular.

Just because the genre conventions have changed from your preferred form doesn't mean that the genre has collapsed. This sounds a bit akin to fans who when something in their favorite franchise changes become convinced that it has been Ruined Forever. (Not linking to TVTropes for obvious reasons).

One man's bitter cynicism is another man's gritty realism and one man's hope is another man's delusionary dream. Looking at another literary genre, did the emergence of hardboiled school "collapsed" crime fiction?

More like DisContinuity, I would say. Which isn't surprising, given that EY's favorite TV shows are "All four seasons of Babylon 5 and all three seasons of Buffy the Vampire Slayer."

(Both of those shows have more seasons than that, but are generally agreed to be of declining quality.)

Although, I wouldn't have guessed someone capable of writing Three Worlds Collide would be so much in favor of hopeful!SF versus stylishly!cynical!SF. But maybe I'm reading it wrong?

Edit: To clarify, I'm not accusing TWC of being stylishly cynical, precisely, but... while it is one of the best-written iterations of "humanity makes First Contact, overcomes communication barrier in record time only to immediately discover irreconcilable differences that inevitably result in the deaths or brainwashing of billions" I've ever read, one must recognize that is what it is, and that's not exactly what I would describe as "feeding people hope".

If you consider the contemporary struggle between Enlightenment values and Postmodern cynicism, Three Worlds Collide is definitely in the former camp. Its message is that no matter what happens, there is still a difference between right and wrong. I don't know what a postmodern answer to Three Worlds Collide would look like (someone should write one, please) but I imagine it would use moral error theory in place of moral realism.

Fifty years ago, book shops didn't have separate sections for science fiction. Now they have large sections.

I can't speak to fifty years ago, but I can tell you that 47 years ago Bookland, a smallish and not extremely ambitious bookstore in a suburb of Wilmington, Delaware definitely did have a science fiction section.

When any given field takes half a lifetime of study to master, who can compare and contrast and properly weight the rate of progress in nanotechnology and cryptography and superstring theory and 610 other disciplines? Indeed, how do we even know whether the so-called scientists are not just lawmakers and politicians in disguise, as some conservatives suspect in fields as disparate as climate change, evolutionary biology, and embryonic-stem-cell research, and as I have come to suspect in almost all fields? [!!! -- SB]

I don’t necessarily dispute the point he makes. But I would like to offer as an alternative perspective Mencius Moldbug’s suggestion that paleoclimatology and climate modelling are not “science” – or at any rate we are not “cleaving reality at its joints” by using the term science to refer to both physics, with its adequately falsifiable hypotheses and controllable experiments, and the likes of climate change and economic modelling. Moldbug regards these as cargo cult sciences which derive a spurious aura of authenticity from their appropriation of the term “science”, and argues that their inherent resistance to the power of the scientific method allows false beliefs to proliferate in these areas if these beliefs serve the interests of political power (which is welded to the University system and its ability to manipulate public opinion.)

There is a significantly greater likelihood that “climate scientists” and “climate science” as a whole are systematically corrupt as truth-finders, in comparison to (for example) physicists. So Moldbug would be more likely to attribute intellectual stagnation to this corrupted-by-power “scientism” of the University system than to the existence of “overspecialized, insular scientific fields” in particular (although the two are not mutually exclusive).

The current crisis of housing and financial leverage contains many hidden links to broader questions concerning long-term progress in science and technology. On one hand, the lack of easy progress makes leverage more dangerous, because when something goes wrong, macroeconomic growth cannot offer a salve; time will not cure liquidity or solvency problems in a world where little grows or improves with time.

Again to supply the Moldbuggian perspective, the ultimate cause of the economic crisis is maturity transformation, of which fractional reserve banking (with its scalar rather than temporally-based accounting) is one example. Frequent banking crises are to be expected until such a time as the maturity-mismatching banking system can no longer reply on implict loan guarantees from the state, and therefore ceases to exist (since in a genuinely free market economic system, banks that practise maturity transformation inevitably go bust and stay bust).

According to this analysis, “liquidity problems” are in fact evidence of the free market assigning a perfectly reasonable (non-MT) value to the overpriced assets in the banking system. And quantitative easing is a means of attempting to push the system from this equilibrium back into yet another cycle of unstable maturity-transformation practices.
Therefore (if one finds Moldbug persuasive) at best the economic crisis has merely proximate links to “long-term progress in science and technology”.

I can't supply evidence in the space available! These are complicated topics that cannot be explained or justified briefly. It's merely an invitation for people to consider an alternative, but coherent viewpoint (hence all the links). That is why I did not claim these analyses to be true but merely described them as Mencius Moldbug's opinion.

EDIT: Okay, to be slightly more clear. Arguing from principles and representative quotations is a good way to give evidence for claims about the way people say things - postmodernism at its best. It is not a good way to get evidence about the world, and can even be counterproductive because of signal to noise problems.

I don't see that there's a better way to divine the truth in historical, economic, and political matters. Bayes's Theorem isn't much use when you have no decent numbers to put into it - regardless of the fact that it is true. How would you conduct a worthwhile Bayesian analysis of the proposition that neocameralism is a superior form of government to democracy? Have I any reason to believe that such an analysis would be better than Moldbug's deductive reasoning? Bear in mind also that statistics are not necessarily trustworthy or fully informative.

I find that a sharp mind - Moldbug's is extremely sharp indeed - is capable of achieving a high signal-to-noise ratio using this style of reasoning where others might not.

After all, a direct confrontation of the reasoning style you disparage and an approach I expect you might consider more "evidence-based" is found in macroeconomics - Austrians as literary economists, Keynesians as quantitative economists. I'm sure you'll agree that the hegemonic Keynesians have not exactly covered themselves in glory.

Arguing from principles and representative quotations is a good way to give evidence for claims about the way people say things

I can only interpret this as the idea that all speech and writing provides evidence only about the speaker or author himself. I disagree in the strongest terms!

I don't see that there's a better way to divine the truth in historical, economic, and political matters.

I didn't ask for a full statistical analysis. Although it would certainly be nice, it seems like a lot of work - deductive reasoning is but a tiny subset of probabilistic reasoning. However, I suppose one could use random sampling methods to make sure that you didn't go far wrong. New field name: asymptotically correct history.

Anyhow, what I said was lacking was evidence in general - arguments that are harder to make when false than when true. This sort of thing often does mean statistics, but not because statistics are magical - because when you use statistics it's easy to see when you leave stuff out.

I find that a sharp mind - Moldbug's is extremely sharp indeed - is capable of achieving a high signal-to-noise ratio using this style of reasoning where others might not.

Correct reasoning is not a magical power, it's a process that anyone can follow. A->B to ~B->~A, non-replicable reasoning is highly suspicious.

After all, a direct confrontation of the reasoning style you disparage and an approach I expect you might consider more "evidence-based" is found in macroeconomics

If we drag economics into this we'll be here all day :P Anyhow, I'd rather pick an example with representation from the "literary" side, and maybe a control group. Unfortunately, no clean examples come to mind.

Arguing from principles and representative quotations is a good way to give evidence for claims about the way people say things

I can only interpret this as the idea that all speech and writing provides evidence only about the speaker or author himself. I disagree in the strongest terms!

Derek Lowe has had several posts on that upcoming problem on his blog, http://pipeline.corante.com/ , over the past 6 months or so. And I strongly recommend his blog to anyone interested in pharmacology. He spends more time on the business and politics aspects than I really like, but also has good posts on scientific method and chemistry, his "Things I Won't Work With" series, http://pipeline.corante.com/archives/things_i_wont_work_with/ , is particularly entertaining.

EDIT: Links fixed. The markup on LW apparently attached the commas to the links.