On the Results of the PhilPapers Surveys

by David Bourget and David Chalmers

This is a very brief discussion of the results of the PhilPapers
Survey and Metasurvey. At a later point we will write up a much
fuller analysis.

We were pleased with the response rate to the Survey. There were
3226 respondents altogether, including 931 from the target group of
1974 philosophers from 99 leading departments, 872 other philosophy
faculty and/or PhDs, 829 graduate students in philosophy, 217
undergraduates in philosophy, and 377 with no listed affiliation.

The response rate from the target group was 47%. This high rate
allows reasonable inferences to the distribution of views among the
target group as a whole. Of course there may be some selection bias
among the respondents: perhaps the responding group has some bias
toward philosophers working in M&E areas, or younger philosophers, or
analytic philosophers, or philosophers sympathetic with the editors'
views. At some point we may attempt to analyze certain demographic
features of the responding group compared to nonrespondents, but we
have not done this so far.

The basic results of the Survey speak for themselves. So far, we have
allowed users to view results for groups divided by population
(faculty, graduate, etc) and area of specialization. There are many
interesting patterns here, but we will save comment until we have
performed a fuller analysis. At a later point (probably sometime in
January) we will issue further results concerning answers to the main
questions broken down by chronological features (age and year of PhD),
geographical features (country of nationality, Ph,D, affiliation),
gender, and other features. We will also issue interquestion
correlations and a factor analysis, as well as other statistical
analyses.

On the Metasurvey, we had 727 respondents, including 216 from
the target group, 221 other philosophy faculty or PhDs, and 210
philosophy graduate students. Around 23% of Survey respondents
completed the Metasurvey. The lower rate is understandable as the
cognitive load of the Metasurvey is much higher than that of the
Survey. (We thought about offering a prize for the Metasurvey
respondent who got closest to the true figures, but decided against in
order not to overly encourage respondents to guess in cases where they
had no idea.) We will also analyze Metasurvey respondents for
selection bias. But again we think that prima facie, there are enough
respondents to give some guidance about the sociological beliefs of
the target population as a whole as well as about those of other
relevant groups. We will focus here on predictions among the
population of target faculty, which is also the population whose
Survey results are being predicted.

Among the Metasurvey results, it is especially striking that for many
questions, the target population's mean estimates of that population's
views are off by 20% or more. The normalized results are perhaps most
useful here, as errors in estimating "other" options may not reflect
errors concerning philosophical views. The biggest errors concern
aesthetic value (estimate 68:32 for subjective:objective, actual
45:54) and the analytic-synthetic distinction (estimate 50:50 for
yes-no, actual 71:29). Respondents also underestimate the strong
support for scientific realism, for not switching on the trolley
problem, for moral cognitivism, for non-Humeanism about laws, and for
a priori knowledge by close to 20% each. The case of laws is perhaps
the most striking, with a 50:50 estimate and a 70:30 result.

The Metasurvey results on the thirty questions break down into five types.

In four cases, the population gets the leading view wrong: predicting
subjectivism rather than objectivism about aesthetic value,
invariantism instead of contextualism, consequentialism instead of
deontology, nominalism instead of Platonism.

In three cases, respondents predict a fairly close result when in fact
things are not close: analytic-synthetic distinction, non-Humeanism,
moral realism.

In two cases, a minority view is underestimated by 4-11%: rationalism,
non-physicalism.

In five cases, the estimates are within 1.2% of the actual result:
naturalism, moral motivation, Newcomb's problem, proper names,
teletransporter.

There is of course much to analyze here. Do respondents' sociological
predictions tend to favor their own views? Do predictions better
track views among philosophers working in the AOS of the question than
in the population as a whole? Do predictions better track results in
one's own geographical area? Are Metasurvey respondents in an AOS
better or worse at predicting the views of the population as a whole?
And so on. We will analyze these questions and others in coming
months.

Why refrain from asking the next questions that are directly related to these questions and distinctions already answered within the PhilPapers survey, namely: Are there analytic a priori judgments or statements? Are there synthetic a priori judgments or statements (e.g., a straight line is the shortest distance between any two points)? Synthetic a posteriori? And (the anti-Kantian) analytic a posteriori?

Depending upon one's view of contradictions, there could be a set of valid arguments supporting a fourth set of analytic a posteriori judgments or statements. Anyway, it would be interesting to see if 1-5% made that distinction or sided with Kant against it. I doubt it would be 0% though, and it is important to see how many would make the synthetic a priori distinction contra Hume, which has important implication in ethics and metaethics.

I don't understand factor analysis well, but it seems as if answers to most questions are predicted by just one factor.

For instance, if you are an anti-naturalist, then it's likely that you are going to be a non-physicalist about mind, think that there is a further fact about personal identity, be a libertarian in free will, and believe in God. None of these answers is significantly predicted by any other factor. (Though I wonder what's fundamental: maybe if you believe in God, the others follow as far as your own reasoning is concerned.)

There are a few questions that are determined by more than one factor. Knowledge rationalism is predicted by: anti-naturalism, realism, and rationalism itself.

But all in all, there seem to be just four basic determinative "personality" factors in philosophy: naturalism, realism, rationalism, and externalism. Aside from externalism, which speaks to the transformative influence of Kripke and Putnam, the other three are old chestnuts. I suppo ... (read more)

This is really a superbly interesting survey. I can't wait for the additional analyses to be published!I've seen this kind of thing done for the economics field, too, but I would love to see it done for the sciences. For example, what proportion of neuroscientists or physicists are dualists? What proportion of each are theists? What proportion of physicists accept Copenhagen vs. Many Worlds vs. other interpretations of quantum mechanics? That would all be very interesting to know.

It's interesting to compare answers to a question between the whole target faculty population and those who work in the AOS associated with the question. The biggest differences by far, unsurprisingly, concern theism and the philosophy of religion. The next biggest differences are in decision theory (two boxing), philosophy of physical science (B-theory), philosophy of mathematics (Platonism). Then epistemology (invariantism and to a lesser extent internalism), general philosophy of science (Humeanism), social and politlcal philosophy (egalitarianism), metaphysics (non-Humeanism). And smaller differences in many other areas.

Of course those differences could be due to (i) specialists making better-grounded judgments, (ii) selection effects in entering the speciality, (iii) specialists' judgments corrupted by an insider literature, and various other sources. I suspect that most philosophers will agree that each of these sources are at play in some cases, while they'll disag ... (read more)

In light of the survey's many items with 3 options (besides "other"), I was rather surprised that the "God" question left out "agnosticism". If pressed to declare whether I "lean toward" one of the options provided, I might choose one or the other, yet believe that philosophically the most responsible position is agnosticism. Moreover, considering that there are many rival versions of theism, most people who believe in a particular (theistic) deity disbelieve many or even all rival versions, so overall they too ought to be agnostic. I wonder why this question was dichotomous.

I wonder which way it tends to go. On the one hand, various psychological results suggest that people overestimate the degree to which others agree with them. On the other hand, it seems to me anecdotally that philosophers often feel that they are a lonely voice of truth in a crowd of errors.

This would be easy to check, if the data are compiled in the right way.

I am not an academic philosopher by a long shot, but merely a curious engineering grad student. I was very interested in the survey results, and decided to sort them by mean square error to determine some degree of consensus.

Unfortunately the table works well when pasted initially, but turns into a mess in preview mode. For the impatient, it's near the end of the article. I'm planning on breaking this up further (ie just faculty responses, etc) over the weekend.

Trolley problem (five straight ahead, one on side track, turn requires switching, what ought one do?): switch or don't switch?

Accept or lean toward: switch 635 / 931 (68.2%)

Other 225 / 931 (24.1%)

Accept or lean toward: don't switch 71 / 931 (7.6%)

In Fiery Cushman's and my survey of philosophers' attitudes about moral dilemmas, we asked about this case and our results don't line up very well with yours. Here's the prompt:

You are standing by the railroad tracks when you notice an empty boxcar rolling out of control. It is moving so fast that anyone it hits will die. Ahead on the main track are five people. There is one person standing on a side track that doesn't rejoin the main track. If you do nothing, the boxcar will hit the five people on the main track, but not the one person on the side track. If you flip a switch next to you, it will divert the boxcar to the side track where it will hit the one person, and not hit ... (read more)

I'd been summarizing the latter results somewhat tongue-in-cheek by saying, "Philosophers can't agree whether the external world exists, but they do agree that *if* it exists, then Obama makes a better President than Bush". I'm glad to see some empirical support for this conjecture.

Of course, one further question that naturally arises is whether Democrats are more or less likely than Republicans or affiliates of minor parties to accept the existence of a mind-independent external world.

Where compared to the faculty/PhD group as a whole (instead of to target faculty), graduate students' results are somewhat closer, because majority views among target faculty typically have reduced majorities among non-target faculty. But the majorities are still reduced among graduate students compared to this group.

Suppose one wants to investigate the relationship between various positions and their coherence, but he cannot do it by the selection filters available online. E. g., he wonders about the relative number of non-physicalist, libertarian atheists. Can he ask the team for the table? Do you plan to add more selection filters online?

Latest replies:

Neil Levy, 2009-12-09 : I would like to see whether my suspicion that Australian philosophy differs from the rest of the analytic tradition in v... (read more)

Some interesting differences between metaphysicians vs general philosophers of science and philosophers of physics (restricted to faculty/PhDs):

- Laws of nature: There are significantly more Humeans among general philosophers of science and among philosophers of physics than among metaphysicians (41% and 34% vs 23%). - Science: There are significantly more scientific realists among metaphysicians than among general philosophers of science and philosophers of physics (83% vs 54% and 66%).- Time: There are significantly more A-theorists among metaphysicians than among general philosophers of science and philosophers of physics (25% vs 10% and 16%). On the other hand, there are more B-theorists among philosophers of physics than among metaphysicians (44% vs 38%), but there are more B-theorists among metaphysicians than among general philosophers of science (38% vs 30%).

I was quite surprised at seeing the results of the question on normative ethics. It appears that in every categorical population listed (i.e. graduate students, faculty or PhD) the answer that received the most attention is 'other.' I guess my first question is what the range of those other answers are, both within and outside of normative ethics.

Beyond that, it appears that virtue ethics has taken root more strongly in the younger crowd, which I am a part of, and I wanted to know if the reasoning for that is because of the way undergraduate departments are set up (I know I spent the better part of my "contemporary moral theory" class reading Anscombe, Foot, and McDowell), or if there is some consensus that virtue ethics is a belief which befalls the younger crowd and that we are all better philosophers when we are able to resurface. It also struck me as odd that in the target faculty group the difference between those subscribing to virtue ethics and those to deontology was ... (read more)

Latest replies:

David Bourget, 2009-12-09 : Hi Andrew, you can see a breakdown of the "other" answers by changing the "response grain" to "medium" or "fine.

Personally, I was most surprised by the results on content internalism/externalism and the analytic/synthetic distinction. Internalism is in much worse shape than I thought, and the analytic/synthetic distinction in a much better shape. If we can take these results at face value, that is.

All Survey and Metasurvey respondents will receive an email giving a link to a page with their responses, including an assessment of how well they did on the Metasurvey. We can't post those results publically, as participants did not consent to that, but people should feel free to post about their own Metasurvey results.

I took the Metasurvey unofficially by making predictions at the start of the Survey. I didn't take it officially, as even by that point I'd seen results from beta testing the Survey. Even so, a few of my predictions were off by a long way. For example, I wrongly predicted a substantial majority for Humeanism, aesthetic subjectivism, Platonism, and invariantism. I did better on the physicalism and analytic-synthetic distinction questions, predicting 60-20-20 in both cases (compared to 56-27-17 and 65-27-18), and was reasonable close on the zombie question, predicting 40-20-20-20 for CMI, MP, IC, other as opposed to 36-23-16-25. &nbsp ... (read more)