Quantophrenia is Back in Town

One downside of a lengthy career is that half-baked policy interventions keep coming round again. As Peter Worsley, an eminent British sociologist, once remarked of racism, some ideas refuse to die as long as there are material interests keeping them alive. The supposed quantitative deficit of British social scientists is an example. Various agencies have periodically attempted to compel undergraduate and graduate students in these subjects to become more numerate, following the assumed practice of North America and Europe. In fact this is really code for ‘forcing British sociologists to become more quantitative’ because no one is actually suggesting that disciplines like economics, psychology or geography are short of numerate graduates – and I rather suspect that no policy actor would try it on with anthropologists.

This is at least the third go-round in the 40 years or so that I have been in the academic business. The material interests seem to be very similar on each occasion. A clique of civil servants, policy wonks and academics in some small elite centers complain that they cannot recruit sociologists with the skills they think they require. Consequently, all sociologists must be obliged to acquire those skills in order to fill the vacancies in this niche labor market. The latest incarnation is Q-Step, a joint venture between ESRC, the Nuffield Foundation and HEFCE, each seeking to ingratiate itself with elements of this clique. The consequences, however, do a disservice to the wider market for sociology graduates, to the policy community itself and ultimately to the society that we all claim to serve.

The term ‘quantophrenia’ was coined by Pitrim Sorokin in his critique of Fads and Foibles in Sociology. It is not an attack on measurement per se. It refers to the cult founded on the belief that quantification is the most, or indeed the only, valid form of knowledge. This usually results from an uncritical extension of methods developed in the natural sciences to the study of social life. As with other cults, sociology’s achievement lies in its exposure of the perverse and irrational consequences that follow from this belief system – and in the investigation of how it is created, maintained and accepted as credible.

As Sorokin acknowledged, a basic grasp of statistical thinking is an essential part of a sociological education. Although I have worked mainly as a qualitative sociologist, I think, for example, that one of my most useful insights into organizational problems in child protection services came from understanding the normal distribution. If a generalist agency is referring cases to a specialist agency, the latter will always think that a high proportion of referrals are inappropriate because the two organizations are seeing different populations: the standard deviation that triggers concern in the former is likely to be close to the median in the latter. The result is an unavoidable degree of structural friction when the generalists think that their proper concerns are being dismissed by the specialists.

Nevertheless, sociology’s great contribution is to ask exactly what is being counted and what this means for the outcome, rather than necessarily doing the counting itself. Our skepticism about quantification is a positive contribution to societies and organizations. Another study I worked on investigated death registration in France, England and the USA. Vast scientific edifices are built on these mortality records. However, the three countries collect the data in such different ways that it is unlikely that they are really comparable. For example, the apparently poor performance of the National Health Service in cancer mortality may well result from more open diagnosis and honest recording in England than in France. If the UK has more cancer deaths because we count them differently, rather than because we treat patients badly, then focusing on matching European death rates is chasing moonbeams.

Sociology’s great contribution is to ask exactly what is being counted and what this means for the outcome, rather than necessarily doing the counting itself

As the management scholar, Peter Drucker, observed: “There is nothing so useless as doing efficiently that which should not be done at all.”

The critical analysis of official statistics has been a strong tradition in sociology since its earliest days. This tradition has been renewed through recent work, particularly in the US, on metrics and models by the sociology of science and technology. The new fashion for playing with ‘Big Data’ does not invalidate any of this contribution: it is no more likely to lead to a social physics than it was in the nineteenth century when social statisticians first began to analyze the large-scale data sets newly created by state bureaucracies. I am tiring of empirical presentations on social networks which produce elegant maps and are unable to answer the question: ‘so what does this mean?’

It was not sociologists, for example, who assumed, in modelling the costs and benefits of high-speed rail in the UK, that the traveling time of business passengers was unproductive and should be valued at zero so that quicker journeys represented a real benefit. We actually sat on the trains, saw the suits working on their laptops and listened to their mobile phone calls. Basic observations like this have unraveled the numerous attempts to find a quantitative justification for the hugely expensive, and highly contested, HS2 project.

Charles Babbage, designer of the first programmable computing device, noted, in different words, that garbage inputs would result in garbage outputs. Sociology’s critical stance toward quantitative data is an essential safeguard against the credulity of a policy community corrupted by the cult of quantophrenia. As such, it has a real market value. In trying to correct a supposed market failure, Q-Step, and its progenitors, have neglected the genuine private sector demand for well-trained qualitative sociologists, particularly in consultancy, computing and IT industries. If this demand is not coming from the public sector, it is still the responsibility of public bodies like ESRC and HEFCE to ensure that an adequate supply of suitably trained PhDs is reaching the market. I was recently at a gathering of graduate students in computer science, mostly funded by EPSRC scholarships, where 20 per cent or so proclaimed that they were doing ethnomethodological ethnographies. It is good that they are being supported but it is disappointing that an exciting opportunity for sociologists is being neglected by the agencies that are supposedly responsible for the discipline.

There are good arguments for improving the basic statistical skills of UK sociology undergraduates. A modest increase in the production of postgraduates with advanced quantitative skills may be desirable in response to niche complaints about a shortage of supply. If the public sector is not using qualitative social scientists to the same extent as the private sector, perhaps someone should be asking what they are missing out on.

There is no justification for requiring membership in the cult of quantophrenia. The employability of sociology graduates rests precisely on their skepticism about this cult, together with their acquisition of the valuable skills of critical observation, interviewing and the analysis of texts, documents and images. Skepticism needs to be informed – much as a theologian might study atheism – but questioning the cultists is one of the services that we perform for the benefit of society.

3 Comments

There is no such ‘clique’.
Q-Step is about repairing the almost complete inattention to any QM in sociology degrees. Its not only employers who say this, it is graduates too, who discover that basic skills in ‘counting’ are fundamental to getting a good job.
No ‘quants’ person who is any good overlooks the manifold dimensions of measurement error and its implications. However they also think about the implications of the only alternative: non-measurement error.

In the natural sciences year one students spend a lot of time in the lab. They do this not to pick up vital technical skills for use in later life, but to learn that the basis of any science, social or natural, is empirical evidence, and that the latter does not grow on trees waiting either to be picked or interrogated by a would-be philosopher. Observation and measurement, how to go about it, and what can and cant be done with the results is fundamental.

Year one of too many sociology courses: a smorgasbord of theory, largely unconstrained by anything as profane as empirical evidence (except by way of ‘illustration’) and useful mostly for penning opinion pieces in the broadsheet press.

‘Skepticism’ about quantification appears radical, until a little bit of thought is given to the implications. If good measurement isn´t possible, then anything, literally, goes. Climate change? Increases in income or wealth inequality? Discrimination? Evolution? All in the eye of the sufficiently skeptical beholder. This is ‘sociology’ of shock jocks, Tea Party and the Daily Mail.

I think there is the danger of a “straw man” argument here. I am quantitatively inclined, but I hope I am not so blinkered as to fail to see the limitations of the approach. I teach research methods, and my main goal is to try to ensure that well-rounded social science graduates are “ambidextrous” between quant and qual (at least before any later methods specialisation). What I find is that over 95% of sociology and political science graduates are essentially innumerate (unless trained in North America or certain parts of Europe). Almost all PhDs adopt a qualitative methodology. My first degree was in history, so I have nothing against qualitative approaches. The problem I have is if some core social science disciplines give up almost completely on a particular analytical skill set and thus rule themselves out of whole swathes of public and policy relevance. This is not the same thing as wishing for quantophrenia. It is just trying to retain analytical versatility and relevance in social science.

The idea that quantitatively illiterate British sociologists’ “skepticism about quantification is a positive contribution to societies and organizations” is risible. Why should any sensible person take seriously criticism of quantitative social science by a professional group characterised by people who by and large cannot even read a crosstab? It is like listening to Nigel Lawson on climate change.

I’m frankly embarrased to be part of an academic community that wears its ignorance like a badge of honour. Unfortunately this learned helplessness communicates itself to undergraduate and graduate students, and the cycle continues, with British sociology becoming ever more marginalised in how much it can influence public policy, or, let’s be honest, say anything interesting at all.

Login

Join in our conversation! While you can comment on any of our articles without registering, create an account now to be able to connect with other members, discuss new topics in our forums, and to get regular email alerts with the latest news.