Those of you who’ve read my books know that in addition to my research on the evolution of large-scale human societies and the rise of centralized states and empires, I am also interested in the reverse process by which an empire loses cohesion and gradually crumbles into a ‘failed state’ (these two directions in my cliodynamics research are reflected in parts I and II, respectively of War and Peace and War). I have found Jack Goldstone’s ideas on processes leading to state collapse and descent into civil war particularly useful, and I and colleagues extended his demographic-structural theory and tested it with empirical data on Medieval and Early Modern England, France, and Russia, as well as ancient Rome (this was published in Secular Cycles co-authored with a Russian historian Sergey Nefedov; ‘secular cycles’ refers to alternating periods of strong economy, population growth and cohesive states with periods of economic and demographic decline and pervasive political violence).

I have given a number of talks on this research, and the most frequent questions I get after the presentation is, “so where are we in the secular cycle? Is the American empire about to collapse?” My initial inclination was to deflect such questions by pointing out that human societies changed dramatically during the last two centuries, and thus we should not expect that theories developed, and tested with data on ancient, medieval, and early-modern empires would work on societies we live in.

However, as our knowledge of secular cycles in historical societies became richer and more detailed, I couldn’t help noticing that many of social, economic, and political trends we observe today appear to be very similar to trends we saw in historical societies that my colleagues and I studied. Naturally, the demographic-structural theory needs to be reformulated, but can it still be useful in understanding the trends affecting our own societies?

Five or so years ago I started collecting data on structural-demographic variables in the United States (as a terminological note, I decided to move up ‘structural’ because the demographic component is not quite as important when the theory is applied to modern societies). The period I focused on was from the beginning of the Republic, roughly 1780, to today. One big advantage of working with American history is that quantitative data of reasonable quality are much more plentiful than for other historical societies I dealt with in the past. Thus, for a number of key structural-demographic variables I was able to find multiple ‘proxies.’ A proxy is a variable that is closely correlated with whatever we want to study, but that is hard to measure. Proxies, thus, provide a way to follow the dynamics of such hard-to-measure things. For example, air temperature is the average velocity with which molecules in the atmosphere bounce against each other. It’s hard to observe directly, but we can readily measure it by observing the height of the mercury column in the thermometer. This height is the proxy for temperature. Physicists and biologists use proxies all the time. To give another example, when I was population ecologist and studied population dynamics of lemmings, we used a proxy for their population densities: the number of lemmings captured in mouse traps (rodents per trap-night).

It is surprising to me that historians and social scientists don’t use the proxies as much as natural scientists do. True, the proxies that we have access to in historical social science are not as closely related to things we want to measure (as in my thermometer example). And we should always check how well our proposed proxies work. Still, it’s a very productive approach.

For example, one of the most important variables in structural-demographic theory is ‘intraelite competition.’ How do we operationalize this quantity? In his 1991 book Jack Goldstone made a brilliant suggestion – we can proxy it by the increase in the number of enrollments in institutions of higher learning. Why should it work? Well, four decades ago another brilliant historical sociologist, Randall Collins, published an important, but largely ignored book, The Credential Society, in which he argued that most youngsters go to college not to expand their minds, but to simply obtain a piece of paper (the diploma) that improves their chances of getting a lucrative job (shame about it). Thus, when competition for elite positions intensifies, more people seek to obtain advanced degrees.

In his study of the seventeenth century crisis in England, Goldstone looked at enrollments at such universities as Oxford and Cambridge. He found that indeed, the enrollments at Oxford, for example, ballooned during the first half of the seventeenth century, in the period preceding the English Civil War. And it wasn’t just a long-term modernization trend in which people came to value education more. When intraelite competition subsided in the early eighteenth century, so did Oxford enrollments. I am not saying that everybody who went to Oxford just wanted to get the credentials, rather than education. But those who were really interested in knowledge were a decided minority.

This ironic cover of Academe, the magazine of the American Association of University Professors, portrays college education as mass-production of diplomaed graduates

Of course we have to be careful. Many factors can affect the number of youth seeking higher education, not only increased competition for high-quality jobs. So we should seek other proxies for the quantity of interest, and check whether they tell the same story. For seventeenth century’s England, we can also look to such proxies of competition as the amount of litigation. Another one that I used for both England and France was the frequency of dueling. It turns out that dueling epidemics tend to develop during periods of high intraelite competition. Makes sense.

This issue of proxies is very interesting, and it is of high importance to making history into science (‘cliodynamics’), but let’s go back to the structural-demographic dynamics in America. Over the last several years I have amassed an incredible amount of data. As I mentioned before, for many variables of interest it was possible to locate a number of proxies, so we can really pinpoint the trends and, even more important, trend reversals. I am currently putting these results together in a book-length publication.

One of the most important variables is sociopolitical instability. Sociopolitical instability includes all kinds of collective (group-level) violence that occurs within states. So it’s that sort of violence that falls between individual crime, on one hand, and interstate warfare, on the other. This variable is important because it is what the structural-demographic theory aims to explain. We need good data about it.

Because this is such a critical variable, I put a lot of effort into trying to find good empirical measurements. I found a number of databases that addressed certain aspects of sociopolitical instability, but no comprehensive such database. So in the end, I realized that I had to construct it myself, which I did by merging the pre-existing databases with comprehensive searches of electronic resources, such as LexisNexis database of newspaper articles.

One curious consequence is that, I believe, I am now a real historian – because I worked with sources in archives. True, my sources were newspaper articles from the nineteenth century, and the archives were electronic ones (so I did not have to inhale the dust of medieval manuscripts). So I suppose true historians will probably not induct me into their guild. No matter, it was a remarkable experience. And, in some ways, an appalling one. For one thing, I can’t believe how much time I actually spent deciphering old newspaper articles while constructing this database. Most people at my stage of career simply don’t do this type of ‘manual’ work. The second appalling part was due to the nature of my research. Since I was investigating political violence, I had to read through some really appalling descriptions of truly horrific incidents (which I will not inflict on you). On a number of occasions I was so affected by what I juat read that I had to drop this project for a while and to switch to something else to regain my faith in the humanity.

In any case, this project is now done, and published. The paper came out last week in the Journal of Peace Research. You can find the paper on my cliodynamics.info site. Additionally, the paper gives a systematic exposition of the structural-demographic theory as reformulated for post-Industrial Revolution societies (initially, the article focused on the construction and analysis of the database, but the journal editors asked me to add the theoretical part, and I don’t regret it).

Many of you will not be able to access the article at JPR, but here is the article reprint, which is the same as published version but without correct page numbers.

I am actually working on it right now and hope to post the first draft in September. But it will be a technical book with lots of data, graphs, tables, a model or two. The graphs in the appendix of the JPR paper actually came from the book draft.

It is interesting that while collective violence (riots and lynchings) has declined, individualized violence (spree killings, rampages etc) has increased in recent decades. Does this perhaps reflect some other trends in society?

Shooting rampages are not really individualized violence. Most of the perpetrators in such events are attacking not individuals, but institutions. They aim to kill the firm, the school, or the whole society. There is no qualitative difference betweeh a rampage shooter and Timothy McVeigh – just in the means of mass murder they use.

You can not discuss violence and rampages without discussing the contentious issue of race and multiculturalism. Criminal violence from whites has remained remarkably consistent in the USA.
However the rate of violence from blacks has varied substantially being at the level of 3x whites during the days of jim Crow, rising to a factor 15x during the 60&70’s during hippie policing policies and now falling down about 7x withthe “broken window” policing. With regard to the general increase in individualised violence against institutions, Mr Putnam of bowling alone points out (reluctantly) that as multiculturalism has increased you have been getting reduced rates of trust and indentification with institutions overall. The US as it exists at the moment is probably on the path to domitian before dissolution.in

This is fascinating. I didn´t know that anyone had addressed sociopolitical long term stability in History in such a scientific and rigorous way. One thing is just hypothesising about causes and effects in History, but it is a very different thing to test hypothesis and do research with real data. I guess this is what cliodynamics is about.

I would like to read both “War and peace and war” and “Secular cycles”. Which one should I read first?

Read War and Peace and War first – it’s a popular account of my theories and results. Then, if you are interested in the science underlying this narrative, consult Secular Cycles (and for other aspects, the earlier Historical Dynamics).

Is there a wider possibility of using more theoretical means to sort through the demographic and socio-political material? Statistics are all very well but I am feeling a dearth of social theory itself as part of the problem of violence in social structures.

Are social theory positions such as those of classical Weber or Marx et al likely to add refinement to the interpretation of your exhaustive work? Have you considered the refusal of policy makers to adopt theoretical positions as part of the problem? Is there too much of an organic view of “society” prevalent in US approaches to conflict?

Despite the way my research has been portrayed in the popular media, it is very much theory-driven. Data is for testing theories, not for data-mining. Check my cliodynamics.info webpage for the background. The J Peace Res article starts by presenting theory: