Death of Religion Predicted by Demographers Torturing Statistics

Because what’s the use of a blog that brings together Christians and atheists if it doesn’t take time to snark on bad science reporting/bad study methodology? I like geeking out about this, but if it’s not your cup of tea, never fear: the next post in queue covers the ex-gay lecture I mentioned I was attending on Friday.

UPDATE: Thanks to Darksmiles for pointing out an error in this post. I misread the scale of some of the graphs in this paper, since the study listed them as “percentages” in the legends and the actual graphs axes were labelled as proportion. That makes a hundredfold difference and completely nixes one of my critiques of the paper. I’ve dropped that bit, but left up and modified the parts about extrapolation and chart making.

To briefly recap the methodology, the researchers model future religious affiliation as being primarily dependent on the attractiveness of a particular religion. They define attractiveness to be primarily a function of how many people are currently affiliated with the religion, since a community with few parishioners will not benefit from the same level of pride in unity, resources, energy, etc as would a more robust parish.

If you have a little mathematical intuition, you won’t be surprised to learn that this model predicted a decline in religious affiliation that would grow faster and faster until religion was extinct.

Ok. So let’s have a quick chat about extrapolation and modeling. Take a look at these plots from the paper:

So, the time scales for the data they’re considering extend til 2005 in the Aland islands of Finland, 2000 in one canton of Switzerland and a province of Vienna, and 2050 in the Netherlands?!?

There’s no excuse for that graph design. The researchers, who are ostensibly trying to accurately approximate past trends, have no reason to think that trend will necessarily hold 50 years in the future. The rescaling of the graph makes it hard to see the actual data. When I recrop their graph to purge the no-data section, a fair amount of the projected curve plotted is totally irrelevant.

There was never a reason to have that part of the graph included in the figure, since the researchers don’t have a strong reason to assume their projection is correct. This study was entirely focused on checking whether a model was a good fit for pre-existing data; they don’t have a way to check how well it will do at predicting future trends.

That’s not to say it might not predict future trends, only that we can’t know that yet. As years pass, we can check whether new demographic information matches this model’s predictions, but, if they didn’t, all the researchers would have is a decent way to summarize known data.

It’s also worth remembering that extrapolation can go backwards as well. If we ran the equations further into the past, non-religious affiliation would approach zero. What’s interesting to talk about is where the rise begins and what prompts it. The data as they stand could be of interest to religious historians who would try to figure out what variations by geographical area would have jumpstarted this process of deconversion at different points in time.

A variety of factors would have influenced when this process began and its initial speed. Variables and influences just as complex will impact the future trends, and there’s plenty of reason to believe that the researchers’ model of utility associated with popularity will not persist.

To illustrate this point, I want to look at a complaint some religious blogs made about this study. They claimed the methodology was rigged because the entire model was adapted from previous work the academics did to model the way that declining numbers of fluent speakers of a dialect accelerated the death of that dialect. How could this model do anything but predict the death of an organization, they asked?

On Martha’s Vineyard a small group of fishermen began to exaggerate a tendency already existing in their speech. They did this seemingly subconsciously, in order to establish themselves as an independent social group with superior status to the despised summer visitors. A number of other islanders regarded this group as one which epitomised old virtues and desirable values, and subconsciously imitated the way its members talked. For these people, the new pronunciation was an innovation. As more and more people came to speak in the same way, the innovation gradually became the norm for those living on the island.

If I were placing bets, I’d guess that the rise in atheism is primarily a matter of picking up the low hanging fruit — people who were never strongly attached to their religion, had doubts about it, and now have an out and, as time passes, there are more natural atheists who are never born with even a weak tie to a particular faith tradition. After most of the susceptibles get picked up (and I’ll bet there are still a lot of them out there), atheism’s rise would slow, and, stripped of their weakly committed members, churches would become more exaggerated and demonstrative.

I also wonder what kinds of trends the researches would observe if their model didn’t treat religion as a monolith. If there are many small splinter sects (as there are today), each of them provides less social utility and connection than their parent conglomeration, and, in this model, each would be more vulnerable to extinction. I wonder what the best way to build in churn (people moving from one religious sect to the other) would be, to recognize the fact that not everyone who leaves their religion becomes non-religious.

This is very interesting, Leah, and I also did not know you wrote for Huffington Post. Learn something new every day!(Sorry, I've rarely commented but I've paid attention to you from the weekly 7 Quick Takes.)

http://www.blogger.com/profile/07535825702078498433 Darksmiles

Um, the caption may say percentage, but the y-axis of the graphs is actually straight probability of a 1 = 100% variety. The Netherlands is not a 0.4% nonreligious country, it is a 40% nonreligious country. Your interpretation of the data is off 100 fold. The graph you added has a scale that goes up to 600% of the population being nonreligious, which is mathematically impossible. I would suggest removing this blog post.

http://www.blogger.com/profile/16496144988509668275 Leah

Yikes! That was a good catch, Darksmiles (and quite sloppy of me). I'm revising the post now, but I do want to keep up some parts about the dangers of extrapolation. The changes should appear in about 15 minutes.

http://grace-filled.net jen

Very nice.

Anonymous

I came across this site from a link on brandon's blog Siris and it looks like he is taking on a few questions…that RCIA class may come to its conclusion sooner than later…

http://www.blogger.com/profile/16496144988509668275 Leah

@Kathleen: I also found you through 7 Quick Takes. I'm so glad Jen runs that carnival.