Parsing that “98% of Catholics use contraception” figure

I’ll be back with a longer post on the actual substance of the contraceptive mandate and my ideas about the reasonable limits to the idea that religious mandates exempt you from the law, but I can’t pass up the chance to geek out about social science statistics and methodology.

You’ve all probably heard the “98% of Catholic women use contraception” statistic cited by now, and people have started to take a closer look at the relevant study. Let me chat you through some of the objections.

Who was polled?

The survey done by Guttmacher didn’t include all women. Only women who met these three criteria were included in the computation

Between the ages of 15-44

Not pregnant, post-partum, or trying to get pregnant at the time of the survey

Sexually active (had sex at some point in the last three months)

This seems like a pretty good sampling frame to me. It looks like the researcher was trying to see what percent of women who were having recreational sex were using contraception.

The most intuitively correct restriction is age. We don’t expect children to be using birth control, so including them in the survey just means you’re goosing the ‘no’ responses. That would be reason enough to exclude them, since you’re not trying to learn things about their demographic, but there’s a more important methodological reason to keep them out of your sample.

If both populations (Catholic and non-Catholic) had the same age distribution, including children would just drive down the percentages for both subsets. But if the two groups were demographically mismatched, whichever group was younger relative to the other subset would look like it had a lower contraceptive rate.

But…

This means the study doesn’t have anything to say about all Catholic women. It simply says that Catholic women who are having non-deliberately-procreative-sex use contraception at about the same rates as any other women in those circumstances.

So, it would be good to report what percentage of Catholic women aged 15-44 fall into that subset and contrast that proportion with other religious groups. And these two numbers should be reported together.

The table above doesn’t do it, since it just reports the percent of women who have ever been sexually active outside of marriage. As far as I can tell, the number I’m looking for wasn’t calculated by the researchers.

What were they asked?

The first problem may just be sloppy reporting (and non-ideal, but passable survey design). The second one is more serious. Take a look at the graph from the study below:

The 98% number is the result of only counting the NFPers as ‘non-contracepting.’ The trouble is that it’s not clear exactly what “other” and “no method” refer to. According to the methodology:

The category of “other” methods mainly consists of withdrawal but also includes less common methods, such as suppositories, sponges and foams.

I think it may vary by religion whether ‘withdrawal’ really counts as contraception. (Someone can fill me in with a comment).

The big problem is that there’s no explanation of what “no method” means in the paper. Get Religion assumed it meant that the women were not using contraception, which certainly seems plausible. However, it may also mean that women answered yes to “Are you currently using some kind of contraception” but refused to specify the type in the followup question. I’ve seen this style of notation for academic studies before. A reporter should have called up and asked for clarification.

So, the percent using contraception is at most 98%, and if you were really being conservative, you should probably cite 83% (condoms, IUDs, the Pill, and sterilization).

So, in a nutshell…

Catholic women who are in a position where they’d be tempted to contracept, use contraception at about the same rate as everyone else in those circumstances, and that rate is somewhere between 83-98%. How many Catholic women are in this situation in the first place? Beats the heck out of me!

Leah Anthony Libresco graduated from Yale in 2011. She works as a statistician for a school in Washington D.C. by day, and by night writes for Patheos about theology, philosophy, and math at www.patheos.com/blogs/unequallyyoked. She was received into the Catholic Church in November 2012."

http://prodigalnomore.wordpress.com The Ubiquitous

The OFloinn also did a wonderful job parsing the distortion, and with his typical lighthearted seriousness. Money quote:

So the study tells us only that 98% of women of child-bearing age who want to have sex without having babies use some form of birth control. That qualifies as a sort of “d’uh” moment.

http://prodigalnomore.wordpress.com The Ubiquitous

In fairness, too, determining this number was not probably the point of Planned Parenthood’s statistics wing. More likely, they were interested in improving services, such as they are, to their customer base, such as it is.

keddaw

Perhaps it should be rephrased, using Leah’s numbers, as:

83% of Catholic women of child bearing age who are sexually active use some form of birth control in direct contravention of the Catholic Church’s stance on the issue.

Given this number, why on earth should we take the Catholic Church’s views seriously on this issue?

http://prodigalnomore.wordpress.com The Ubiquitous

83% of Catholic women of child bearing age who are sexually active use some form of birth control in direct contravention of the Catholic Church’s stance on the issue.

I think you missed Leah’s No. 2:

Not pregnant, post-partum, or trying to get pregnant at the time of the survey.

Moreover, I thought “trying to get pregnant” was defined even more broadly as including “being indifferent to pregnancy one way or another.”

Which, if true, amounts to saying of those trying to prevent pregnancy, and including those using the Church-approved method, nearly 100 percent indicated some method of trying to avoid pregnancy.

dbp

That was my sense, too– but I got it from reading, “Most sexually active women who do not want to become pregnant—whether unmarried, currently married or previously married—practice contraception.”

Colloquially, that reads to me like “want NOT to become pregnant” rather than “not at the moment actively trying to become pregnant.” But if the questions as posed to the respondents was actually, “Are you pregnant, post-partum, or trying to become pregnant?” Then it might just be poor wording.

Given that this was conducted via in-person interviews, and the people presumably already agreed to participate in a study about their sexual activity, I think the ‘refused to specify” scenario is somewhat less likely, especially if the ‘do not want to become pregnant’ is as narrow as proposed here.

There must be a fair number of people who ‘do not actively want to become pregnant’ who don’t actually take steps to prevent it. In fact, judging how poor people are at tuning their behavior to their goals, given the above supposition, I’d say 11% is actually much lower than I’d expect, there.

http://prodigalnomore.wordpress.com The Ubiquitous

For the raw data, check out supplemental post by The OFloinn, this one addressing Leah’s question about how many women in general use contraception.

http://prodigalnomore.wordpress.com The Ubiquitous

Given this number, why on earth should we take the Catholic Church’s views seriously on this issue?

Why judge ways by the wayward? If a man by the fault of a route drives off a cliff, the road is faulty. But if a man steers perpendicular to the road and runs off a cliff, who would blame the route?

Blame the maintenance crew all you want, blame the driver, blame factors beyond the driver’s control, blame the road itself, but do not blame the route.

anodognosic

Is it possible, in your mind, that it is the Catholic hierarchy that is wayward on this issue?

http://prodigalnomore.wordpress.com The Ubiquitous

Before I respond, I’ll note that both question and answer are an aside, as the point of the previous comment was to making the distinction between moral precept and moral behavior.

For reasons which will convince nobody else — I learned by driving off cliffs, and I’ll leave it at that — I am convinced of the moral teachings of the Church. Speaking for myself, this is my Exhibit A for the case of trusting the Magesterium. As a matter of fact, from this the rest of my faith is built.

Turning the question around, is it possible in your mind that the Catholic Church is right? I refer, of course, to what the Catechism actually says rather than how the persons behave.

keddaw

The point is that the only reason the Catholic Church gets a voice at all is because it claims to speak on behalf of all these millions of Catholics (who are unrepresented in a representative democracy?). If it turns out that the Church actually says things that are in opposition to the beliefs, actions and possibly well-being of most of its members then it has no legitimate claim to make proclamations on their behalf.

I have also heard it said (not by you) that anyone who goes against the teachings of the Church is not a Catholic, reducing the number of actual Catholics down to a small cult. But that was a strict Catholic that said that, not me.

Suzieq

The issue is whether one can be forced to follow a law that Pharmacists, doctors, nurses, as well as priests, and yes, even contracepting women, understand that to be forced to perform or to provide materials to allow an abortion is a moral evil. And luckily under the US Constitution we are protected from this moral evil.

http://prodigalnomore.wordpress.com The Ubiquitous

The Dalai Lama doesn’t speak for many people at all and he has a voice. He’s quoted more often, and more positively, than the Pope in the secular press.

millions of Catholics

I think you mean more than millions. But suppose the Church only represented one old man living alone in a Rome apartment — that does not change the truth value of what he has to say. If the Pope walked alone in the March for Life, he would still be right.

Now, if your point is that the bishops cannot speak for the laity, I’m not convinced that even if true makes any difference. They can certainly speak for themselves, after all, as Prominent Citizens. Status alone makes them newsworthy, another quality independent of correctness.

keddaw

The Dalai Lama doesn’t speak for many people at all and he has a voice. He’s quoted more often, and more positively, than the Pope in the secular press.

Yes, but he doesn’t claim to speak for tens of millions of Americans or have politicians champing at the bit to fawn all over his pronouncements. The Dalai Lama’s idiotic proclamations have to stand or fall based on their efficacy, sense or relation to the real world – they don’t get written into law automatically (I know the Pope doesn’t have this ability yet, but they used to and it’s closer than we’d like today!)

…that does not change the truth value of what [the Pope] has to say

But the truth value comes from its relation to reality, not his position as Pope. But, as Pope, (most) people don’t challenge what he says and (many) Catholics like you uncritically accept what he has to say, even when he’s factually and ethically wrong.

http://prodigalnomore.wordpress.com The Ubiquitous

Where do they claim to speak for the laity? They speak, you know, for the Church, which the laity professes to believe in. Insofar as the laity actually believes the Church, the bishops speak for them. Insofar as they don’t, the bishops do not.

Also notable is that bishops canonically do not just speak for themselves but for all Catholic-qua-Catholic organizations. Bishops have the authority to grant or revoke the label of Catholic, however defiantly appropriated such a label gets.

But the truth value comes from its relation to reality, not his position as Pope.

I agreed with you before you said it.

But, as Pope, (most) people don’t challenge what he says and (many) Catholics like you uncritically accept what he has to say, even when he’s factually and ethically wrong.

Uncritically? That seems a high charge. Factually? That’d be a curious point — we don’t claim he is necessarily right except on proclamations of faith and morals. Besides, the minor premise has changed before, as in the case of Aquinas and Augustine on abortion — it’s a matter of fact that we discern life as beginning earlier, not a matter of values. Ethically? Beyond begging the question, the Pope’s stated ethical precepts are the reason I trust his judgment. To wit, they are when I was most critical, a word which here means, “I thought about it long and hard,” and not, “I resolved to disagree before I read it.”

As an aside, you should have been around in the time of the Borgias when the person of the Pope was less admirable. Too bad you got saddled with a Vicar of Christ who is an absurdly eminent scholar and profoundly sound leader.

But this discussion is a red herring rabbit trail in search of a wild goose. Given that actual current issue of the bishops speaking together is not a question of contraception, right or wrong, but a question of religious liberty:

You are right to say that they are not speaking for Catholics. Instead, they are speaking for Anglicans, Lutherans, Evangelicals, Calvinists, Eastern Orthodox — I don’t think Christianity has been this united since the Arians died out. Even the Jews are on board with us on this one, and, if only one I know of, an atheist.

deiseach

100% of Catholics have lied at some point in their lives. So we should toss out the requirement in civil courts to swear an oath/make an affirmation that the testimony I am about to give is true, because I am a lying Catholic?

Ted Seeber

I would point out that the problem in this case isn’t the contraception, it’s the sexual promiscuity to begin with.

In other words, Pope Paul VI was absolutely correct when he claimed, in Humanae Vitae, that widely available contraception would turn men into assholes, women into sluts, and destroy the concept of a *sacramental lifelong marriage*.

http://prodigalnomore.wordpress.com The Ubiquitous

The OFloinn has a good, supplemental post, this one addressing Leah’s question about how many women in general use contraception.

The first thing we note is that 38% of the women in the cohort report that they do not use contraception. If this is so, how can 98% of women use it? Inquiring minds want to know.

Our biggest chunk of the 38 percent is the 20 percent total of women in the cohort who haven’t had sex in the last 3 months. Seems there’s a lot more celibate, by choice or otherwise, than we’d otherwise let on.

One other thing which occurred to a commenter elsewhere is that the figure is more accurately “98 percent use or have used &c., &c., &c.” contraception. Can I get a fact check on this?

Seems to me that the distortions just get wider and wider.

The Licensed Fool

Thank you for a good and balanced article.

It would also have been interesting to see the results compared with a similar sample for sexually inactive women of the age group mentioned – those who could be having recreational sex but, for whatever reason, were not. Those results divided by religious affiliation and reasons for sexual inactivity may have added something.

Remembering of course that chastity untill marriage is a choice which can be motivated by religious belief (as well as community norms) and therefore could have been statistically valid as an example of compliance with church doctrine.

It would have provided a good point to compare the ‘headline’ statistics to. LF

http://darwincatholic.blogspot.com DarwinCatholic

I would have also liked to see all the groups broken down by active/semi-active/non-active religious practice. They clearly gathered the data, because they note that sexual activity among non-married women among all the religious groups surveyed among those who attend services weekly or who say religion is “very important” to them. But they don’t break down any of the contraceptive usage stats that way.

Also, I’m a little unclear why you wouldn’t ask come basic set of questions about doctrine near the end (after you’re already got untained usage data) along the lines of: - Does your religious denomination have any moral teachings against birth control usage? (No | Only Allows It Under Some Conditions | Never allows artificial birth control) - Do you agree with your denomination’s moral teachings on birth control? (Yes | No) - Do you follow your denomination’s moral teachings on birth control? (Yes | No)

Similar questions on sexual activity outside of marriage would also be interesting. The thing being that just because someone violates a moral precept at times (even often) does not necessarily mean that person disagrees with it.

leahlibresco

Yes! It would have been great to see the breakdown by religious activity.

As to why they don’t have the questions at the end, part of it is that it’s expensive to place surveys in the field and every question you add increases the chance the respondent will stop participating simply due to time. At my research-y day job, we’ve often had to cut demographic questions that interest us, and then wish we had swapped them for something else with the benefit of hindsight.

And don’t forget, when this study was done and the report was written, it wasn’t necessarily designed for the talking point it ended up yoked to. They could definitely have done a better study if they were trying to gauge to what extent Catholic women followed Church teaching on contraception, but they may have been trying to answer at totally different question.

More reason to make raw data publicly available!

deiseach

There is also a fascinating result in that NSG survey, which no-one – so far as I can see – has used, not even those opposed to the mandate:

Go to Table 15, which tabulates the reasons women stopped using a specific contraceptive method. For “Too expensive” or “My insurance didn’t cover it”, these total out at about 3% each (hey, if reputable media entities can be sloppy with figures, then I can be, too!)

Most common reason for stopping using contraception? “You had side effects.”

So the impression we are getting that millions and millions of women cannot access contraception – oh, sorry, “preventive health care*” – because they can’t afford it may be not quite all it’s cracked up to be. If 64% of women stopped using the Pill because of its side-effects versus 3% stopped using it because it was too expensive, you could actually argue that the bishops – by refusing to provide coverage for these drugs – are being more concerned about women’s health!

*The only thing the proposed insurance coverage for contraception, emergency contraceptives (‘morning-after’ pill) and sterilisation prevents is pregnancy, and for the majority of women, pregnancy is not a disease or life-threatening condition in the same way that diabetes, hypertension or clinical depression is.

deiseach

Call me a cynic, but perhaps the reason for pushing the contraception agenda is for the same reason I’ve seen argued quite nakedly by some pro-mandate comments on blogs: if poor/minority women can’t afford it, then providing free mandatory coverage is a social investment in the future because if we let them get pregnant, their children are more likely to become criminals and be a cost to society in terms of crime, violence and prisons.

http://darwincatholic.blogspot.com DarwinCatholic

Yeah, fair enough. The purpose of the survey was clearly pretty different from the use that it’s been put to recently.

I’ve been wishing for ages that someone would do a survey that covers this kind of thing in detail, but the one’s I’ve seen just kind of nibble around the edges. National Catholic Reporter does a survey every so often which, among other things, asks about adherence to teachings on contraception among Catholics, but it doesn’t ask it in what strike me as the most interesting ways. For instance, figure 5 here shows 60% of Catholics saying you can be “a good Catholic” without agreeing with the Church on birth control:

Table 17 covers answers to “Who should have final say about what is right or wrong? Church leaders, individuals or both?” 10% say leaders, 66% say individuals 22% say both

But neither of these really address the convergence of practice and belief among the people whose lives would actually be a different as a result

http://prodigalnomore.wordpress.com The Ubiquitous

Right or wrong in what sense? Final in what way? Phrased that way, and if I didn’t know it was Fishwrap survey, and if I didn’t think very carefully, I’d tentatively be in the 66 percent.

I say this though certainly the 22 percent have the right gist and the 10 percent probably the right understanding, depending on how carefully they read between the lines.

Ted Seeber

Most conservative Catholics would consider National Catholic Reporter doing a survey of this kind to be Confirmation Bias, in and of itself. They aren’t very well known for promoting Church teaching over there.

deiseach

I love your disgruntled crustacean (and probably had a similar expression on my face years ago when trying to draw up a graph for the results of an experiment in a chemistry class, where the data points were all over the feckin’ shop but the report we were supposed to produce mandated that they should be connected by a straight line because that was the point of the experiment because we were demonstrating the theory).