Understanding MANA Stats: a Q & A with Missy Cheyney

Melissa Cheyney, PhD, CPM, LDM, will again be presenting at the Midwives Alliance of North America convention this year, providing membership with an update on the MANA Stats Project.This year at the convention, she will also be announcing the upcoming release of two research articles that examine outcomes from MANA Stats 2004-2009. I sat down to talk with Missy about her presentation at MANA, her hopes for MANA Stats in the future, and how this critical research project can be a tool for expectant parents. Missy is the Chair of the Midwives Alliance Division of Research and an Associate Professor of Medical Anthropology and Reproductive Biology in the Department of Anthropology at Oregon State University (OSU). Learn more about her here. You can learn more about MANAstats here.

Jeanette: So first of all, tell me the basics. What is MANA Stats?

Missy: The MANA Statistics Project (known as MANA Stats) is a web-based data collection tool that has been gathering data about maternity care and birth outcomes since 2004. There have been three main data collection form versions since the project’s inception: the 2.0, 3.0 and 4.0 versions. Overall, more than 60,000 courses of care now comprise the MANA Stats data registry, with more than a 1,000 new records submitted monthly. The majority of records are for planned home births.

Midwives' participation in the MANA Stats registry is voluntary, except in Oregon and Vermont. Midwives log clients into the system early in care before the outcome of care is known. The system then prompts midwives to complete records for all clients who are logged. Once the data is entered, they go through a rigorous review process to make sure the data are as accurate and reliable as possible.

J: And what’s the news about your upcoming research articles?

M: I’m excited to report that a group of researchers and I have two papers on the MANA Stats 2.0 dataset coming out in the Jan/Feb 2014 issue of the Journal of Midwifery and Women’s Health.

J: Before we talk more about the upcoming articles, let’s discuss the basics of the work of the Division of Research. What are the goals of MANA Stats?

M: MANA Stats is designed to serve three purposes.

The first is that it provides contributing midwives with a tool to track outcomes for her or his own practice. The MANA Stats system auto-calculates midwives’ practice outcomes instantly so they can keep close track of their own outcomes. Ideally, as a midwife sits down with a prospective client who’s asking questions, he or she can provide up-to-the-minute information about transport rates, neonatal mortality rates, or whatever data are relevant to the questions the client is asking. This is an important part of shared decision making.

Second, once the sample size is large enough to allow for analysis, researchers on the MANA Division of Research calculate outcomes that midwives can use to compare to their own practice statistics. These larger analyses are communicated to contributors at the annual MANA meetings and via direct contributor communications. They allow midwives to perform quality assurance and quality improvement in their own practices.

The third thing we do is to maintain datasets for researchers. We export our data from the online format it is collected in and respond to requests for data access. Although fewer researchers have requested data than we initially anticipated, all requests to date have been granted. We welcome all data access applications for research projects that have IRB approval from the primary investigator’s home institution and that have research questions that can be addressed with our dataset. Researchers can ask innumerable questions related to place of birth, normal physiologic birth, midwife-led birth, and many other interesting research questions.

J: At the MANA convention you provide midwives with benchmarking statistics based on the MANA Stats data set. Why is this important? Can’t midwives just look at vital statistics (birth certificate) data for their states?

M: Midwives cannot go to Vital Records to ask how they’re doing because Vital Records – except in a few states – does not evaluate outcomes based on intended place of birth. When you are looking at your own outcomes and asking, “how am I doing?” you want to make sure you are comparing apples to apples. We want to compare our outcomes to all women who went into labor intending to deliver at home (or in a birth center), regardless of where they actually delivered.

If you were to look at national statistics—which includes high-risk women who give birth in the hospital—it would not be appropriate, for example, to compare your c-section rate to Vital Records data on c-section rates. We would expect there to be a higher c-section rate in the hospital because, presumably, more higher-risk women are giving birth in the hospital.

Without MANA Stats or the American Association of Birth Centers’ Perinatal Data Registry, it is difficult to find an appropriate benchmark. How would midwives know if their transfer rates were approximately what other midwives are getting? Or whether it is uncommonly high or low, if there isn’t a benchmark by which to compare it?

That’s the goal of these convention presentations: to provide contributing midwives with those benchmarks. We’ve been providing them to membership in some form annually since 2007. Our ability to provide them, however, is tied to sample size. We have been able to talk about demographics for years and even c-section rate because something like a c-section is not a rare event as it occurs in around 6% of cases. It has only been quite recently that we have had enough data to evaluate rare events like deaths with any degree of reliability. In the upcoming article on the 2.0 dataset, even waiting until we reached a sample of size of close to 17,000, there is still much that we could not evaluate because the power of our tests was too low. We need to keep collecting data, and we need all midwives to participate.

J: And why doesn’t MANA make those benchmarking numbers open to the public?

M: It has always been our goal to publish them in a peer reviewed journal, to provide the public with the highest level of accountability.

J: Explain why the peer review process is so important to you and to MANA.

M: I think it is very important that research papers that come from the MANA Stats Project’s datasets go through the process of rigorous peer review required for publication in an academic journal.

The debate around homebirth is very polarized in our country. Many of us hold very deep convictions about it. I am a midwife who attends homebirth. Like most midwives, I feel homebirth is an excellent option for low risk women who hope to avoid unnecessary interventions and to support the processes of physiologic birth. Physicians who only receive our transfers and so only witness the times when mothers need interventions only available in the hospital, like epidural or pitocin augmentation, may feel very differently. How do we remove these biases from our research? The answer is that we cannot completely. What we can do is allow our work to be reviewed critically by others before it goes to press. Our reviewers may see things we have missed or see alternative interpretations of our data.

In a peer-reviewed publication process, outside researchers, who don’t know who you are, look at your work. They look very closely at your methods, and they critically evaluate the tests you have run and the interpretations you have made. They hold you to a very high standard. This can serve as an important corrective to researcher bias.

In the absence of rigorous peer-review, really you can say anything that you want to. And with a controversial topic like home birth, we want that added measure of protection to have as many eyes as possible on it. We want to make sure that we haven’t made any mistakes, that we’ve been really transparent in how we’ve calculated our numbers, and that it meets the standard of rigorous peer-review for an academic journal.

Otherwise there’s nothing to keep anyone from making claims to safety just by going on the Internet and saying "it is safe; here are our rates." We want that level of scrutiny on our work and peer review is the way to achieve that.

J: We see the CDC release their numbers each year, and we see some states release their numbers each year. Why doesn’t MANA release their data every year?

M: A CDC-style report every year that takes MANA stats and reports on safety has been a vision of mine for some time. We would love to do that, and I received a grant from FAM to study the feasibility of doing just that.

To have meaningful data on rare events like infant death, we need to increase the sample size by increasing the number of participating midwives. Only about one percent of people in the U.S. choose a home birth. In addition, not every midwife in the U.S. uses MANA Stats to track their data, although participation has grown each year. It has taken six years to get a large enough sample size to allow for this first set of publications. It takes time for a project like MANA Stats to build a sufficient sample size to have credibility with a peer reviewed journal.

The good news is that our contributor base is expanding, and the number of homebirth and birth center births are on the rise in the U.S. We now have more than 400 participating midwives. In 2012, we collected data on around 10,000 courses of care. I will be presenting some preliminary findings from this dataset at the convention, and it will be one of the next articles that I work on.

Also, MANA does not have the organizational capacity of the CDC. MANA is a largely volunteer-driven organization, and MANA Stats has a very small budget. The CDC and the various states releasing annual data have paid staffs—some quite large—devoted to this effort. We welcome any and all funding opportunities that would allow us to engage in similar efforts to the CDC and other reporting bodies.

In my life as a researcher at Oregon State, I really cannot imagine a project as large as MANA stats running on volunteer time and such a miniscule budget. It would be a multi-million dollar endeavor in that world. I am so grateful for the committed staff on the coordinating council of the DOR who work tirelessly to support contributors, review data, export data, and analyze findings fueled only by a desire to know more about midwife-led births at home and in birth centers. All of this is done under almost constant criticism. Their work is herculean, and I am proud to be a part of it.

J: Please tell us more about the research that is on the horizon.

M: We’ve had seven researchers or sets of researchers apply to use our data set, all of whom have received access, and now some of the publications from those requests are starting to either go to press or are under review. Some are students who have used the data for a Master’s thesis but may not have the intention to publish the findings. Four articles have emerged that are either in press or under review, including the two that I mentioned, on the MANA Stats 2.0 dataset coming out in the Jan/Feb 2014 issue of the Journal of Midwifery and Women’s Health.

J: One of those two articles looks at the outcomes of the MANA Stats 2.0 dataset. What can we expect to learn?

M: This article looks at the demographics of the MANA Stats data set 2004-2009, including the intended place of birth and the type of midwife in attendance (we have mostly CPMs but also some CNMs, naturopathic midwives, doctors of osteopathy, or midwives who identify as un-credentialed or traditional midwives who utilize the data set).

It also looks at standard maternal-child health outcomes and home birth indicators, like transfer rates, i.e. intrapartum transfer, neonatal transfer, maternal postpartum transfer, and it looks at reasons for those transfers. It also examines cesarean section rates and spontaneous vaginal birth rates. It also examines intrapartum, early neonatal, and late neonatal mortality. Finally, it explores rates and type of tearing, hemorrhage, and NICU admissions.

J: How can the outcomes research inform consumers?

M: I think many consumers are asking themselves: "What are the trade-offs for the place of birth that I am choosing?" There’s no perfect place to give birth, and there’s no place to give birth that will have a mortality rate of zero.

So what a woman is actually doing is negotiating her own individual risk profile, as well as her own personal and family values as she chooses a place to give birth. An informed consumer needs access to information on all of the potential birth options: home, birth center, and hospital, as well as by provider type: CPM, CNM, OB, etc.

Women are often forced to make difficult decisions that involve trade-offs around rates of intervention, the likelihood they will be able to know who will attend them when they go into labor, cost, and access to medical back-up should a complication arise. If you are in the hospital, you may have significantly more intervention than you might at home. But then the question might be, at what cost? What's the impact of interventions when they are unnecessary? Do they increase morbidity? Those are questions you cannot ask if you do not track outcomes by intended and actual place of delivery.

Consumers today have to piece together information from studies that are often conducted outside of the U.S. So they are also making inferences about safety and benefits across different medical systems. There’s not a perfect place to get information about place of birth and provider type. But MANA Stats is bringing something to that discussion. Our sample size is getting large enough to start to look at things like rare events that will figure in for a woman trying to decide where she wants to give birth and with whom. What is her risk of having a c-section? What’s her risk of having an unnecessary c-section? What’s the risk of the baby having problems? Those are all questions a woman might want to ask, and there’s not one place that she can go to get this information right now. I think that's a problem, and I am excited to see all that has been happening in the research world over the last few years—the National Birth Center Study II, the Home Birth Consensus Summit, and the Institutes of Medicine Birth Settings Workshop. What an exciting time!

J: And what is your hope for MANA stats long term?

M: My hope is that someday we will not need MANA Stats for tracking safety. My hope is that the system of national surveillance will eventually accurately take into account provider type and intended place of birth, because many of our discussions right now are around safety; those are the most fundamental questions that people are asking.

However MANA Stats will still need to collect data because, from a researcher’s perspective, safety questions aren’t the only questions one could ask. I’m really interested in the process of care. How does a particular style of care lead to different outcomes? What components of midwife-led care lead to lower rates of unnecessary interventions? How can normal physiologic birth best be supported? How does midwifery care improve outcomes for communities of color who have a higher risk of mortality under standard medical maternity care? There are still some questions about process of care that are best evaluated by a more nuanced data collection tool like MANA Stats or AABC's PDR that are designed for home and birth center settings.

I am also excited about the future of data sharing. What if ACNM, AABC, and MANA could share data between our systems or bring it all together so data could be collapsed into one large data set across provider type and across intended place of birth? That would really be ideal.

J: I know one goal is to expand the number of researchers accessing the data. How can a researcher apply?

M: Interested applicants should go to the website. Interested researchers can read through our handbook and contact me with any questions. We are also committed to ensuring that midwives and student midwives have access to the data. If someone is interested but doesn’t have access to an Institutional Review Board (IRB) for example, we can help match students and midwives without formal research experience or institutional affiliations with expert researchers through a program called Connect Me. Midwives and students midwives can serve as content experts and learn the processes of research design and analysis as they collaborate with more experienced researchers. Two student midwives have successfully used the Connect Me program to date, and they will be presenting their findings at this MANA meeting.

J: Why is Institutional Review Board (IRB) approval so important?

M: IRB or ethics boards exist at institutions to ensure the rights of human research participants are protected. Unfortunately this process had to be created to prevent future atrocities like the Tuskegee experiments or some of the experimentation on human beings done in concentration camps in Nazi Germany.

To prevent the exploitation or coercion of research participants, universities and other organizations conducting or sponsoring research require that all researchers who intend to use data collected from human subjects be familiar with national and state-level guidelines for working with human subjects data. Institutional Review Boards are also usually inter-disciplinary, and they review research proposals to make sure that if there are any places where special protection of participants needs to be in place, that that is done.

There are two reasons why this is important to us. First, we want to make sure the privacy of participants is respected and protected. For example, in MANA Stats there are such a small number of early neonatal deaths that it wouldn’t be impossible for a researcher to figure out who some of the mothers and babies are in the sample. By combining the date of birth and the location where the birth occurred, a Google search could potentially give you the name of a participant.

Our second reason is very practical: The code of federal regulations # 21 part 56 requires it, and researchers cannot submit findings for publication unless they have gone through appropriate procedures to access data. You must have IRB clearance before you analyze data for publication.

J: Any final thoughts?

M: Yes two things. First, I encourage all those engaged in the place of birth debate to remain open, curious, and critical about what the flurry of research coming out now can tell us. It is difficult to move science forward when pro- and anti- sides of a debate become entrenched. The answer to the question of where women should give birth is very likely more complicated than simply answering yes or no to whether one thinks homebirth should be an option. I don’t think every woman is a good candidate for home birth, but I am also very frustrated with its wholesale dismissal by some as an option ever. We need a more nuanced research agenda that examines questions of safety, benefit, risk, cost, and access to care for multiple subsamples of women with various risk profiles. Secondly, no matter where a woman chooses to give birth or where we as providers feel the safest choice might lie, we have a duty to work together across difference to support and care for our nation’s mothers and babies.

Jeanette McCulloch, IBCLC, has been combining communications work and women’s health advocacy for more than 20 years. She is a co-founder of BirthSwell, which is working to improve infant and maternal health—and the way we talk about birth and breastfeeding—by making social media accessible for birth and breastfeeding professionals. She is a board member of Citizens for Midwifery and is active in local, statewide, and national birth and breastfeeding advocacy projects.

Photo credit: Scott Kemp

Updated 10-29: The URL of this post was changed due to technical issues. All comments submitted with a full name and valid email address have been moved to the new, correct, URL.

Thanks to both of you for putting this comprehensive and thoughtful research piece together... I really appreciated the part about what I call - either/or thinking - which cannot be useful or valid as we look at how and where we give birth. Women, families and babies are endlessly diverse and so the care that is available to them is most useful and therefore healthful when it reflects their diversity and their individual needs ...

As Melissa states so beautifully above "The answer to the question of where women should give birth is very likely more complicated than simply answering yes or no to whether one thinks home birth should be an option. I don’t think every woman is a good candidate for home birth, but I am also very frustrated with its wholesale dismissal by some as an option ever."

Post new comment

Thank you for commenting! Please be sure to include your first and last name below, which will be public. Please also include your email in case we have any questions about your comment. Your email will be kept private and not published on the blog. For questions about our comment policy, please read our Online Community Guidelines.

Your name: *

E-mail: *

The content of this field is kept private and will not be shown publicly.