Do correlation statistics create bad policy?

UNconventional Wisdom is a periodic guest blog where the conventional wisdom is held up for fresh inspection, often with divergent recommendations. If you have some "UNconventional Wisdom" to share, email [email protected]

Recent months have revealed that many policymakers are basing their policy decisions on correlations in data, rather than looking for the actual causes of problems.

For example, Consumer Financial Protection Bureau research on payday loans shows correlation between overdraft fees and deposit advance loans. Are they related? Or are they both symptoms of a deeper financial problem? Will a proposed policy change be effective if it ignores such underlying issues?

This leads to a deeper problem plaguing our nation—a lack of understanding about basic mathematical principles and critical thinking. Perhaps the causation that should lead to new policy is this:

A nation that doesn't understand basic statistics is bound to be misled by bad statistics.

There are numerous examples of questionable usage of stats in policy debate, but it would be instructive to consider a few that are currently in active use.

College grads earn more

College graduates earn substantially more pay over the course of their lifetime, it is stated. A recent National Center for Educational Statistics study suggests lifetime advantage, after expenses, averages $500,000. The statistic is not in question, and it is appropriate to share if your children are considering options for their future.

However, many politicians and advocates are using this statistic to support their proposals to reduce poverty. In short, send the impoverished to college and they will become new members of the middle class. There are individuals who succeed on that path, but it is a horrible policy idea.

First, consider that many college graduates have not been successful at all.

An unprecedented number of recent grads are still unemployed and living in their parents’ basement. Many more are underemployed compared to their supposed potential. With that to show for their efforts, we now have $1 trillion in student loan debt.

At the opposite end of the scale, students who specialize in computer science, engineering, and statistics earn two-to-four times more than their peers. This demonstrates that the type of degree earned is also important when considering this statistic.

Simply checking the college degree box may improve the odds of an interview, but it is hardly the golden ticket to success many education advocates claim. A stronger argument might be that a break year before college will empower students with a level of maturity in their studies; time to volunteer or work in an industry they wish to pursue; and an opportunity to save for their education to reduce the amount of student loans they will incur.

Small class size improves learning

It is hard to argue with the oft-used axiom that a smaller class size increases student achievement—unless you look at the data behind the statistic.

First, some of the best classes taught in the U.S. are huge. There are college professors with class sizes of hundreds of students. While one might gain more from a smaller class with those remarkable instructors, the associate faculty and teacher’s aides are often less effective.

The lesson is: The best teachers are already teaching. Adding more to the system to decrease class size simply increases the odds of getting a less competent one.

Second, a recent conversation with a high school teacher revealed another commonly accepted “fact” in the class size statistic. Her school claims a student-teacher ratio of 15 to 1; impressive to any parent seeking personal attention for their child. The problem is that most classes, including hers have as many as 45 students.

How is this possible? Because of a fallacy behind the statistic.

The fallacy of this statistic is that many “teachers” in her state are working as counselors, administrators, or special education tutors. This commonly used stat is misleading to the audiences it is presented to.

Home owners make better citizens

This theory had at least a minor role in the housing collapse. Many housing advocates present statistics supporting the argument that home owners score higher on nearly every desirable scale: income, stability, child success, money saved, credit score, taxes paid, and even lower obesity.

It is clear to them that as a nation we should maximize home ownership. Sadly, this is a textbook example of correlation without causation.

Using the simplest measure, it is, or should be, hard to buy a home without income and a good credit score. The proof of this came when Fannie Mae and Freddie Mac sought to increase home ownership among subprime consumers. They had to reduce or eliminate both income requirements and credit score requirements.

In the aftermath of the crash of 2008, we saw how well these policies worked in the real world. A consumer who can't afford their home and has a proven history of not paying their bills will not magically become a model citizen as a result of home ownership. It's hard to accept that someone with zero equity in a home is actually a good candidate to be a home owner.

Following the crash, regulators demanded that banks return to reasonable lending policies: proven income, decent credit score, and a down payment.

Advocates began their protests even while the market was still reeling from the crash. The response to them should have been, "Get your statistics straight before you present them." If only more members of Congress knew to say this.

Payday loans trap consumers

CFPB research showed that direct-deposit-advance users are much more likely to get overdrafts than nonusers. Sixty-five percent of users had at least one overdraft during the 12-month study compared to just 14% of nonusers. A strong correlation. Further, the median user had an outstanding direct-deposit advance 7 of the 12 months during the study.

There is no doubt that a relationship exists, but is the elimination of payday advance loans the solution for reduced overdrafts? Or, will the elimination of overdraft fees reduce dependence on advance loans?

An objective review shows that people in financial distress turn to both of these products to manage or cope with their bills.

If both products are eliminated or reduced, are consumers helped? Certainly, they will have two less tools available to manage their finances. While these are clearly not desirable products in most situations, they represent options for consumers in need. Before developing policy based on correlation, we should step back and consider the root causes and how to address them.

How can we improve policy decisions?

One of my favorite movies is “Trading Places,” starring Eddie Murphy.

At the core of the movie is the question; is it the position or your experience that creates a successful trader?

In this fictional story, the con-man made Wall Street investor adapts beautifully and even takes pride of ownership in his free new home. Reality works a little differently. People protect what they worked hard to earn and esteem that which came with difficultly.

We need more citizens who understand the basic principle of causation versus correlation to truly have a positive impact on policy. To accomplish this, we need to rethink the way we approach educating. One component of Common Core that works is its emphasis on critical thinking. It is possible to learn common sense and teachers should be rewarded for incorporating it into their lessons. This will only succeed if they are willing to embrace students asking questions and challenging their instructors’ knowledge.

Instead of home ownership for every citizen, we should be advocating for an education that prepares them to understand the implications of their decisions—before creating policy that ignores the core issues at hand.

About the author

Eric Lindeen is the marketing director for Zoot Enterprises Inc., a global provider of origination, acquisition, and decision management solutions for large financial institutions.