Hierarchy of Methods

Last time, we considered Richard Carrier’s two common-sense ways of deciding whether a method for finding the truth about anything is reliable:

Does it (can it) make predictions that come true?

Does it produce a convergent accumulation of consistent results?

It’s almost comical to see how we humans prefer the worst methods and shun the best. Actors and sports stars earn millions for their paid endorsements because they exert a strong and measurable influence on our buying decisions, yet what could be less reliable than the bought-and-paid-for testimony of a non-expert?

At the other end of the spectrum, it is said that popular books on science lose half of their remaining readers with every equation. And those are readers who are presumably interested in science in the first place!

Although I have never paid much heed to celebrities and I love equations, I am shocked to look back on my life and see how unsound my epistemology has been. Even my logic and reason has often taken as its starting point the conclusions of less-reliable methods. I have been far too impressed with expert testimony, choosing to trust people because they shared some of my views. A shared (inbred?) set of views ought to have made me cautious, not trusting. I thought I was using the critical-historical method when in fact I knew next to nothing about what distinguishes a sound historical investigation from hear-say.

With all that in mind, here is Richard Carrier’s list of methods, from best to worst. Quotes are from his book, Sense and Goodness Without God: A Defense of Metaphysical Naturalism, chapter 3. As you go down the list, see if you agree that those near the top have the most “predictive success” and “convergent accumulation of consistent results.”

The method of pure reason by logic and mathematics takes first place. “The logical-mathematical method is supremely successful [because it has] produced the broadest, most complete and most consistent success. Moreover, when a proposition of logic or mathematics is challenged and seriously debated, the most widespread and solid agreement is achieved in comparison with any other method or subject.” Unfortunately, this method is not always available, so we continue with…

The method of science. Science is not perfect, but “we have proven again and again that the results of thorough scientific investigation are more reliable than the results of our own casual observation, producing far more extensive agreement and far more surprising successes, with the most impressive examples of convergent knowledge in history.” When careful science is not possible, the next best thing is…

The method of experience. Although “I saw it with my own eyes” may seem to establish the strongest possible case, Carrier cautions that “if someone comes up with a scientifically or logically well-proven claim that contradicts our direct experience, then we have good reason to believe that our experience is in error, because a single unexamined experience cannot possibly be more trustworthy than a hundred well-analyzed and tested ones.”

The critical-historical method is next. What distinguishes sound, critical-historical investigation from sloppy, biased work is a huge subject in itself. Carrier (an historian himself) says that our skills here are much better than they used to be, but this method still “takes fourth place [because] lacking the ability to observe its object directly, its results are as indirect as its evidence, and by being less direct, [are] less certain.”

Expert testimony is surprisingly far down the list. But when you think about it, expert testimony even at its best is only derived from the more reliable methods above. Sometimes, testimony is given on very thin evidence. As Carrier says, in some fields “we find very little agreement among qualified experts [i.e., little convergent accumulation of consistent results], and a vast influence of ideological bias that is rarely placed under any objective control.”

The method of plausible inference, like expert testimony, starts from the more reliable methods and extrapolates from them, so it must be less reliable. Still, it is sometimes valid to make inferences from a few facts to general principles.

Pure faith has proven to be the least accurate. This is not a slam on any particular religion; it refers to the method of faith in general, not your faith in particular. Many religious people cite evidence from the other methods to support their views, and you may be one of them. Still, Carrier urges us to consider the number of claims based solely on faith have been overturned through history. This ought to make us cautious about faith as a method.

What do you think of this list? Is it as sobering for you as it was for me? Do you aspire to shift your thoughts higher on the list?

7 responses to “Hierarchy of Methods”

I have mixed feelings about the list. In a lot of ways, everything seems to be divided between science and art, the absolute and the relative. The above list would be most applicable to the science of life. There is no scientific formula or equation to determine which form of music the best, for example; it’s purely opinion no matter how scientific someone or a group of people want to try and make it. The difference between science and art is that science tends to be discovery and understading of the unknown while art is an individual expression of the sum of something (a person, landscape, etc).

Additionally, in regards to the science of life, it can be broken down collectively or individually. Whether a person wants to live collectively or as an individual is a choice. This list is most applicable to someone who wants to live collectively. A person who wants to live as an individual would put experience at the top of the list and use the other methods (in some cases) to understand that experience. No amount of knowledge can compete or argue with someone’s experience, which is an individual perception of an event that has directly affected them.

Faith, why doesn’t it surprise me that it’s at the bottom of the list? Maybe because the Church as redefined it to mean something purely abstract so they could use it to lord over people. The Hebrew word for faith is amunah (אמונה). It means ‘firmness’ as in something built on a solid foundation. It’s parent word is aman (אמן) and it means ‘support’ as a father nurses a child (hence the reason why Jesus said a person must have faith like a child). A brother word is emet (אמת) and it means truth. The writer of Hebrews properly describes faith as ‘substance and evidence’ (Heb 11:1). There is no such thing as blind faith or believing in something in spite of the evidence, that’s not amuah, faith. Faith is firmly built on a foundation of truth. The foundation of truth, according to the Bible, is the Torah or Law of Moses which came from God, according to the record, who is the Father of Israel and supports Israel as a ‘nursing father’ (Ps 119:142,151, Num 11:12). The Church does not embrace the Torah, however, so they had to redefine the word so that they were the source of truth. And since their truth is relative, it has become arcane and is at odds with science holding, instead, to the superstitions it was based upon. (I have nothing against how the author uses ‘faith’ as he uses it in the context according to most people’s understanding of the word.)

@Joshua: While there is not currently a mathematical equation to determine which form of music the best, I find it easy to imagine that, given a reasonable definition of “best” (e.g. high average ratings of North American listeners) and enough study, people using the the scientific method could develop a formula (computer program) that would accurately predict which songs people will rate high or low, on average. It wouldn’t surprise me to learn that multiple people are working on this problem right now.

Joshua, let’s please use commonly used and accepted definitions for things. Definitions of faith may vary, but common to virtually all of them is a belief in something.

For example, I am writing this comment because I have faith that when I click “Post” on this comment, it’ll go up on the web, and that the electricity in my house will stay on long enough for me to finish writing the message. Maybe not! But I feel the chances are good enough to attempt to write a message. My faith is based mainly on prior experience (plus various bits of scientific and mathematical knowledge I have about computers and the internet). Because I’m relying on faith (a truth hoped for and not yet seen), I know there’s a chance I could be wrong, in which case I will lose this message and you’ll never see it. My past experience of posting messages successfully (and occasionally, unsuccessfully) is, I’d say, more reliable than my faith that this message will be posted.

You have a point that my faith (that this message will be posted) is built on a firm foundation… of experience. Thus experience scores higher.

My former faith in God was based on the faith and experiences of others; but after many years of church attendance, scripture reading and prayer, I received almost no personal experiences of my own to suggest God exists, despite fervent prayer in accordance with James 1:5-6. Now sure, maybe God never spoke to me for all those years because I didn’t believe firmly enough, or because I was a sinner (although it doesn’t say, there, that the sinner cannot communicate with God). But this post is about “deciding whether a method for finding the truth about anything is reliable”. Do you disagree about the evaluation criteria (predictive power, and “convergent accumulation of consistent results”)?

Having chosen evaluation criteria, how does faith stack up? Well, so far its predictive power has been very poor for most people, including many firm believers. Does faith produce consistent results? No.

Even if we all witness Jesus Christ come to visit, floating down on a light beam from space, as various satellites confirm His trajectory, even then “faith” as a general concept would still land at the bottom of the list. Why? Because Muslims, Buddists, native American tribes, African tribes, and even many atheists had faith too: faith in falsehoods. The predictive power of their faith was nearly zilch, and it didn’t produce a “convergent accumulation of consistent results”.