Thursday, June 2, 2011

Two Bayesian ways to assess null values. Which is better?

Psychologists (and any other researchers) have been trained to do data analysis by asking whether null values can be rejected. Is the difference between groups non-zero? Is choice accuracy not at chance level? These questions have been addressed, traditionally, by null hypothesis significance testing (NHST). NHST has deep problems that are solved by Bayesian data analysis. As psychologists transition to Bayesian data analysis, it is natural to ask how Bayesian analysis assesses null values. The article explains and evaluates two different Bayesian approaches. One method involves Bayesian model comparison (and uses “Bayes factors”). The second method involves Bayesian parameter estimation and assesses whether the null value falls among the most credible values. Which method to use depends on the specific question that the analyst wants to answer, but typically the estimation approach (not using Bayes factors) provides richer information than the model comparison approach.

That's the abstract from a recent article*, which you can find >here< in its pre-publication form or >here< in its published form. Note that the published form suffers from a production error in which Equation 1 was omitted from p. 301; see the pre-publication version for the intact equation! The article covers some of the topics in the book (Doing Bayesian Data Analysis) but in a succinct and stand-alone way. The article also has an example (loosely based on results of Bem's recent results regarding "feeling the future") showing how Bayes' factors can be extremely sensitive to the choice of alternative-hypothesis prior.

The article is part of a special section of the journal on Bayesian data analysis. Two other articles in the section can be found at the journal website (click the link to the published form, above).

UPDATE: See this additional example for another illustration of how the estimation approach makes more sense than the model comparison approach for null assessment.