Monday, August 17, 2015

Cherry picking theories

I recently realised that some experimentalists cherry pick theories to describe their experimental data. I heard a talk by a theorist who reported having several disturbing conversations along the following lines.

Experimentalist: We fitted our data to your theory.

Theorist: But the theory is not valid or relevant in the parameter regime of your experiment.

5 comments:

This is not cherry picking - this is "fitting apples into the shape of cherries".

To me cherry picking implies choosing a dataset (model) because it fits the model (dataset) //among models (datasets) with equal attributes//. I.e. there is no reason to make the choice other than that it fits.

If the model (dataset) is not applicable to the dataset (model), it's plain wrong.

While cherry picking is bad (because it does not falsify the datasets or models that are not chosen! - and falsification is the only way to narrow down towards a "final" choice), choosing a model (dataset) that is demonstrably incompatible because of boundary conditions, is Wrong, as in "you Lie and that is Wrong". Capitalization on purpose!

However, lacking any alternatives and with honest caveats, I think there is deserved interest in the success of a theory beyond its apparent domain of applicability. The independent electron model is an obvious example of this, and required the breakthrough machinery of Fermi liquid theory to be fully understood.

You are giving the most generous interpretation of what some do.In the problematic cases I am thinking of experimentalists do not acknowledge -the existence of alternative theories-clearly state that the theory they are using is beyond the regime of the experiment.

In the worst case they say "because the theory fits the data, we have proven the theory is correct".

Tim, I would say that depends on how the domain of applicability was defined.If a model (equation) is solely valid because it was derived using a boundary condition ("parameter a has to be larger than 0", and if it is not, the resulting equations are incorrect), then applying this equation to an experimental system where parameter a is demonstrably negative is mathematically wrong.Stretching a domain, and decreasing the quantitative agreement can have its benefits indeed.

Subscribe To

About Me

I have fun at work trying to use quantum many-body theory to understand electronic properties of complex materials.
I am married to the lovely Robin and have two adult children and a dog, Priya (in the photo). I also write an even more personal blog Soli Deo Gloria [thoughts on theology, science, and culture]

Followers

Disclaimer

Although I am employed by the University of Queensland and funded by the Australian Research Council all views expressed on this blog are solely my own. They do not reflect the views of any present or past employers, funding agencies, colleagues, organisations, family members, churches, insurance companies, or lawyers I currently have or in the past have had some affiliation with.

I make no money from this blog. Any book or product endorsements will be based solely on my enthusiasm for the product. If I am reviewing a copy of a book and I have received a complimentary copy from the publisher I will state that in the review.