I really enjoyed this interview with my colleague, Nancy Nersessian. (Yes, she’s a Professor in the College of Computing.) It helped me understand better why her perspective is revolutionary, and why she’s been racking up awards for the importance of her work.

One of her arguments is that they way we think about the scientific method is wrong, that our “received” notion of the scientific method is not how scientists really work. Rather than test hypotheses, scientists do experiments to influence their models of how the world works. The hypotheses they test come out of those models, and a “failed” experiment doesn’t disprove the hypotheses as much as it feeds more information into developing a more correct model. That’s another reason why failed experiments are so important — they lead to better models.

Georgia Tech’s Nancy Nersessian talked about a project that’s been running at her university since 2001 to investigate how bioengineering scientists think and work, and how to pass their skills on to students. Nersessian said that there is a “received view” of the scientific method — you formulate a hypothesis and then test it to either validate or invalidate it — and then there is the way scientists actually go about their day-to-day work.

In the real world of scientific investigation, she said, scientists usually rely on a model-based process rather than a hypothesis-driven one. They formulate models based on what they know from previous research and then derive testable hypotheses from those models. Data from experiments don’t validate or invalidate hypotheses as much as they feed back into the models to generate better research questions.

As I understand it, what makes Nancy’s work so powerful in the philosophy of science community is that she has provided important evidence of model-based reasoning in authentic settings. She did historical studies of scientists like Maxwell and Rutherford to show that her model-based reasoning can be used to explain their innovation and creativity. Lately, she has been studying activity in biomedical engineering laboratories to show how her models explain their work, and then she’s followed up with curriculum design experiments to create opportunities for undergraduates to learn those practices and ways of thinking.

Very interesting and it’s certainly making me think. I am puzzled as to how a hypothesis based approach could be fully divorced from a model – there must be an iterator of some sort for generating alternative hypotheses, even if it’s informal. I have to go off now and read all of Nancy’s work.

I’m not sure I’d characterize this as “the scientific method is wrong” so much as “the simple view most take of the scientific method is incomplete.” I also think it varies quite a bit by discipline. More mature disciplines (e.g., physics) are almost entirely about theory-based model testing, whereas this is less true in other scientific areas.

Although I had never thought about this in such as explicit way, this is something that I often talk to my students about. Frequently when we are writing up experiments, my students will want to discard any experiments that did not prove useful, ignoring the fact that it has changed the way that they view the problem they are tackling, their understanding of the model, and that “the failed” experiment is just as important to include.

We try to encourage student’s to adopt the same approach when dealing with problem solving – as software engineer’s we very rarely identify the exact solution to a problem immediately, instead, it is a process of experimentation, refining our understanding of the problem and exploring options. When we first get students in their first year of university, one of the challenges here is to move them from a view where they are focussing on the right or correct answer, to understanding the importance of the problem solving process, and the relevance of failure within that.

Thanks for the post – I’ll have to read more on this to see how we can better challenge this problem!