In the previous post I linked to Dr. Linda Chalker-Scott's article about compost tea and there's a little sidebar there that deserves its own post. See, Linda's field of study is called "synthetic" research, meaning she assesses everyone else's research. (Who knew?) And here's her advice for us laypeople who are trying our damnest to make sense of often conflicting advice.

Questions to ask when assessing scientific objectivity and credibility of gray or popular literature

Does the author have legitimate ties to a mainstream academic or scientific institution?

Does the article refrain from attempts to sell a product?

Does the article present verifiable information?

Does the article appeal to reason rather than emotion?

If the answers to these questions are “yes,” then the likelihood is high that the information is objective and credible.

I don't know if Linda had the Soil Food Web lady in mind when she wrote this list but it sure applies – at least number 2, and possibly others. I especially like the caution about appeals to emotion – whether in horticulture OR politics.

14 responses to “How to judge “expert” advice”

It only makes sense to me when doing research to seek out credible professionals within their field of expertise but as one who has been entrenched in academia for a number of years I have found that even the so called professionals will have conflicting theories, advice and opinions between themselves.
At this point it often comes down to who you really trust and respect such as Peter del Tredici at the Arnold Arboretum or Don Mahoney at the S.F. Botanical garden .
Another interesting point in case is when the professional, him or herself , revises their own opinions and findings after many years of practice ( I’m thinking of Dr. Shigo the famed arborist whose has contradicted his own findings over the length of his career ).

Sometimes I am surprised that more garden folks haven’t keyed in to the whole trend of “evidence-based medicine” from the world of health care. Much of what the good professor discusses is “evidence-based gardening”. Physicians now must routinely hold up to scrutiny almost every diagnostic tool and treatment regimen once handed down as the authoritative truth based on “expert practice”. Many once-common techniques in medicine have fallen by the wayside thanks to a rigorous analysis of outcomes. Much of what is discussed in the above post is the basic approach advised to doctors when approaching a published study in a medical journal.

This is exactly what my guest rant this summer was about – too much writing is in some way represented as credible and too few people are informed on how to identify solid research and understand the writing. Thanks to Dr. Chalker-Scott for these guidelines!

It is critical that we gardeners, working in a field full of ancient advice, wive’s tales and snake oil look to the scientific method to determine what is truly effective.
Just because we keep doing things the same way doesn’t mean it is the best way.

To Michelle D: one amazing thing about science is that when new information comes to light, scientists reexamine their work. Some of those scientists may have a harder time changing their opinions, but they can’t refute good science.

I can understand this advice, however I write for a Garden Tool Company in the UK (www.qualitygardentools.com) and I try constantly to get people to use ‘the right tool for the job’. Yes, we do sell tools but my ‘expert’ advice is designed to help gardeners, not sell to them. A classic example is when gardeners use garden rakes for clearing leaves (these rakes are designed for levelling soil and removing stones etc and can damage lawns and pathways), this practice is just one example of gardeners needing education. Tools are dangerous and many of the injured gardeners who write to me have either used tools for a job they were not designed for, or have used the wrong technique. If I didn’t write my blog for a tools company I would find it hard to get paid and 18 years of experience in the garden tool industry would be lost. Experts are not always trying to sell products, but they often need to work for companies to ensure they get paid!

The other thing to take into account — and this is why having ‘synthetic’ researchers like Linda is so valuable — is how many other studies produce similar results. One study may show benefits to a particular product or practice. But if many studies point to the same thing, the results are more ‘generalizable’.

What’s really important, though, is whether or not you can use the information in your own garden. That depends a lot on similarities between your conditions and the test conditions, including:

Climate: What works where the tests were conducted might not be transferable to your climate.

Soil: What works on the test site’s loam might not work on your clay.

History/’system’: What works on a research plot carved out of a farm field might not work on your long-established perennial flower beds, raised bed vegetable garden or lawn.

Some might get frustrated with researchers when they seem to hedge their conclusions. But I respect them when they say, ‘this is what we found in these years under these conditions’. They’re not supposed to make sweeping generalizations.

I appreciate the work that Linda Chalker-Scott does. We are bombarded with many differing “scientific facts,” regarding gardening. It’s encouraging to know that there is a field such as “synthetc research” that sorts out the facts for the lay person.

I’m also a huge fan of Linda. She’s got a great sense of humor too. Imagine, a “professor” with a light, fun side!

Master Gardeners are taught to look at three resources – prefereably on university or government websites – before giving advice to the public. In my column, I often link to websites with good advice, but always caution the reader when some of the advice is not applicable to our soil, climate or other conditions.

Master Gardeners? PLEASE….. We threw Master Gardeners out of our anual spring garden shows beacuse of bad advice they gave to our customers. Items such as suggesting chemicals that have been off the market for years, new lawns do not need fertilizer for three years, top soil is “endangered” use cow manure instead for soil. The list goes on. I cannot tell you the arrogance at which they treated my staff at the shows because after all “We’re Master Gardeners”.
A bus load came to our store on tour using the same arrogance when I thought they were customers and offered to answer a question…”No, we do not need your help. We’re Master Gardeners”. I swear on my bible this is exactly what they said.