Tuesday, September 13, 2016

Is Prediction an Exquisite Fiction?

As I described in a previous post, a long-standing topic of discussion
is the usefulness of a given scientific endeavor or study. Along these lines,
science is often divided into BASIC and APPLIED. Applied science is – by
definition – useful. It cures some disease. It improves crop levels. It saves
some endangered species. Basic science
is – at least classically – not obviously or immediately useful. Instead, it addresses
a (hopefully) interesting question – interesting at least to the researcher. Sometimes
called “curiosity-driven” science, basic research might one day have great
utility but, at the time it is conducted, its uses aren’t obvious.

Basic science was once considered an admirable pursuit –
perhaps even preferable as an intellectual, university-based enterprise. More
recently, however, universities and funding agencies want to hear how your
research – whether basic or applied – will have “broader impacts” or “direct
benefit to the people of ...” No longer is it enough for the science itself to
be interesting and clever and well designed; it also has to have a clear
utility. When justifying a research project, these pay-offs are expected to be
clearly and forcefully presented, usually at the outset of a proposal and in an
explicit section at the end.

For basic scientists in ecology and evolution, these applied
justifications tend to involve conservation (e.g., saving some endangered
species or place), management (e.g., of natural resources), discovery (e.g.,
new drugs), or ecosystem services (e.g., greater biodiversity generates greater
productivity or resilience or whatever). In many cases, the specific link
between the science and the proffered application is PREDICTION. For example,
“we need to be able to predict what is going to happen, in the face of
environmental change or management actions, if we are going to design effective
strategies for conservation or management.” This sort of justification is a
natural and easy one because we can always say “If we don’t understand the
system well, we can’t predict it. My research will help us to understand the
system better, which will improve prediction, which will be useful, right?”

Just last week I – along with 21 other scientists –
published an opinion/review paper in Science amplifying this last point. Specifically,
we need to predict what will happen with climate change and – to do so accurately
– we need much more information about organisms, communities, and ecosystems
than we currently have. In this post, I
would like to play Devil’s Advocate to my own paper by arguing that prediction
is often hopeless.

From our Science paper.

A first important distinction is whether we wish to make a
prediction or whether we wish to make an ACCURATE prediction. It might seem
obvious that we want the latter but even the former is sometimes hard. That is,
we might not have enough information about a given system to even speculate
effectively as to whether or not some action (e.g., climate change) will have a
particular effect on a particular species. Most of the time, however, we are
able to make some sort of prediction based on intuition or similar systems or
mathematical models or experiments or whatever. So the real concern becomes
“how correct (accurate/precise) will be our predictions?”

The accuracy of prediction will depend on the type and precision
of prediction. For instance, we might first want to predict simply WHETHER a
given environmental change or management action will have an effect at all. Here
we might be safe in many instances. Will climate change influence biological
diversity? Yes! If the environmental change is large, something will respond to
it. However, this isn’t the sort of prediction that we – or the public or
managers or governments – care about.

We might next want to predict the DIRECTION of an effect. In
some cases, this will work fairly well. For instance, we can safely say – based
on many examples from nature – that climate warming will advance the timing of
reproduction of many plants and animals and that commercial fisheries will lead
to smaller body size in harvested populations. A few exceptions will certainly occur
but these will tend to be of the type that “prove the rule.” In many other cases,
however, predictions as to the direction of an effect will be incorrect. Will
climate warming increase or decrease local biodiversity? Hard to say. Will fish
harvesting increase or decrease productivity? It depends. In such cases,
increased information – including from “basic science” – might improve
predictions.

Experience teaches, however, that expectations developed from theory,
from related systems, and from detailed information are – not infrequently –
incorrect.

At the most precise level, we might want to predict an
effects size, such as a particular rate or endpoint state. How fast will
species be lost with climate warming? How many species will be present 25 years
from now – and where will they be? How small will harvested fish become and how
quickly will they recover when fishing ceases? I suggest that – in many cases –
predictions of this sort will be hopelessly inaccurate, except perhaps by blind
luck. Each system (and year) has so much contingency that prior information
will not be sufficient. Of course, this is precisely the logic that we invoke
when seeking funding: “We can’t make accurate predictions unless we get more information,
so give me some money to get it.” It is certainly true that if one had complete
information on the driving forces in any given system and complete information
about how those driving forces will change in the future, then accurate
predictions of endpoints and rates might be possible. But this “complete”
information is generally unattainable.

In short, many of the arguments one reads in proposals that the
particular basic science being proposed is critical for better prediction are really
just smoke-and-mirrors or, perhaps more accurately, a bait-and-switch. Five
years later: “Although I didn’t make better predictions, I did do some cool
stuff anyway, no?” Of course, these studies can also weasel out of
accountability by saying “Here is some new information that other people might
find useful in making better predictions” or they might say “Here are some new
predictions.” – with the last being particularly disingenuous because the
accuracy of those predictions won’t be known for sometimes decades.

My point in this post isn’t that basic science should be
abandoned in favor of applied science. My point instead is that it would be
nice if we could all just drop the applied BS at the start and end of our
proposals. That isn’t why we are doing the study – it is just what we think the
reviewers want to hear. The reality is that science has made incredible strides
in the past few centuries – and most of those advances, I will speculate, were
made by basic rather than applied science. Think of all of the ramifications
Darwin’s theory or natural selection, and – coincidentally – all of the
incredible and amazing applications. At the time, however, Darwin – and the
people who read his book – didn’t focus on its potential applications but
rather its potential to explain how the world around us came to be.

I had better circle back to that Science paper for which I
am here playing Devil’s Advocate. It is certainly true that we don’t have enough
information to make good predictions of how biodiversity and species ranges
will change with climate change. It is also true that getting more information
about those species and environments has the potential to improve predictions –
although we won’t know if we are correct for decades. Thus, I am not disputing
the main arguments we made in the paper. Instead, I am using it as a jumping-off
point to argue that additional information is probably even more useful simply in
improving our understanding of the world around us, whether or not we attempt
predictions. Sometimes this improved basic understanding will eventually have
massive benefits for biodiversity and the humans that depend on it.

I think it cheapens, and potentially slows, progress in
science to require it (or encourage it) to have obvious immediate applications.
The best route to the best possible future applications is to simply turn
researchers loose to study what they feel is most interesting, whether applied
or basic. Basic research isn’t flawed and in need of an applied crutch to hold
it up.

---------------------------------------

After I posted this, I was told about a similar post on Dynamic Ecology:

6 comments:

I agree completely. Without making this any more political than necessary, I would note that when there are members of Congress and candidates for President trying to make political hay out of ridiculing scientific research (fruit flies? why on earth should we be spending American tax dollars investigating fruit flies??), spelling out applications for one's research is a form of self-defense. And when the NSF and other funding agencies require such justifications in grant proposals, that is a form of self-defense on their part, too. Basic research is a dirty word nowadays because politicians have made it a dirty word. Until those politicians get punished at the ballot box for their ignorance and demagoguery, I have little hope that things will change for the better.

Hi Andrew, I have no problem with anything you wrote above as long as we accept that our inability to predict is clear evidence of how little progress ecologists have made in understanding how the world works. This may be because it is extremely difficult (impossible?) to understand how the world works or it may be because we haven't done as good a job as we could have. My guess is that it is more the former than the latter - most ecologists I meet and work with are extremely bright, hardworking and committed scientists. When we can only make predictions like warming will usually speed up ecological processes it demonstrates that we know little more than common sense and physiology would tell us. So, I agree that we should stop making claims that we can't back up (that we can make good predictions) but prediction is how we demonstrate understanding and it should be our centerpiece. The fact that we can't make good predictions now simply demonstrates how far we have to go. Best, Jeff Houlahan.

I think we have made great progress in ecology and evolution but I also agree that we have a long way to go. However, I suspect many aspects of many systems will remain unpredictable at the level many people want. Nature is complex and contingent and context-dependent.

Hi Andrew, you say that "we have made great progress in ecology..." - what would the evidence for that statement look like? If you had to convince a critical and rigorous audience of the truth of that statement how would you do it? Best, Jeff.