Other things on this site...

Unpredictable impact

There's a big change happening in UK science+engineering at the moment, and it goes by the name of Impact. What does it mean? When we do science we often do it just to find new things out, yet whether we intend it or not one of the great things about science is that it actually makes important changes to the world outside our research group. Impact is formally defined as being that effect that we have - on business and economy, on health, on public policy, on culture and the arts. There are billions of ways that impact spreads.

This has always been a very unpredictable thing and pretty hard to measure, so the government now has created a formal process for trying to account for the types of impact that we get out of research - and even further, to think hard about impact when deciding what research to fund. In a lot of cases the predicted impact will now account for up to 25% of the considerations in rating academic departments or allocating funding.

Sounds reasonable? Well many scientists are against it - and it's not because they don't like having to justify themselves (they already have to do that when they write grant applications etc), but because the real impact of science often happens in surprising ways, sometimes many years down the line. Take DNA fingerprinting for example. The scientists who came up with it were working with DNA, trying to measure various things, but they had no idea that the best thing they could do was make an unruly collection of DNA form patterns on a sheet of film - they discovered it by accident. And now it's an important part of many of the most serious court cases we have. Think of all the people who were convicted or freed based on DNA evidence - that's some serious impact there.

There are lots more examples of unpredictable impact - such as:

Email, when it was invented, was only able to send messages to people using the same mainframe. No-one predicted that tweaking it to send messages around the world would make it one of the most important communication tools we have.

Gregor Mendel - a lone priest planting peas in a garden, trying out different cross-breeds and making careful notes. It wasn't until years after his death that biologists realised how Mendel's laws of inheritance fit with Darwinian evolution, and formed the foundation of modern biology, with massive impact throughout society.

Texting. A phone is for phoning, right? Text messages were never planned to be the mainstay of what mobile phones were about, just a way to get a message through when you couldn't talk. But now many people text more than they call.

Liquid crystal displays eventually arose from the basically curiosity-driven research of Friedrich Reinitzer looking at the chemical cholesteryl benzoate. Now it's used in TVs, phones, watches...

Fibre optics was demonstrated as a curiosity and a demonstration of physical principles in the 19th century; but it wasn't until way into the 20th century that it became important for data transmission, for example in phone networks.

And the opposite is also true - history is littered with examples of discoveries/inventions that were widely expected to change the world, but didn't:

Video messaging: the phone companies seem to have thought that if we liked text messaging we were going to love video messaging. No.

Artificial intelligence: In the 1960s the artificial intelligence research community was an incredibly optimistic one, with leading lights such as Marvin Minsky basically thinking they would be able to recreate the intelligence of a whole human brain within a few years, and then we'd all be having conversations with robot pals. That optimism came crashing down. Sure, you can now buy robot pals, and sure, we're still researching artificial intelligence and indeed using it in various applications, but it hasn't yet been the revolutionary impact it was going to be.

Hovercrafts and maglev: these have become the clich�s of misplaced futurology. After their invention they seemed to have been poised to take over the world - but no, we're still mostly using the good old wheel to get around.

So with all this evidence, it's not surprising that scientists are worried about this new approach of trying to plan your impact - much of the curiosity-driven stuff that has real impact could well get sidelined in favour of things which might be a bit less imaginitive but which seem like they'll definitely make some public or business connection.

OK fine - seems like there's some misguided bureaucracy coming down from government, and we have to try and make sure it doesn't end up stifling what it's supposed to be helping. But there's a bigger question that maybe we can think about. As I've said, "impact" is very hard to pin down or predict, and we don't really know how predictable it could or should be. But in many grant applications and suchlike, scientists are now writing down their predictions about the impact they'll have. Are those predictions useful data? Could we use "impact plans" as a great big study about whether impact can be predictable?

We could for example wait for five years, then look back at the pile of impact plans and ask, how many of those predictions (the ones which got funded, at least) came true? What percentage? What proportion of the observable scientific+engineering impact made over the next five years will have been predicted, in writing, in advance?

It would still leave a million questions unanswered, especially about unidentifiable impact (subtle things which are hard to count), long-term impact, and really it would still be a very reductive way to think about how science affects our society. But I wonder... would that make all these "impact statements" worth their while?

POSTSCRIPT: Some further examples of unexpected impact, collected after this article was written:

Nice unpredictable-impact example from Martin Rees in his Reith lecture (Tuesday 1st June 2010), of the laser: its possibility was there in Einstein's ideas at the start of the 20th century; then there was a 40yr gap before it was actually conceived and made to happen; and the inventors of the laser would never have predicted DVD players and laser eye surgery 40yrs after that...

"Measurement of the ratios of heavy and light isotopes of oxygen, once a limited area of geochemistry, eventually allowed the interpretation of prior climate change." (Quoted from this Science editorial 2013)

• Name: a reader
• Website:http://www.areader.com
• Email: areader art areader dort com
• Date: Saturday 5th December 2009 01:47
I agree. Impact as specified and described in funding proposal is, at best, speculative. In addition, considering the pressures and limited funding resources, researchers tend to overhype the foreseen imact of their proposals. Unfortunately, the judges of the grants applications, at the same time, have little time and knowledge to review each application. This leads them to relying on "metrics" such as "speculative impact", which hardly qualifies as a metric in the first place. Even more established metrics, such as "impact factor" of journals or conferences have been critisized extensively (e.g. see contents of journal "Scientometrics"). In my view, this is a phenomenon of our society that seems to lean toward "the fast and the furious" science, where fast results is becoming the norm of a scientist's survival, with only lip service paid to how **really** good and how long term beneficial those results could be. This attitude can have catastrophic consequences for not only the science enterprise but for the human race as a whole. Unfortunately, not many seem to pay attention to this; and the ones that do, are quickly put at the sidelines.

• Name: Dan
• Date: Tuesday 1st June 2010 20:48
Nice unpredictable-impact example from Martin Rees in his Reith lecture today, of the laser: its possibility was there in Einstein's ideas at the start of the 20th century; then there was a 40yr gap before it was actually conceived and made to happen; and the inventors of the laser would never have predicted DVD players and laser eye surgery 40yrs after that...

Add your comments:

Name:

Email:

Website:

Comment:

I am a:

(Note: only if you enter "human" into this box will your comment be processed. The reason
for this is that I've had hundreds of pesky robots trying to submit spam to my site.)

Everything is optional - and email addresses will be marmalised to protect you