“Robots” vs Environmental Managers

By Matthew Holden

Can automated algorithms do better than humans in conservation games?

Given all the real-world complexities involved when managing ecosystems, do quantitative methods (which ignore most of these complications) really help decision-makers achieve better environmental outcomes? How do these quantitative methods compare with the alternative: humans making decisions based on intuition, experience and their best judgement?

Unfortunately, it is difficult to answer this question in real-life because experiments in management are usually not repeatable. That is, once a manager “acts” based on their experience, it’s usually impossible to compare the results to how well an alternative decision, aided by a mathematical model, would have performed.

But what is difficult or impossible in real life can sometimes be achieved in the virtual world. Computer games allow us to pit human-based decisions against automated model-based decisions : “robots” following simple mathematical rules. For every game a human plays using intuition, a robot plays using predetermined instructions that are optimal given some quantitative model. The model, being a simplification, is an approximation so it is always, to some degree, wrong.

We explored this approach using environmental science university students as our test humans. We had them play an online computer game where the players tried to harvest a hypothetical salmon population in order to maximise long-term sustainable profit.

If the player harvests too few fish, they don’t make much money. But if they harvest too many fish early in the game, there are no fish in the ocean to harvest during the later turns. The player decides how many fish to take out of the ocean on each turn, balancing the future benefits of leaving fish in the ocean against the present profits from fishing. To allow the students to gain some experience managing the fishery, all players played a practice game before playing the game for a score.

Unbeknown to the students, robots were competing against them behind the scenes, making decisions based on simple mathematical rules. We then compared how well the students managed the fishery to how well the robots did.

Humans on average scored 63.6% of the maximum profit they could have achieved if they played the game perfectly. In contrast, the robots, even when using mathematical rules that were based on completely incorrect descriptions about how the salmon population changed through time, earned 78.9% of the maximum possible profit. On average the robots made better decisions than humans who used intuition and past experience from playing the practice game. This shows there is real value in the greater use of quantitative methods in environmental management.

One additional finding from the experiment is that, as the students progressed through the game, they did not get better at choosing the right number of fish to remove from the ocean. For example, students didn’t make better decisions on their 20th turn than they did on their second turn. However, the robots did improve while they played the game.

This may surprise you. Humans are flexible compared with rigid mathematical rules, so shouldn’t they be able to adapt as they develop new experiences? In our game, this did not happen in a way that improved the students’ score.

One possible explanation is that the students were reacting to the data they observed rather than using the data to test hypotheses. A next step in this line of research would be to ask the students questions during the course of the game to see if probing them to think about their decisions in a more structured way (like the computer “thinks”) actually improves their performance.

We’re not saying that humans should be removed from the decision-making process. Humans will be absolutely necessary for defining conservation goals (and revising them), engaging stakeholders, choosing appropriate models and analysing data, and deploying actions on the ground. While we would argue for increased transparency and objectivity (which modelling can help to provide), we are still a long way away from robots taking over the field of environmental management.

Matthew Holden is a member of the ARC Centre of Excellence for Environmental Decisions. He is based at The University of Queensland.