A report published on Thursday by the Royal Society highlights the effect that a rapidly growing human population will have on the world's economy and environment. People and the Planet recognises that technology will play "an increasing role" if we are to feed a global population estimated to reach 9.3 billion by 2050 without turning significantly more natural ecosystems over to farmland.

Is there a future for organic agriculture in a world where yields are so important? I'd argue yes, but only as part of a new approach that incorporates the best features of all agricultural systems.

The Royal Society report is published in the same week that new research confirms the yield gap between organic and conventional agriculture, but which also shows that, given the right crop and growing conditions, organic can almost match conventional yields.

Organic systems provide a number of tangible benefits over conventional agriculture in terms of biodiversity and soil conservation. However, given the need for some crops – particularly cereals – to keep pace with rapidly growing demand, the gap between the yields that can be provided by organic systems and the yields that are so desperately needed is growing.

In a paper published in Nature this week, US and Canadian researchers used a meta-analysis of available information to conclude that, on average, organic yields are 25% lower than those achieved in conventional agriculture. Depending on the type of crop, yield gaps varied significantly. For example, organic fruit production had just 3% lower yields on average than conventional fruit production, whereas cereal production was seriously hampered, with yields reduced by 26% compared with conventional methods employing chemical pesticides and fertilisers.

Organic systems performed better when high levels of organic nitrogen were present, the organic system was well established and rain-fed irrigation systems were used, though they still didn't match those of conventional agriculture.

Correct soil pH and the use of best management practices also narrowed the yield gap, leading the authors to conclude that "today's organic systems may nearly rival conventional yields in some cases – with particular crop types, growing conditions and management practices – but often they do not."

These new results support a study published earlier this year that also demonstrated a significant yield gap between organic and conventional. Researchers in the Netherlands used a meta-analysis to show that the yields from conventional systems were on average a fifth higher.

These data should spur an increase in the amount of research into organic agricultural systems, which is currently tiny compared with conventional crop production. While perhaps never reaching the maximum yields possible with conventional systems, the advantages of organic – including biodiversity and soil conservation – should encourage us to look more closely at this type of agriculture.

The key will be to move away from the hard-line ideology of an organic versus conventional debate and instead examine what features of all agricultural systems could be utilised in a multi-faceted approach, using complementary ideas from each camp. The importance of creating and maintaining high levels of soil biodiversity – a crucial component of organic agriculture – needs to be recognised in any new system. But the limiting factor of low levels of nitrogen in organic systems must be overcome.

As the authors of the Nature study put it, there should not be winners and losers in this debate. The result should be a combination of what is best about organic and conventional crop production:

"There are many factors to consider in balancing the benefits of organic and conventional agriculture, and there are no simple ways to determine a clear 'winner' for all possible farming situations. However, instead of continuing the ideologically charged 'organic versus conventional' debate, we should systematically evaluate the costs and benefits of different management options. In the end, to achieve sustainable food security we will probably need many different techniques – including organic, conventional, and possible 'hybrid' systems – to produce more food at affordable prices, ensure livelihoods for farmers, and reduce the environmental costs of agriculture."

We need a new agriculture: one that is not limited by ideology but is informed by science and that is relevant for an era of rapidly growing human population and an ever increasing demand for food.

Eoin Lettice is a lecturer in the School of Biological, Earth and Environmental Sciences at University College Cork, Ireland. He blogs at Communicate Science