From a relatively small $3 billion market in 2003, a UN report recently estimated the total amount invested in food commodities had jumped in size to more than $55 billion by 2008.

However, gambling on the price of food comes at a cost. The high volume of trading in food commodity funds is leading to higher and more volatile prices, say campaigners, which affect poor families in less industrialised countries the hardest as they can’t afford basic foods and also make it more difficult for farmers to plan and invest.

Charities like Christian Aid and even UN representatives are now calling for tighter regulation to curb volatile price rises – something the banking industry is strongly opposing.

To help you navigate the arguments being made by both sides, former city broker Brett Scott provides this guide to understanding food speculation

Real Food vs. Derivatives

It is important to recognise the distinction between the real global markets in food, and the agricultural derivatives markets that are referenced to those real global markets in food. A Ukrainian commercial farmer exporting wheat to North Africa is engaged in a different undertaking to someone entering into a futures contract based on wheat. The former is a physical ‘spot’ transaction, and the latter is a synthetic ‘derivatives’ transaction.

Derivatives are contracts between counterparties. There are ‘exchange-traded’ agricultural futures and options, and there are ‘over-the-counter’ swaps and forwards facilitated by dealers at investment banks. These derivatives contracts are, more or less, analogous to bets on the real market in food, much like a sports bet is a bet on a real world sports event. Taking a ‘position’ in wheat futures, for example, is putting oneself in line to gain or lose from a change in the price of wheat. It is conceptually similar to, but not the same as, buying real wheat. It is as if one were buying wheat.

The original purpose of agricultural derivatives was to allow producers of real commodities to use a bet to protect themselves from decreases in commodity prices (short hedging), and users of real commodities to use a bet to protect themselves from increases in commodities price (long hedging). If you’re not a producer or a user of real commodities though, all you have is a bet, and then you’re called a speculator. The controversy around ‘food speculation’ then, refers to speculation in agricultural derivatives markets, not literal transactions in physical grains.

The controversy can be broken down into two separate issues. Firstly, are financial players in commodity derivatives markets causing derivatives prices to disassociate from what the price ‘should be’ if it were reflecting the fundamental balance of supply and demand in the underlying commodity? Secondly, does such a disassociation in futures prices get transmitted into the real price of food people end up paying?

They might seem like straightforward questions, but they’re not. Can you, for example, assign causation between speculative activity on an American derivatives exchange and a literal change in the cost of a bag of wheat in South Africa? It is this grey area that allows the debate to fracture into confusing technical trivia.What determines the price?

Most financial theory assumes futures prices are determined by the thing the future is betting on, much like a horse race determines the value of a bet on that horse race. Indeed, in financial theory the prices of derivatives are structurally tied to the value of the underlying asset that they’re based on, and are supposed to be kept that way through a process called arbitrage, which ensures that if the price of the derivative moves too far away from the spot price of the underlying asset, it is pulled back towards the spot price.

Here’s a real world example from the oil markets: In early 2009, oil futures prices rose ‘too high’ relative to the spot price of oil that could be purchased in the present. Arbitrage traders bought physical oil, stored it in tankers, and simultaneously sold futures. The effect of the transaction is that they managed to buy oil at one price in the present, and simultaneously lock in a higher price for selling it in the future. This type of trade theoretically allows traders to secure a risk-free profit, provided that the cost of ‘carrying’ the oil (e.g. storage, insurance, finance costs) does not exceed the difference between the spot and the futures price. Theoretically, through this process, the spot and futures price should come back in line in order to remove the risk-free profit opportunity.

That sound complicated? It is pretty complicated. For example, who actually has the ability to charter ships to store oil? Does it matter which grade of oil is stored? Does this bring the futures price back down, or does it bring the spot price of oil up? I get the impression that nobody really knows, because the fact of the matter is that, in commodities markets, arbitrage processes are complicated because actually accessing the physical commodities to enforce arbitrage relationships is difficult.

The conventional wisdom regarding the mathematical relationship between spot prices and derivatives prices has thus been challenged by concerns that this relationship can break down, and actually invert, such that underlying spot prices begin to move towards disassociated derivatives prices, not the other way around. It’s analogous to the idea that betting on horses can determine the outcome of a horse race, a classic case of ‘tail wagging the dog’.Why banks started betting on food

The fact that physical commodities are dirty and cumbersome means that speculation in physical commodities has generally been dominated by specialist physical trading houses like Glencore and Cargill. There are indications that banks are increasingly getting involved in physical trading as well, but initially at least, speculative activity by financial players developed in the commodity futures markets. Financial classics like Market Wizards, published in 1988, have fascinating interviews with speculative traders describing their exploits in the 1970s soy, corn and wheat futures markets, where they operated alongside those that used the markets for non-speculative purposes. Many of these traders would now fit into the mold of hedge funds, or ‘proprietary trading’ units of investment banks.

Futures though, need to be managed on a day-to-day basis. That’s fine for a specialist hedge fund, but not for ‘institutional investors’ like pension funds. If hedge funds are like nifty raptors, pension funds are like slow-moving brontosauruses, often oblivious to the day-to-day ups and downs of the market. Most pension funds investing in stocks, bonds and property do not have the time or inclination to jump into and out of futures contracts. They like to buy something that can be held for several years, not contracts with 6-month expiry dates.

The whole idea of ‘investing’ in commodities then, really only came with the advent of ‘index investing’ products designed by investment banks in the 1990s. These include products like exchange-traded fund and commodity index-linked notes. These presented an appealing story for investors: The investor simply puts money into the product, and the investment bank that manages it executes a trading strategy in the background to give the investors a return directly linked to a commodity index, as if the investor were buying real commodities, but without the hassle of it all. Over time, these products have allowed the number of financial players in commodities to skyrocket.

Commodity Index products though, are actually based on futures. Here’s the technical part: To create an index product, dealers at banks enter into ‘swap’ transactions with investors, through which they pass returns from the futures markets on to those investors. Investors don’t necessarily know this – the swap transaction may be embedded, or implied, in a packaged investment product – but the net effect remains the same: The dealer is basically holding futures positions on behalf of investors, and the activity in the swaps market, as it were, creates a ‘shadow’ in the futures markets. For those who are interested, this shadow can be observed in the CFTC ‘Commitment of Traders Reports’, which display futures trading position held by market participants. The category called ‘swap dealers’ are always shown holding ‘long’ positions, which means they are buying futures, probably on behalf of investors.Is speculation to blame?

But why does all this technical stuff matter? The vociferous debate centres on whether the increased involvement of these financial players has led to disruptions in commodity futures prices. It all got rather heated following a 2010 report by two academics, written on behalf of the OECD, casting doubt on any connection between ‘financialisation’ of commodities markets and the huge price spike in commodities in 2008.

The nuances of the debate are too numerous to list here, but some straw men are worth tearing down. This is not a debate about whether or not speculators are good for markets in principle. Finance textbooks suggest that speculators are useful because they provide liquidity, which simply means they increase the number of orders in a market, which in turn increases everyone’s ability to trade. In so doing they are thought to enhance the process of rational ‘price discovery’ in markets by allowing the balance of supply and demand to effectively express itself in clear prices based on fundamental considerations.

There’s no doubting that speculators can play an important role in the functioning of markets. I, for example, spent two years operating in some of the most illiquid markets in the financial world, newly developing types of exotic derivatives. I longed for speculators to come in, just to get things moving. The key issue though, really concerns the degree of speculative involvement in markets.

It may be the case that a market with 30 per cent of activity accounted for by speculation might work well, but what about a market with 60 per cent? What about 85 per cent speculation? These are not linear relationships – a market does not inevitably get more and more efficient as more speculators come in, and it is easy to imagine that there’s a tipping point where too many speculators destabilise prices rather than help them. Most speculation is short-term trading for short-term profit and it works if it’s done amidst a market concerned with long-term fundamentals. But what if it’s done amidst a market that’s already largely constituted by speculators? That’s speculation on speculation, and that’s how bubbles form.

Any trader could tell you that, but the dry technical debate drones on in academic journals, devoid of the much richer source of colour on the issue that comes from those who are actually involved in these markets. Indeed, an informal subset of this debate is actually found in the financial trading world, rephrased in terms of the distinction between fundamental and technical traders.

A fundamental trader is a speculator concerned with speculating on supply and demand of commodities. A technical trader is a speculator concerned with speculating on market patterns formed by the actions of other traders. Fundamental traders are worried that the increasing number of technical traders is increasing the randomness and ‘noise’ found in markets. If technical traders base their decisions on other traders, and their numbers are increasing, the market becomes circular and self-referencing and runs the risk of disconnecting from supply and demand concerns altogether.

The key to trading in that environment then becomes a variant of the ‘greater fool theory’ – running with trends and trying to get out before everyone gets psyched out. This is no secret.Why not all investment in food is bad

Focusing on the issue of short-term speculation though, may obscure a major concern in the markets. Pure speculation on the part of proprietary traders and hedge funds seems to exacerbate upward and downward movements in markets, creating volatility, but how does this relate to the activities of longer term players like pension funds? It’s not apparent that institutional investors overtly ‘speculate’ in commodities. A pension fund does not put its money into a commodity ETF, only to take it out a week later. They’re thinking about long-term trends, not short-term oscillations. They’re ‘investing’.

A further difference is that, unlike hedge funds that are able to ‘short’ markets to bet on a decline, large institutions tend to be ‘long only’, in that they only invest in things they think will go up in value. In practice then, if a large number of institutional investors decide to invest in the commodities markets through index products, the indirect result is only an increase in the demand for futures contracts. Derivatives markets though, are zero sum games – for a contract to exist, there has to be a buyer and a seller taking opposing views. If institutional investors only constitute a source of demand, but not supply, the question then emerges about who sells those futures contracts, and whether the index investors represent a structural source of upward pressure on derivatives pricing.

There’s a curious argument found in the OECD report that caused controversy in this regard. It’s the idea that, because derivatives markets are zero-sum games, these new investors have to be matched by sellers, and thus by definition cannot represent a source of excess demand in futures markets, and thus have little effect on the price. It’s a weird angle to take, because anyone involved in derivatives markets knows that if participants in general are more interested in buying, the price of the derivative must move up to induce others to sell.

The degree of that increase can be a function of the depth of a market – a smaller market has less ability to absorb big orders. Imagine Sainsbury’s arriving at a small village fruit market in search of stock for its shelves. It’s simply not true that sellers naturally materialise without a disruption to price, and it seems plausible that a drastic increase in money flowing only one way into futures markets could have an upward impact.

There are many more technicalities in the debate, including the physical dynamics of commodities, strange words like ‘contango’ and ‘backwardation’, ‘convenience yields’, and ‘rolling’ of futures contracts. There’s the potential for circularity of index products – the fact that they seek to track the commodities market by investing in the commodities market, but as they get bigger, more of the commodities market is constituted by their positions, which means they might sort of track themselves. There’s the related issue of the potential for an increase in correlation among commodity derivative prices.

Index investing is often based on a ‘basket’ of commodities, which simply means an investor buys a product that exposes them to a broad array of commodities at once. This translates into simultaneous purchasing of different commodity futures, which might create pressures for a range of commodity futures to move in tandem. This poses a potential problem for diversification – if money flows out of index products then the whole commodity futures market might crash at once, rather than just single commodity futures.It is never possible to predict exactly how markets respond to new pressures. Trading decisions are not determined by abstract economic models of rational behaviour. Academics might use mathematics to prove that markets are efficient, yet nobody pays attention to that, and certainly don’t waste time ensuring that markets are abiding by static theories. Markets work on dynamic human strategies and emotions, as witnessed by the numerous spectacular crashes we’ve seen over the last few decades.

How food prices rise

The reason why these debates matter goes beyond the efficiency of pricing in derivatives markets. It’s how that derivatives pricing can then impact the real price of food, especially in less industrialised countries dependent on food imports. Statistical studies have pointed to a breakdown in the causality between agricultural spot and futures prices, showing futures prices determining spot prices. What might be the mechanisms for this though?

Firstly, benchmarking. There is no straightforward answer to ‘what is the global price of wheat?’ It’s grown in different countries in different grades, and when it comes to a buyer and a seller agreeing on an actual transaction in physical wheat, how do they decide what the price is? It helps if there’s an external and independent benchmark against which they can price. That’s why they can turn to the derivatives price to help them. The futures prices serve as de facto benchmark prices, the only publically visible indication of apparent overall supply and demand. If, for example, import contracts are referenced to the CBOT futures price, that serves as a direct transmission mechanism from futures prices into real food prices.

Secondly, food prices can rise through hoarding. This is the real world arbitrage situation. If futures prices rise, that can incentivise those that have the ability to warehouse food, for example, large commercial grain dealers, to do so whilst selling futures. Locking it up in storage in anticipation for future delivery can create a supply shortage in the present, driving up prices.

Some skeptics point to how foodstuffs with no derivatives markets have also shown price rises, thereby attributing all food price rises to the fundamental issues like the growing demand from east Asia. It’s a fair enough point to make, but it’s also an argument that glosses over the concept of substitution. If the global price of major grains rise, then consumers have incentives to switch to alternatives, which creates convergence in prices.

The concept of substitution raises another contentious issue – the link between energy prices and food prices. The rise in oil price creates incentives to move towards biofuels, which are based on corn and sugar, one of the reasons why oil speculation is seen to have an effect on food prices. Energy also happens to be the biggest input into commercial agriculture, through mechanisation, transportation, and fertilisers created through energy-intensive natural gas methods.

These issues are not likely to be resolved any time soon. In the mean time, food prices are showing dangerous signs of heading to new highs. The talk in City pubs is all about commodities, and when that happens, strange things start to occur. It’s time to get serious about this. You can follow Brett Scott at www.suitpossum.blogspot.com

REVIEW The End of Money and the Future of Civilization Thomas H Greco Jr's new book on money challenges the growth fetish and gives practical examples of how community-based exchange systems can save the economy, and the environment, from collapse