Value for Money in the Rail Industry – where next?

The benefits of the radical reforms to the rail market in the UK are much debated. Whilst the period since privatisation has seen growth in passenger and freight markets, continued improvements to safety, an overall increase in customer satisfaction and improved operations this has been delivered at a significant cost to the public purse.

Both rail infrastructure costs and train operating company costs have risen sharply; the former from roughly £15bn to £30bn when comparing the second 5 year period after privatisation to the first 5 year period. Train operating company unit costs are now nearly 20% higher than at privatisation. This experience contrasts sharply with the experience of other industries which have seen privatisation and the introduction of competition.

In May 2011 Sir Roy McNulty published a report for the Department for Transport on Value for Money in the rail industry. Here I review the contribution of Dr Andrew Smith, Professor Chris Nash and Phillip Wheat to identifying and quantifying the level of inefficiency in the rail industry before looking at the key barriers to efficiency that McNulty identified. Andrew and Chris offer their reflections, one year on, of some of the bigger challenges that remain to be addressed.

One of the key research challenges to be overcome when considering how to establish whether the management of the UK rail infrastructure is efficient is to establish what to benchmark it against. There was only one privatised Railtrack and subsequently one Network Rail. The research work first therefore developed innovative econometric techniques which sought to unpick the different drivers of rail infrastructure costs over time to allow the performance of different global infrastructure managers to be benchmarked. A lack of international benchmarking was an important gap in previous regulatory reviews. This work is particularly complex as infrastructure operators are structured differently and often divided into several business sub-units.

Two separate, new econometric studies were therefore developed. The first used national data over a long-time period to generate estimates of Network Rail’s efficiency performance relative to best practice and also track progress over time. Flexible panel data stochastic frontier techniques were applied; to our knowledge for the first time in a regulatory context. The second quantitative study was applied to a dataset for several countries whilst also extending the dataset by incorporating disaggregate data for different regions within each country. New methods were developed which allowed the persistent element that applies across the company (external inefficiency) and the part that varies at sub-company level (internal inefficiency) to be separated.

The outputs of the first econometric study was used by the Office of the Rail Regulator as the central piece of evidence on Network Rail’s relative efficiency performance in the 2008 Periodic Review of the company’s finances. The second study was used as corroborating evidence. An efficiency gap, estimated at 37%, resulting from the econometric model was directly used to derive a funding settlement for Network Rail covering the fourth control period (2009/10 to 2013/14). In line with regulatory best practice, whilst using the ITS study as the core evidence, ORR also commissioned a wide range of engineering, bottom-up studies, that also indicated a substantial efficiency gap.

In addition to the work on rail infrastructure costs, ITS has also done new work to compute and then explain changes in train operating company costs since privatisation and the introduction of competitive tendering in 1997. Our research showed that unit costs are roughly 20% higher than at privatisation, which is a disappointing result compared to the experience of rail tendering elsewhere in the world and indeed the results of tendering in other sectors. This work was quoted in the McNulty review and formed part of the recommendations, namely that operators should at least get costs back to pre-privatisation levels, before then also targeting further cost savings.

Our research showed some of the key drivers of unit cost growth, including a very substantial increase in real wages of rail staff, far above average earnings growth in the wider economy during this period. The work also showed that the franchising authority’s decision to place a large number of distressed TOCs on management contracts for an extended period led to a substantial deterioration in efficiency relative to other TOCs. Specifically, following the shift to management contracts, the affected TOCs were found to be 23% per cent more expensive than industry best practice. Whilst the use of management contracts may have been a useful expedient – the franchising authority faced a situation where half the sector ran into trouble – it was nevertheless a costly decision, particularly given that the contracts persisted for several years (the latter, to allow the franchise boundaries to be re-drawn prior to the next round of re-franchising).

The McNulty report draws on the evidence from the research described above and presents an analysis of the main barriers to efficiency. These are many, complex and interrelated:

“fragmentation of structures and interfaces, the ways in which the roles of Government and industry have evolved, ineffective and misaligned incentives, a franchising system that does not encourage cost reduction sufficiently, management approaches that fall short of best-practice in a number of areas that are key cost drivers, and a railway culture which is not conducive to the partnership and continuous improvement approaches required for effective cost reduction.” (p5, McNulty Review)

I asked Andrew Smith and Chris Nash for their take on some key issues that remain to be addressed one-year on from McNulty. Andrew identified two key points which suggest that there is still a need to understand the causes and not just the differences, although work remains to be done here too:

“Firstly, whilst there is or was a substantial gap between Network Rail’s efficiency and European operators, continuing to collect and share good quality, comparable data remains a challenge, as does modelling heterogeneity between railway systems. It does also need to be remembered, however, that we are comparing Network Rail against state owned infrastructure managers, so the efficiency potential as compared to private firms could be even greater, given the general evidence on the impact of privatisation in other industries across the world. The splitting up of Network Rail into ten routes now offers the prospect of domestic, comparative competition, as used in other regulated sectors, which will at least help target internal inefficiency.

On the train operations side, in contrast to rail infrastructure, McNulty relied entirely on top-down trends-based evidence to set the efficiency challenge. There was therefore no absolute efficiency comparison between British TOCs and their global comparators (more precisely, some was carried out, but it was inconclusive). So, the argument was as follows: British Rail was inefficient, so unit costs should have come down but did not, and should therefore fall, at least to pre-privatisation levels. Costs should then reduce further, in line with the experience of competitive tendering in rail elsewhere in Europe and the wider evidence on the impact of competitive tendering in other sectors. There was also little or no bottom-up evidence that might explain the gap in terms of the working practices that need to be adopted to get costs down. McNulty rather relied on faith in harnessing the incentives of the private sector, combined with competition for the market, through the use of longer franchises, combined with less tightly defined franchises. This may be the right answer, but it is far from proven.”

Chris Nash reflected on the on-going misalignment of incentives:

“McNulty quotes the work of one of our PhD students, Rico Merkert, as showing that the additional transactions costs of vertical separation of infrastructure from operations are not great as a proportion of total systems costs, but the bigger issue he raises is misalignment of incentives. Despite having sophisticated track access charges, to reflect the damage done by different types of rolling stock, and penalties for poor performance, train operators and infrastructure managers still lack sufficiently strong incentives to work together to reduce costs. A number of new forms of alliance are being tried, but achieving appropriate incentives whilst not disadvantaging other operators remains a challenge.”

For me, this research is an excellent example of the best type of impact that can be delivered when academics work closely with industry, government and regulators to bring independent analysis to a problem. It is interesting to note that the Department for Transport has yet to issue its response to the McNulty Review. This serves as a reminder of the challenge of keeping important agendas under the spotlight and the rather more difficult task of changing policies and practices in a highly complex governance environment. These findings also cast a shadow on the presumption that greater private sector involvement in managing our strategic road network will necessarily be a good thing, as the Prime Minister suggests. Rail is a different sector for sure, but the scale of the complexities and unintended consequences of privatisation suggest that any propositions which emerge from the current Treasury review need full and honest scrutiny.