Summary of the impact

Reductions in railway infrastructure and operating costs, through efficiency gains, deliver benefits to taxpayers (via lower subsidies) and/or passengers (via lower fares). Research undertaken by the Institute for Transport Studies (ITS) at the University of Leeds from 2005 onwards revealed a 37% efficiency gap in relation to rail infrastructure costs and operations, relative to international best practice. The key impact of this research was to inform the Office of Rail Regulation's (ORR) setting, in 2008, of annual efficiency targets for Network Rail for the subsequent five-year period, resulting in a reduction in costs from £18.2bn to £15.8bn over the five year regulatory `control period' starting 2009/10. A secondary impact of the ITS Leeds research was to provide key benchmarking and evidence in more recent ORR efficiency assessments (2010) and Sir Roy McNulty's long-term policy-setting Rail Value for Money (VfM) study (2011). Extending the reach of these research impacts, the water and sewerage regulator OFWAT has, from 2013, adopted the ITS Leeds approach for its latest periodic review.

Underpinning research

A key responsibility of the Office of Rail Regulation (ORR) is to determine the efficient cost of operating, maintaining, renewing and developing the British rail infrastructure. The determination of these costs, referred to as the periodic review, occurs every five years and, given the structure of the rail industry in Britain, effectively amounts to a review of the costs incurred by Network Rail. In contrast to other regulated utilities, previous periodic reviews (prior to 2008) did not assess the efficiency of Network Rail against external benchmarks. Recognising this gap in knowledge, ITS Leeds was contracted in 2005 by ORR to develop two international benchmarking studies.

Econometric modelling — efficiency studies

Between 2005 and 2008, researchers from ITS Leeds carried out new research funded by the ORR to investigate the cost and efficiency of the rail system in Britain relative to its international peers. Employing advanced time-varying efficiency modelling techniques, Dr Andrew Smith identified drivers of rail infrastructure costs, and the relative efficiency of rail infrastructure managers (IMs) around the world [1; iii] using a unique panel dataset of 13 European rail infrastructure managers collected by the International Union of Railways. This work exposed a substantial efficiency gap between Network Rail and its international peers of around 37%.

As part of this work, Smith and his colleague Phill Wheat also adapted stochastic frontier techniques to deal with `sub-company' datasets (where firms are sub-divided into several business units). This new approach allowed inefficiency to be decomposed into a fixed element that applies across the entire company (external inefficiency) and an element that varies at the sub-company level (internal inefficiency). The `sub-company' analysis revealed large variations in performance and practice between regional units within Network Rail [2; iii], as well as a substantial efficiency gap between Network Rail and its international peers.

In parallel to the above research, Wheat and Smith developed a new international collaboration with Professor William Greene at NYU Stern School of Business to develop statistical methods for assessing the uncertainties associated with the estimates of inefficiency generated by stochastic frontier methods [3; i, iii].

Econometric modelling — evaluation of policy reforms

The research referred to above is part of a larger body of rail cost research on whole train systems (i.e. train operations as well as rail infrastructure) conducted by ITS Leeds over many years, supported by funding from EPSRC [i], DfT [ii] and ORR [iii]. Since 1993, ITS researchers have carried out econometric and wider policy evaluation work to assess the impact of competitive tendering/franchising on costs. Introducing competition into passenger rail, via competitive tendering, forms an important part of the European Commission's proposals for further reform of the European rail sector. This strand of research was the first in the literature to demonstrate the sharp rise in train operating company costs in Britain (previously the policy focus had been on rising infrastructure costs); see for example [4]. Reference [5] found that a key reason behind rising train operating company costs in Britain was the franchising body's decision to place operators in financial distress on "cost-plus" type management contracts for extended periods.

Research team with current title/grade and period of employment

Professor Chris Nash (Lecturer, 1975-88; Senior Lecturer, 1988-89; Professor, 1989-date) Dr Andrew Smith (Lecturer, 2004-11; Senior Lecturer, 2011-date) Phill Wheat (Research Officer, 2005-08; Research Fellow, 2008-10; Senior Research Fellow, 2010-date)

Key grants 1998-2013

i. Rail Research UK, several grants (e.g. RG.TRAN.471326, PI Nash, RG.TRAN.481002, PI Smith), worth approximately £200k in total.

ii. Department for Transport (RG.TRAN.477546, PI Smith), £38k.

iii. ORR funding, numerous grants (e.g. RG.TRAN.471451, RG.TRAN.473416, RG.TRAN.475450, RG.TRAN.477940, RG.TRAN.478240, RG.TRAN.478541, RG.TRAN.482896, RG.TRAN.474920), worth approximately £250k in total. PI Smith.

iv. OFWAT funding, three grants (e.g. RG.TRAN.484975, RG.TRAN.101259. PI Smith, worth approximately £49k in total.

Note: The repeat funding from Rail Research UK and ORR is indicative of the quality of the Leeds research and its ability to meet project objectives, as is the new work for OFWAT following on from the successful research carried out for ORR. It should be noted that the ORR contracts were awarded following a competitive tendering process.

References to the research

The above output is based on an earlier project report produced for ORR and published on their website: Smith, A.S.J (2008) International Benchmarking of Network Rail's Maintenance and Renewal Costs: An Econometric Study Based on the LICB Dataset (1996-2006), Report for the Office of Rail Regulation, October 2008. http://www.rail-reg.gov.uk/upload/pdf/pr08-itslicb-301008_20081117141529.pdf

The above output is based on an earlier project presentation produced for ORR (co-authored with ORR) and published on their website: Smith, Wheat, and Nixon (2008) International Benchmarking of Network Rail's Maintenance and Renewals Costs. http://www.rail-reg.gov.uk/upload/pdf/pr08-its_010608.pdf

The McNulty report (see Section 4 below) referenced (in footnote 106, p284) a project report produced for ORR based on the above output, as well as an earlier conference paper version of output 4: Smith, A., Nash, C. and Wheat, P. (2009) `The effect of passenger rail franchising on costs: evidence from the passenger rail sector in Britain (1996 to 2009)'.

The McNulty report (see Section 4 below) referenced (in footnote 25, page 34 and footnote 108, p285) an earlier conference paper version of output [5]: Smith, A.S.J. and Wheat, P.E. (2009) `The effect of franchising on cost efficiency: evidence from the passenger rail sector in Britain'.

Note: All ITS Leeds researchers are in bold. References [1], [3] and [5] should be assessed for research quality. All references are published in international journals with rigorous peer review processes.

Details of the impact

An undesirable consequence of rail reform and privatisation in Britain has been sharp increases in both rail infrastructure costs (from roughly £15bn during the first five years after privatisation to £30bn during the second five year period) and train operating company unit costs (up by around 15% since privatisation). The ITS Leeds research commissioned by ORR has enabled the regulator to benchmark the efficiency of Network Rail, by providing the key evidence for the regulator to set efficiency targets. The efficiency improvements delivered by Network Rail have directly benefited taxpayers via lower subsidies (although these benefits could instead be directed to passengers through lower fares).

Economic impacts — improving the efficiency of rail infrastructure provision

The first of these studies [1] was used by ORR as the central piece of evidence on Network Rail's efficiency performance in the 2008 Periodic Review [A]. The efficiency gap, estimated at 37% by the ITS Leeds modelling, was used directly to derive a funding settlement for Network Rail covering the fourth `control period' (CP4; 2009/10 to 2013/14). A quote from the ORR's final conclusions document emphasises how the model was used: "...we [ORR] consider that it is reasonable, transparent and prudent to adopt the 37% value from the preferred model as our estimate for the efficiency gap for each of maintenance and renewals in 2006-07." [A, p157]. Corroborating the role played by ITS Leeds, ORR [A, p126] noted: "...the econometric analysis work was undertaken by ITS working closely with both Network Rail and ourselves".

The value of the efficiency targets set by ORR, based on the ITS Leeds evidence, was to reduce the cost of maintaining and renewing Network Rail's infrastructure from around £18.2bn to £15.8bn over the five year regulatory control period (2009/10 to 2013/14). Note that this reduction is smaller than 37% because ORR phased the target gradually over a ten year period. A letter of corroboration from ORR [B] notes: "Following the PR08 process, assisted by the ITS econometric efficiency model, ORR deemed that the efficient cost should be only £15.8bn (in 2006-7 prices). This work thus directly influenced the determination by the ORR of Network Rail's funding for CP4, much of which is provided by taxpayers".

The second study commissioned by ORR [2] was also an international econometric modelling exercise, but this time expanding the sample size by utilising sub-company (regional) data within individual railways (drawn from a sample of participating European and North American railways). The model from this second exercise was quoted in ORR's 2008 determination as a supporting, corroborating study to the first study outlined above [A, p125].

Setting standards for rail regulatory reviews

In its review of ORR's regulatory efficiency analysis, the National Audit Office in 2011 noted that: "...the Regulator initiated an innovative international benchmarking exercise with the Institute of Transport Studies (University of Leeds) during its 2008 Periodic Review" [C, p31].

Since the 2008 Periodic Review, the approach was further developed and used in subsequent assessments; ITS Leeds modelling supported the ORR's Annual Efficiency Update process in 2010 [D]. Furthermore, Smith and Wheat were seconded part-time to ORR during 2012 and 2013 to advise the regulator on its benchmarking methods for the 2013 Periodic Review; their contribution is acknowledged in the 2013 Draft Determination [E].

Informing policy — the long-term future of Britain's rail system

The modelling work [1, 2] also formed the basis of future efficiency targets for Network Rail (CP5; 2014/15 onwards) documented in the Sir Roy McNulty Rail Value for Money study (which commenced in 2010 and was published in 2011) [F]. This is arguably the most significant report on rail costs in Britain, which has assessed efficiency and value for money since privatisation and takes a long-term perspective on its future.

A letter of corroboration from DfT [G] notes: "The [McNulty] study represented a major review of rail costs, efficiency and value for money, and set out a new policy direction based on its analysis and findings. The study identified savings for the whole industry of between £2.5bn and £3.5bn per year (or 20-30% of total system costs), of which the savings identified for Network Rail from the work conducted by ITS accounted for £1.8bn per year (£1.2bn per year earmarked to be achieved by the end of CP4 and a further £0.6bn per year by the end of CP5)". Through implementing the recommendations of this study (and because of growing passenger revenues), the level of subsidy paid to the railway is forecast to fall from £4bn in 2010-11 to around £1bn by 2019.

In addition, ITS Leeds research (a project report for ORR based on [4] and an earlier conference paper version of [5]) is also quoted by McNulty [F] as a key piece of evidence, amongst others, in determining future targets for train operating costs.

Changing practice in other regulated sectors

Extending the reach of the research impacts to other sectors, ITS Leeds was commissioned (with consultants CEPA) by the water regulator OFWAT in summer 2012, to develop new econometric models and advise on how they could be used to set a `corridor' for allowed cost levels, whilst complying with the OFWAT's aim to achieve a "light touch" approach to regulation. The preliminary report [H] released in 2013 sets out the approach developed by ITS Leeds (with CEPA). This work has fed directly into OFWAT's 2014 regulatory review (PR14), as OFWAT has now decided to adopt the ITS Leeds approach to econometric modelling for PR14. Furthermore, Smith has been appointed (from August 2013) as the regulator's academic adviser on cost assessment.

Sources to corroborate the impact

A. Office of Rail Regulation (2008) Determination of Network Rail's Outputs and Funding for 2009- 14 (October 2008). http://www.rail-reg.gov.uk/upload/pdf/383.pdf

B. Letter of corroboration from the Deputy Director, Railway Markets and Economics, ORR.

C. National Audit Office (2011) Regulating Network Rail's Efficiency.
http://www.nao.org.uk/report/regulating-network-rails-efficiency/

D. Office of Rail Regulation (2010) International cost efficiency benchmarking of Network Rail. The main report (see p5) and supporting paper (the latter, co-authored with ITS) can be found at, respectively:
http://www.rail-reg.gov.uk/upload/pdf/econometric_update_2010_orr_benchmarking_report.pdf
http://www.rail-reg.gov.uk/upload/pdf/econometric_update_2010_its_paper.pdf

E. Office of Rail Regulation (2013) Draft determination of Network Rail's outputs and funding for 2014-19 (see p300). http://www.rail-reg.gov.uk/pr13/PDF/pr13-draft-determination.pdf

F. Sir Roy McNulty report (2011) Realising the Potential of GB Rail: Final Independent Report of the Rail Value for Money Study: Detailed report (see, for example, page 30)..
http://www.rail-reg.gov.uk/upload/pdf/rail-vfm-detailed-report-may11.pdf

G. Letter of corroboration from the Deputy Director, Rail Analysis, Department for Transport.

H. OFWAT reference.
https://www.ofwat.gov.uk/pricereview/pr14/wholesale/rpt_com201301cepacostassess