Summary of the impact

Methods developed at the University of York for measuring NHS productivity have changed how the Office of National Statistics values the NHS in the national accounts. Our methods, which take into account improvements in the quality of care, have been incorporated into submissions to the Comprehensive Spending Reviews that determine the NHS budget and are internationally influential. Research on productivity at hospital level has influenced the tariffs set by the Department of Health for reimbursement of specialist hospital care. Research on the productivity of hospital consultants influenced the reviews of doctors' pay and rewards by the Doctors' and Dentists' Pay Review Body and the National Audit Office and formed the basis of benchmarking tools distributed for use in the NHS.

Underpinning research

Productivity is a key indicator of efficiency and competitiveness. Measures of public service output and productivity are important elements of public accountability for how the £106 billion annual NHS budget is spent. York researchers have conducted a programme of research to improve the measurement of NHS productivity at macro, meso and micro levels. Methods developed at York have been used to seek explanations for variations in productivity.

National NHS productivity

In 2005, a York team (Street, Castelli, Gravelle, Dawson) developed novel and substantially improved methods to measure the productivity of the NHS in England (1). This was followed by an ongoing research programme which has continuously refined and improved the methods over the subsequent 8 years.

The productivity estimates that researchers at York developed were derived from detailed secondary analysis of routinely collected NHS data. They represented an advance on standard practice in other sectors and internationally in three important ways (1). Firstly, the index of outputs was comprehensive, capturing activities in 5,381 healthcare categories for all NHS patients treated by either NHS or non-NHS providers, incorporating information about every patient treated in hospital including outpatient and accident and emergency departments, mental health and community care settings, diagnostic facilities, and primary care. Second, the quality of output was assessed by including indicators such as health outcomes, patient satisfaction, waiting times and readmissions. Third, better measures of NHS input, particularly labour and capital, were compiled. Annual estimates of NHS input, output and productivity growth are now produced by York researchers for the Department of Health (DoH) (2), incorporating updated methods and new data each year. They include a range of sensitivity analyses to ensure robust estimates. Methods to analyse regional productivity have also been developed, with estimates showing significant variation across the country which indicate potential savings of £3.2bn (3).

Specialist hospital care

Research on specialist hospital care, undertaken by Street and Daidone in 2010 and 2011, applied advanced econometric methods to hospital data for over 26 million patients. They isolated the extra costs associated with the provision of specialist care, accounting for a range of other factors that may also influence hospital costs. Results showed that higher costs were legitimately associated with the provision of specialised care only for a small number of conditions and groups, including cancer, cystic fibrosis and children's care (4).

Hospital consultant productivity

Bloor and Maynard developed and applied methods of measuring the productivity of individual hospital consultants (5). Linking NHS data with information about consultants from the Medical Workforce Census for the first time, datasets of inpatient activity were derived for all consultants in ten specialties in England. Using multi-level modelling, factors were identified that predict consultant productivity. Consultants with a `maximum part-time contract' (permitting substantial private practice) were found to treat more NHS patients on average than their full-time NHS colleagues, as did those with clinical excellence awards (bonus payments) (6). This finding was reinforced in a later study which included exploration of associations between clinical excellence awards and consultant productivity (7). Finally, using interrupted time series, it was shown that reform of the consultant contract in 2003 failed to improve consultant productivity, and indeed in some specialties such as trauma and orthopaedic surgery, the effect was negative (8).

Contributors to the research:

Karen Bloor, (Senior Research Fellow (SRF), Prof, 1991-); Chris Bojke (SRF 2009-); Adriana Castelli (Research Fellow (RF), SRF 2004-); Silvio Daidone (RF, 2010); Diane Dawson (SRF 1997-2005; Nick Freemantle (SRF,1993- 2000); Rosalind Goudie, (RF, 2011); Hugh Gravelle (Prof, 1995-); Mauro Laudicella (RF 2007-10); Alan Maynard (Prof 1971-); Andrew Street (1999-SRF, Prof); Padraic Ward (RF 2008-12).

References to the research

All outputs are in high quality peer-reviewed journals except (5), which is a peer reviewed report. All research funding was competitively awarded or subject to peer review for scientific quality.

Grants:

Dawson et al. 2004-5 Dept of Health. Developing new approaches to measuring NHS outputs and productivity; £169,130

Street A. 2008-10 Measuring productivity in the NHS: updates and methodological improvement. Dept of Health Policy Research Programme; £155,310

Street A, Bojke C, Castelli A. 2011-14 The productivity of the NHS: national, geographical and organisational analyses. Department of Health Central Commissioning Facility; £823,611

Research on specialised costs and some national productivity work was undertaken within a CHE programme grant (2006-11): DoH Policy Research Programme grant; £2,304,205

Bloor K, Maynard A. 2002-2006 Department of Health Policy Research Programme. Measuring productivity of hospital consultants using Hospital Episode Statistics for England; £340,000

Bloor K. 2008-2012 NIHR Career Development Fellowship. Exploring and explaining variations in consultant clinical activity; £407,000

Details of the impact

York research has had an impact on policy and practice in relation to the measurement of overall NHS productivity and the analysis of variations in productivity at hospital and at consultant level.

National NHS productivity

The annual NHS budget is approximately £106 billion, accounting for 8.2% of gross national product. By developing improved methods of measuring NHS productivity and providing methodologically robust evidence on annual NHS productivity growth, York research has played a significant role in the processes by which the size of the NHS budget is determined. The 2002 Spending Review committed the Government to a new Public Service Agreement target of 2% improvements in productivity per year. The DoH turned to York researchers to find a way to incorporate quality of care improvements into the measurement of the output of the NHS. This had not previously been attempted. The methods that York developed then were adopted as national policy and have been used continuously. These methods are incorporated into the Office for National Statistics (ONS) estimates of NHS productivity, which feed into the annual UK National Accounts (source 1). Every year since, the ONS has used the method developed by York as a key element in their triangulation of evidence. The statement "ONS would like to thank Professor Andrew Street, Dr Adriana Castelli and the Centre for Health Economics (CHE) team at the University of York for their work on healthcare quality adjustments" appears in all ONS publications relating to NHS productivity. The ONS states that "Estimates of Health gain, Short-term survival and Waiting times are provided by the Centre for Health Economics (CHE) at the University of York. These are derived from patient-level records and offer detailed information at the level of the individual procedure for Inpatients and Day-cases. These are combined into a single measure of quality following the guidance of CHE et al ...." (source 2). These quality adjustments have added an average of 0.5 percentage points to estimates of annual output growth (source 2) and without York's research, NHS productivity would have been underestimated with negative consequences for the size of future budget settlements for the NHS and hence less investment in NHS services.

Estimates of input, output and productivity growth, along with details of the methodological issues that York research addresses, are produced regularly for the DoH for use in their calculations of NHS productivity. Researchers have produced a tool for DoH analysts to use in order to explore in detail the productivity estimates and the impact of different assumptions on future productivity estimates. The estimates provided by York are vital for negotiations with ministers and with agencies that have the power to influence the size of the NHS budget. The research has been used directly to provide numerical answers and context for Health Select Committees and Public Accounts Committees, and Hansard records show that York is often cited by name (source 3). The National Audit Office report on Agenda for Change raised the issue of how measures of NHS output were quality adjusted and, at the Public Accounts Committee, the DoH responded by referring to the research commissioned from York in order to improve the method of adjustment (source 4). The 2008/9 Public Expenditure Inquiry asked for a progress report on the York work and in answering the question the DoH cited the York study on inputs and also indicated that York would be producing productivity estimates at a Strategic Health Authority level (source 5).

York research has also been influential internationally. Representatives from the Italian, Swedish and Japanese governments visited York in 2011, 2012 and 2013 respectively, to learn more about the methods used to measure productivity in the UK. York researchers are working on measurement of productivity for the Italian Ministry of Health (with colleagues at the University of Rome Tor Vergata), replicating for Italy, the methods used in England. York staff have run invited workshops on methods of productivity measurement to policy makers, national statisticians and academics nationally and internationally.

Specialist hospital care

Research on the costs of specialist care was commissioned from York as part of "a fundamental review of the current methodology" used to calculate specialised service top-ups for the Payment by Results tariff (source 6). Our research resulted in changes in policy in 2011/12, including the introduction of new top-ups for neurosciences and spinal surgery and revision of the level of existing top-ups for children's services and orthopaedics (source 6, paras 11 and 94-101). York's report is included on the DoH website alongside the Payment by Results Guidance documents.

Hospital consultant productivity

Staff costs are the largest single component of NHS expenditure. York research into productivity at consultant level has had significant policy impacts in three ways. Firstly, the method of describing and exploring variation in consultant productivity was adopted as a benchmarking tool by the DoH's Workforce Directorate and the NHS Institute of Innovation and Improvement as part of their `Delivering Quality and Value' programme, which in 2008 distributed comparative data on consultant clinical activity to all hospital Trusts in England (source 7). This document states that their process: "uses the methodology outlined in York University's report: Measuring Productivity of Hospital Consultants using Hospital Episode Statistics in England".

Second, this research influenced the recent review of clinical excellence awards by the Doctors' and Dentists' Review Body (DDRB). In evidence to the Review Body, Bloor and Maynard recommended "introducing an extended career structure for doctors, with earned increments and a senior consultant grade" and this recommendation was adopted by the DDRB in their report (source 8), and a `principal consultant grade' is now under negotiation with the British Medical Association (source 9). Our work was also cited by the DoH in their submission to DDRB: "research by the University of York ... showed that consultants in surgical specialties with local Clinical Excellence Awards were those who had the highest levels of productivity" (source 10). The DDRB review team contacted York for supplementary evidence and further details, and cited York research (references 6 and 7 above) in their final report (source 8).

Finally, the findings of our evaluation of the effect of the consultant contract reform on consultant productivity (reference 8) informed the recent National Audit Office review of the management of NHS consultants: "recent work by York University shows a downwards trend in finished episodes per consultant ... [the researchers] concluded that the contract had no or a negative impact on the declining trend in the ten specialty areas analysed" (source 11).

Sources to corroborate the impact

1. Public service outputs, inputs and productivity: healthcare. Edition 5, March 2011 ONS:
   http://www.ons.gov.uk/ons/rel/psa/public-service-productivity/healthcare-2011/public-service-output--input-and-productivity.pdf
2. Sources and methods: public service productivity estimates:healthcare. December 2012 ONS:
   http://www.ons.gov.uk/ons/dcp171766_289768.pdf
3. See, for example, Hansard HC Deb 17 May Col WA307; Hansard HL Deb 24 March 11, col WA254; House of Commons Public Accounts Committee 29^th Report of session 2008-09 (HC 310, 18 June 2009); House of Commons Public Accounts Committee 26^th Report of session 2010-11 (HC 512, 9 Mar 2011); House of Commons Health Committee Second Report of Session 2011-11 Volume 1 (HC 741, 14 Dec 2010).
4. House of Commons. Public Accounts Committee NHS Pay Modernisation in England: Agenda for Change Twenty-ninth Report of Session 2008-09 Report, together with formal minutes, oral and written evidence 18 June 2009
   http://www.publications.parliament.uk/pa/cm200809/cmselect/cmpubacc/310/310.pdf
5. House of Commons Public Accounts Committee. NHS Pay Modernisation in England. Further supplementary memorandum from the Chief Executive, NHS DoH; April 2009.
   http://www.publications.parliament.uk/pa/cm200809/cmselect/cmpubacc/310/09030209.htm
6. Department of Health. Payment by Results Guidance for 2011/12. Feb 2011.
   http://webarchive.nationalarchives.gov.uk/20130507170152/https://www.gov.uk/government/up loads/system/uploads/attachment_data/file/151911/dh_126157.pdf.pdf And DOH note on CHE research: http://webarchive.nationalarchives.gov.uk/20130507170152/https://www. gov.uk/government/uploads/system/uploads/attachment_data/file/151916/dh_124454.pdf.pdf
7. Department of Health (2008) Delivering Quality and Value: Consultant Clinical Activity http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/prod_consu m_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_082902.pdf
8. Evidence to review body (see "evidence submitted by individuals" http://tinyurl.com/evidence-individuals): http://www.ome.uk.com/DDRB_CEA_review.aspx. Review body report: Review Body on Doctors' and Dentists' Remuneration. Review of compensation levels, incentives and the Clinical Excellence and Distinction Award schemes for NHS consultants. Cm 8518; London: The Stationery Office; December 2012.
9. British Medical Association. BMA engages members on consultant contract proposals (press release): http://tinyurl.com/bma-news-consultant
10. Department of Health. UK wide review of compensation levels and incentives for NHS consultants: Evidence to the Review Body on Doctors' and Dentists' Remuneration by the Department of Health:
    http://webarchive.nationalarchives.gov.uk/20130106083031/http://www.ome.uk.com/Document/ Default.aspx?DocumentUid=7A282E6C-6C9B-489C-9366-6E8DDD49FFCC
11. National Audit Office. Managing NHS Hospital Consultants. Report by the Comptroller and Auditor General; HC 885, session 2012-2013; 6 February 2013. http://www.nao.org.uk/wp-content/uploads/2013/03/Hospital-consultants-full-report.pdf