Summary of the impact

Research carried out by LSHTM to develop and test patient reported outcome measures (PROMs), which measure health outcomes from the patient perspective, has demonstrated the feasibility of routinely collecting such measures before and after elective operations. In 2009, the Labour government mandated that PROMs should be collected on all NHS patients in England undergoing one of four surgical operations, a policy endorsed by the coalition following the 2010 election. This remains the only nationwide programme of its kind worldwide, providing essential data for comparing providers' performance, patient choice and other quality improvement approaches.

Underpinning research

Patients' views are essential to achieving high quality care. Their perspective complements that of clinicians, providing unique insights into their own perceptions of health status and health-related quality of life. It is therefore important to find ways of involving patients in reporting on their own health outcomes.

The goal of research carried out by LSHTM since 1996 has been to create PROMs — measures of patients' health and health-related quality of life collected before and after surgery. Involving patient interviews and the development and testing of questionnaires, the research has been led by Nick Black, Professor of Health Services Research at LSHTM since 1995.

The research fell into three phases: the development and psychometric testing of PROMs (1996- 2004); methodological research to ensure accurate analysis and interpretation of these measures (1993-2012); and applied research into the feasibility of the routine use of PROMs to assess the quality of care of providers (since 2005). While many universities around the world have contributed to the first of these areas and some have focused on the second, Black and colleagues have been unique in researching the third.

In terms of developing and psychometrically testing patient questionnaires, under the leadership of Donna Lamping (Professor of Health Psychology, LSHTM from 1992, until her death in 2011, initially Senior Lecturer), major contributions have been made to the development of new PROMs for a wide range of surgical procedures: stress incontinence (1996); benign prostatic hypertrophy (1998); menorrhagia (1998);^3.1 venous disorders (2003); plastic surgery on hands/arms and on head /neck procedures (2004); and coronary revascularisation (2004).^3.2 Rigorous qualitative and quantitative methods were used by LSHTM experts in sociology, psychology, epidemiology and statistics.

Work in the second area was undertaken on a range of methodological aspects of the use of PROMs, including: the influence of patients' preoperative expectations; the impact of late response and non-response to follow-up; ^3.3 the use of minimally important differences;^3.4 and many others.

Finally, Black, Jan van der Meulen (Professor of Clinical Epidemiology, LSHTM since 2000, then Senior Lecturer) and other colleagues have made a unique contribution to the routine implementation of PROMs in the NHS. In 2005 they undertook a systematic review of instruments for the routine assessment of outcomes following five common elective operations (hip and knee replacement, varicose vein surgery, hernia repair and cataract surgery). This led directly to a study to develop pre- and postoperative questionnaires for four procedures and to test the feasibility of using them routinely in the NHS. Findings reported in 2007 confirmed that it was possible to recruit patients, follow them up and make risk-adjusted comparisons of providers, all at reasonable cost. To confirm these findings on a larger scale, in 2008 Black established a much larger study with about 35,000 patients. In parallel, since 2009, studies have included equity of use and outcomes, the impact of choice of metric^3.5 and clinicians' and patients' views of how best to feed back comparisons of health care providers' performance to maximise the likelihood that the data will stimulate improvements in the quality of care.^3.6

References to the research

Development of PROMs

Methodological research

Applied research

Key grants

Lamping, Development and Validation of Disease-specific Patient-based Measures of Outcome in Plastic Surgery, British Association of Plastic Surgeons, Stoke Mandeville Burns and Reconstructive Surgery Research Trust, NHSE Anglia & Oxford Regional Office R&D, 1996-1998, £47,890.

Black, Patient Reported Outcome Measures for Evaluating Elective Surgery: Systematic Review of Instruments and Development Study, Department of Health Policy Research Programme, 2004- 2006, £299,000.

Black, Patient Reported Outcome Measures: Methodological Development for National PROMs Programme, Department of Health, 2008-2012, £817,000.

Details of the impact

The research into PROMs carried out at LSHTM has led directly to their introduction across the NHS in England for four elective surgical procedures. It is the first time that such measures have been introduced on a nationwide scale with the aim of comparing the performance of hospitals.

After Black and colleagues reported their research findings to the Department of Health (DH) in July 2007 and recommended the use of PROMs, the NHS Management Board decided in December 2007 that from April 2009 it would be mandatory for all NHS patients undergoing the designated operations to be invited to complete pre- and postoperative PROMs questionnaires (2009/2010 NHS Operating Framework).^5.1 The Chair of the DH PROMs Stakeholder Reference Group, Dr Andrew Vallance-Owen,^5.2 recognised that `Nick Black and his colleagues were pivotal in giving the DH the confidence to start the National PROMs Programme', and Lord Darzi, then Minister of Health, addressing the Health Select Committee, stated: `I think the best investment we have made was actually in this report ... It is good reading actually. It has come from LSHTM, and was done jointly with the Royal College of Surgeons.'^5.3 In June 2008, High Quality Care for All (a key policy document for the DH under the Labour government) provided a strong endorsement: `Just as important [as clinical measures] is the effectiveness of care from the patient's own perspective which will be measured through patient-reported outcome measures.'^5.4

During 2008-2009, Black and colleagues briefed Conservative shadow ministers several times on the opportunities that PROMs presented. Despite the change of government in 2010, there has been seamless political support for the use of PROMs; in July 2010 the coalition government in its White Paper, Equity and Excellence — Liberating the NHS stated: `Information generated by patients themselves will be critical ... and will include much wider use of effective tools like PROMs.' Ministers' positive views on PROMs were reflected in the NHS Operating Framework 2011/2012 (pp. 25-26)^5.5 and by the inclusion of PROMs in the NHS Outcomes Framework 2011/2012 (p20).^5.6

The value of PROMs for more sophisticated estimates of NHS productivity has been recognised by the Office of National Statistics^5.7 and the National Audit Office.^5.8 The latter stated that, `When producing productivity measures, new data on quality such as PROMs should be considered' (p. 10).

The historical importance for surgeons was recognised in 2012 by Professor Norman Williams, President of the Royal College of Surgeons, when he stated that `the introduction of PROMs has been a major development in the history of surgery'.^5.9

Since April 2010, PROMs data have been published online (http://www.ic.nhs.uk/proms) for the use of clinicians, managers, commissioners and the public. The huge scale of the programme is indicated by the 515,000 patients who participated over the first three years (over 70% of those eligible). In 2012, the DH's Branch Head of Strategy, Finance and NHS Directorate, David Nuttall, commended Black and colleagues' continued research after the introduction of PROMs as having `informed decisions on how data should best be analysed, presented and used'.^5.10

Another indication of success is that coronary revascularisation will be included from 2014 using a PROM (CROQ) developed at LSHTM. The University of Oxford has undertaken preliminary work on the use of PROMs in long-term conditions, but the impact described in this case study is a direct result of the work of LSHTM alone.

Black has raised awareness and understanding of PROMs among clinicians, NHS managers and patient organisations by chairing many national conferences and giving 11 keynote talks since 2009, including at the Royal College of Nursing (March 2010); the Pre-Operative Assessment Association (September 2010); the NHS Medical Directors Conference (October 2010); the National PROMs Summit (December 2012, 2013); and the International Society for Quality in Healthcare (October 2013) at which his contribution internationally was recognised by the award of the first Career Achievement Prize in PROMs, funded by the Medical Outcomes Trust in the USA. He is advising the US National Institute of Medicine and the American Medical Association.

The introduction of PROMs was widely reported in the press, raising awareness and understanding among the general public. In a Financial Times report on 19 August 2011, Professor Sir Bruce Keogh, Medical Director of the NHS, was quoted as saying he hoped PROMS `would shift the focus among doctors [away] from technocratic results, where an operation was deemed a success regardless of whether the patient remained in pain'.

Sources to corroborate the impact

5.1 Department of Health (2008) The Operating Framework for the NHS in England 2009/10: High Quality Care for All. London: DH, http://www.connectingforhealth.nhs.uk/systemsandservices/infogov/links/opframework20092010.pdf (accessed 16 October 2013).

5.2 Chair, Department of Health PROMs Stakeholder Review Group.

5.3 Health Select Committee (2008) NHS Next Stage Review (HC 937-ii), 17 July, uncorrected transcript of oral evidence, http://www.publications.parliament.uk/pa/cm200708/cmselect/cmhealth/uc937-ii/uc93702.htm (accessed 16 October 2013) (q.154).

5.4 Darzi, A (2008) High Quality Care for All: NHS Next Stage Review Final Report, Cm 7432. London: The Stationery Office, http://www.official-documents.gov.uk/document/cm74/7432/7432.pdf (accessed 16 October 2013).

5.5 Department of Health (2010) The Operating Framework for the NHS in England 2011/12. London: DH, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/216187/dh_122736.pdf (accessed 16 October 2013).

5.6 Department of Health (2010) The NHS Outcomes Framework 2011/12. London: DH, https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/213789/dh_123138.pdf (accessed 16 October 2013).

5.7 Hardie, M, Cheers, J, Pinder, C and Qaiser, U (eds) (2011) Public Service Output, Inputs and Productivity: Healthcare. Newport: ONS, http://www.ons.gov.uk/ons/rel/psa/public-service-productivity/healthcare-2011/public-service-output--input-and-productivity.pdf (accessed 16 October 2013).

5.8 National Audit Office (2010) Department of Health: Management of NHS Hospital Productivity, report by the Comptroller and Auditor General, HC 491, Session 2010-2011. London: The Stationery Office, http://www.nao.org.uk/wp-content/uploads/2010/12/1011491.pdf (accessed 16 October 2013).

5.9 President, Royal College of Surgeons.

5.10 Deputy Branch Head, Choice and PROMs, Commissioning Development, Department of Health.