Summary of the impact

Research conducted by the National Primary Care Research and Development Centre (NPCRDC) at the University of Manchester (UoM) has shaped the design of pay-for-performance schemes in primary health care in the UK and overseas. Specifically, the NPCRDC developed methodologies to: 1) design and test new indicators of care quality; 2) revise and retire existing indicators; 3) structure the financial incentives awarded for indicator achievement to maximise quality improvement and minimise harm. These methodologies have been implemented by the National Institute for Health and Care Excellence (NICE) in the UK, where they have generated improved clinical care and a reduction in inequalities in the quality of care for people with common chronic conditions (e.g. asthma, cardiovascular disease, diabetes) managed in general practice. Findings from the research have been disseminated internationally, where they have influenced pay-for-performance schemes in Germany and the United States of America.

Underpinning research

See numbered references in section 3.

The impact is based on research that took place at the UoM from 1995 to date, with the first major publication in 1998. The key researchers are:

• David Reeves (Reader in Statistics, 2001-date)
• Stephen Campbell (Professor of Primary Care Research, 1993-date)
• Tim Doran (Reader in Health Inequalities, 2004-2013)
• Evan Kontopantelis (Senior Research Fellow in Statistics, 2005-date)
• Helen Lester (Professor of Primary Care, 2006 -2011)
• Martin Marshall (Professor of General Practice, 2000-2006)
• Martin Roland (Professor of General Practice, 1992-2009)
• Matt Sutton (Professor of Health Economics, 2008-date)
• Jose Valderas (Clinical Lecturer in Primary Care, 2007-2010)

The research programme initiated in 1995 has (1) developed measures of quality of care in general practice (2) described and explained variations in quality and (3) developed and tested interventions to improve the quality of care. The programme is underpinned by a conceptual definition of quality of care which showed that quality measurement must address clinical excellence (e.g. adherence to clinical care standards) as well as patient experience, underpinned by sound organisational systems and processes for driving quality improvement (1).

Findings from the research informed the design of the Quality and Outcomes Framework (QOF) for primary care introduced in 2004 in which UK general practices are paid according to their performance against a range of quality of care indicators. NPCRDC evaluated the impact of QOF on the clinical quality of care and patient experience in general practice, showing that pay-for-performance improved clinical quality and reduced inequalities in care but also adversely affected some aspects of patient experience. The research was published in the highest impact journals in the field including the New England Journal of Medicine, Annals of Family Medicine, British Medical Journal, Health Affairs, Journal of the American Medical Association and the Lancet.

This case study describes the impact of research designed to maximise improvements in clinical quality while minimising the adverse consequences of pay-for-performance schemes in general practice. Findings from the research shaped subsequent refinements to the pay-for-performance scheme in UK general practice from 2008 onwards. Specifically NPCRDC developed innovative methodologies to:

1) Design and test new quality indicators
NPCRDC developed a protocol to test and validate new indicators of care quality in general practice including identifying unintended consequences of their implementation such as neglect of patients with conditions, or aspects of care, not included in pay-for-performance schemes (2,3).

2) Revise and retire existing indicators
NPCRDC showed that removing quality indicators from a pay-for-performance scheme can lead to subsequent declines in the quality of care addressed by that indicator (4). The NPCRDC also developed a framework for assessing when to remove/retire a quality indicator from pay-for-performance schemes so as to minimise any decrements in quality (5).

3) Structure pay-for-performance to maximise quality and minimise harm
NPCRDC showed that, at relatively little financial cost, exception reporting (excluding patients for whom quality targets are deemed inappropriate or who actively decline intervention) provides some protection from inappropriate and coercive treatment for patients whose providers are subject to pay-for-performance schemes (6).

The research showed that physician incentives provide short-term gains and are not a magic bullet for quality improvement, nor are they entirely responsive to the complex needs of individual patients. Rather, it demonstrated the need for multilevel approaches to change as part of a wider strategy for quality improvement.

References to the research

Details of the impact

See numbered corroborating sources in section 5.

NPCRDC's research has had a substantial and ongoing impact on the pay-for-performance scheme in UK general practice, known as the QOF. This scheme covers all general practices in the UK, shaping the care they provide to all patients with one of the common chronic illnesses covered by the QOF (e.g. asthma, cardiovascular disease, diabetes).

1) Design and test new quality indicators
From 2009 NICE formally adopted the protocol developed by the NPCRDC to test and validate new indicators for inclusion in the national UK Quality and Outcomes Framework (S1). Piloting new indicators before roll out nationally has proved value for money as it identifies potential problems with reliability, feasibility, acceptability and unintended consequences, and can also identify indicators that should not be included because they may cause harm to patients. The cost of piloting a new indicator is £150,000 which is only 0.0005% of the overall cost of £1billion, which the government spends on the QOF each year.

2) Revise and retire existing indicators
From 2010 the framework developed by NPCRDC for revising and removing indicators from pay-for-performance schemes was adopted by NICE and NHS Employers to inform their annual reviews of the QOF. For example, our methodology led to the retirement of 12 quality indicators in April 2011 and 7 quality indicators in April 2012 (S1).

3) Structure pay-for-performance schemes to maximise quality and minimise harm
NPCRDC research into exception reporting (i.e. allowing clinicians to exempt patients from quality indicator measurement where they believe the indicator is not appropriate for the individual patient) has demonstrated that it protects patients from inappropriate care without triggering widespread fraudulent behaviour by providers. This research provided the evidence-base supporting the case for retaining exception reporting within the QOF (S1) in the face of opposition to the provision, and has informed the international debate about the need for an exception reporting provision in physician incentive schemes (S2).

Methodologies developed by NPCRDC for designing and testing new quality indicators have influenced policy and practice in the USA, Germany and other countries. In the USA, our methodological work comparing different approaches for aggregating indicators into composite measures was recommended by the American Medical Association Physician Consortium for Performance Improvement (S3) and the USA Quality Forum (S4). In Germany, our methodologies for designing and testing new quality indicators have been adopted by the Institute for Applied Quality Improvement and Research in Health Care (AQUA-Institute), which has been commissioned by the German Federal Joint Committee to develop sets of quality indicators and instruments across both inpatient and outpatient healthcare sectors (S5).

The need we demonstrated for multilevel approaches to change as part of a wider strategy for quality improvement was cited in influential policy reviews conducted by the World Health Organisation (S6), the OECD (S7), and the Commonwealth Fund (S8).

Sources to corroborate the impact

S1. Letter from NICE (G Leng) attesting to NPCRDC role and influence in QOF indicator piloting and removal, and debates about exception reporting.

S2. Van Herck P, Annemans L, De Smedt D, Remmen R, Sermeus W. Pay-for-performance step-by-step: introduction to the MIMIQ model. Health Policy. 2011;102(1):8-17. DOI: 10.1016/j.healthpol.2010.09.014

S3. Physician Consortium for Performance Improvement. Measures Development, Methodology, and Oversight Advisory Committee: Recommendations to PCPI Work Groups on Composite Measures, American Medical Association, 2010

S4. National Quality Forum, Composite Performance Measure Evaluation Guidance, National Quality Forum, April 8 2013.

S5. Letter from AQUA Institute in Göttingen (J Szecenyi) which used a modified version of the indicator development process as part of the Federal Government work.

S6. Elovainio E. (2010) Performance incentives for health in high-income countries: key issues and lessons learned, World health report 2010. Background Paper 32, World Health Organisation, 2010.
http://www.who.int/healthsystems/topics/financing/healthreport/32PBF.pdf

S7. Cashin C et al. (2011) Major Developments in Results-Based Financing in OECD Countries: Country Summaries and Mapping of RBF Programs. OECD, March 29th 2011.

S8. Squires D, Incentivizing Quality Care Through Pay-for-Performance, The Commonwealth Fund, 2012