Summary of the impact

Patients are more likely to get the most effective healthcare, at affordable cost to the NHS, as a result of research methodology, developed by researchers at the University of Bristol, that allows the efficacy and cost-effectiveness of multiple treatment options to be compared, based on all the available evidence, much more efficiently than in the past. Since 2008, these methods have been used to inform Clinical Guidelines issued by the National Institute for Health and Care Excellence (NICE) and in submissions to NICE's Technology Appraisals. Guidance in NICE's Technology Appraisals is mandatory and therefore impacts directly on clinical practice. The methodology is used in decision making by NICE's equivalents in other countries including Canada, Germany, and South Korea, and by consultancy firms that conduct analyses for pharmaceutical companies.

Underpinning research

The University of Bristol's (UoB) Multi-Parameter Evidence Synthesis (MPES) research group, led by Professor Ades (2002-present), has been funded by Medical Research Council programme grants, fellowships, and research grants from July 2001-present. Other group members contributing to the impact are Drs Lu (2002-12, Senior Research Fellow), Welton (2002-present, Senior Lecturer), Caldwell (2002-present, Research Fellow), and Dias (2007-present, Research Fellow). All research outputs were published in peer reviewed journals.

MPES methods statistically combine evidence from multiple sources. The research underpinning the impact was the development (2002-present) of Mixed Treatment Comparison (MTC) methods (also known as Network Meta-Analysis). Standard meta-analysis combines estimates of the relative effects of two treatments (e.g. drug A versus drug B, or drug C versus placebo) from all available randomized trials. However there are often many competing health technologies for a given condition/patient group (e.g. drugs A, B, C, D and placebo). MTC methods allow information from all the trials of all the treatments of interest to be compared simultaneously (A versus B versus C versus D, etc). The simplest example of an MTC is an "indirect comparison", where in the absence of trials comparing A versus B, the effect of A versus B is inferred from trials comparing A versus C and trials comparing B versus C.

Because MTC methods allow more evidence to be combined, relative treatment effects can be estimated more precisely than with standard meta-analysis methods. Treatments can also be ranked according to both efficacy and cost-effectiveness, allowing policymakers and guideline development groups, whether in insurance- or state-funded health systems, to make better-informed decisions, supporting equitable and optimum resource allocation by health service purchasers.

The UoB MPES group firstly set out the theoretical framework for MTC methods, showing them to be natural extensions of standard meta-analysis, and developing robust software for estimating them, published in peer review journals 2004-2008[1,2,3,4]. The scope of the methods was further extended and generalised, so that it could be applied to trials reporting a variety of different types of outcomes (probabilities; rates; continuous; ordinal; competing risks)[5]. The group has also developed theory on the robustness and reliability of MTC, and developed methods to check consistency of the different evidence sources[5,6].

These methodology developments were led and conducted by the MPES group, which also developed general computer code (written for the freely available WinBUGS software) to conduct MTC and made this code freely available through its website. These methods are now widely accepted, and were adopted in the 2008 and 2013 updates of the NICE Guide to the Methods of Technology Appraisal. NICE, through its Decision Support Unit, commissioned the MPES group to write a series of Technical Support Documents, including general code for a range of different types of outcomes and evidence structures [5] to guide those making submissions to NICE. These were first published on the NICE Decision Support Unit website in 2011 following peer-review, and have subsequently all been published in the journal Medical Decision Making (July 2013, Vol 33 No.5), following further peer review. All other references [1,2,3,4,5,6] are published in peer-reviewed journals.

References to the research

Details of the impact

The UoB MPES group contributed to the 2008 and 2013 Revisions of the NICE Methods Guide, which define the role for MTC methods in submissions to NICE [a,b]. The Technical Support Documents developed by the MPES group, available on the NICE Decision Support Unit website [5], are cited in the NICE 2012 Guidelines Manual [c,d], which define the methods to be used in NICE Clinical Guidelines. The same methods and WinBUGS code are recommended by the International Society for Pharmacoeconomics and Outcomes Research, the leading society for Health Technology Assessment methods [e].

Impact of MTC methods via NICE Technology Appraisals

Mixed Treatment Comparisons have been presented in submissions underlying 59 (46%)[f] of the 129 NICE Technical Appraisals (TA) since January 2009, covering a wide range of clinical areas[f]. Of these, 24 (41%) directly cite papers by the MPES group and/or clearly use WinBUGS code written by the MPES group [f]. NICE Technology Appraisals determine whether new technologies are cost-effective for the UK National Health Service (NHS), and if they are then they must be adopted by law (i.e. provided to patients by the NHS) within three months of the guidance being issued. NICE Technology Appraisals therefore impact directly on healthcare policy governing which new treatment options are available to health professionals to treat patients. Because NICE guidance is primarily based on cost-effectiveness, it secures more health-related quality of life per pound spent by the NHS.

For example, TA199 [f] cites MTC methods (p.213), the research of the MPES group (references 204, 205, 230), and uses WinBUGS code derived from the MPES groups code (p.224). Based on MTC methods TA199 recommends that Etanercept, infliximab or adalimumab may be used for the treatment of active and progressive psoriatic arthritis, but that the least expensive should be used based on locally available prices. This was based on the view that these products were equally effective, a conclusion that could not be reached without MTC methods.

Impact of MTC methods via NICE Clinical Guidelines

The UoB MPES group has provided consultancy work to assist the incorporation of MTC methods into several NICE Clinical Guidelines (CG) in a range of clinical areas[d]. For example, research by the MPES group is cited in CG153 Psoriasis (Oct 2012). The research methods that were developed during this consultancy work, and which were used in these guidelines, have since been published [d]. Although NICE Clinical Guidelines are not mandatory, there is evidence that uptake is generally good for those guidelines evaluated in the ERNIE Uptake Database, available on the NICE web-pages. NICE Clinical Guidelines are based on cost-effectiveness, and so uptake of the guidelines implies impact in terms of maximising the health-related quality of life obtained from NHS resources.

Impact of MTC methods in Canada

The Canadian Agency for Drugs and Technologies in Health (CADTH) has used MTC methods in its reports and recommendations on second and third line therapies for Type 2 Diabetes inadequately controlled on metformin, and for their Therapeutic Review on Biologics in Rheumatoid Arthritis [g]. The various biologic drugs have not been compared directly in head-to-head in trials, and so MTC methods are essential to compare their effectiveness and cost-effectiveness. As in the UK, the impact was to make the biologic drugs available in Canada as the most cost-effective use of resources.

Impact of MTC methods in other countries

MTC methods are being used in other countries. In Germany the Institut fur Qualitat und Wirtschaftlichkeit in Gesundheitswesen (IQWiG) approved the use of MTC methods in 2009, and used them in the Final Report on treatments for essential hypertension (A05-09). The National Evidence-based healthcare Collaborating Agency (NECA) in South Korea has used MTC methods since 2009 [h].

Commercial Impact

The use of MTC by pharmaceutical firms in 46% of all submissions to NICE represents a substantial commercial activity in itself, with several consultancy firms now specialising in conducting MTC analyses on behalf of pharmaceutical company clients, and marketing this specialism. These consultancy firms send staff on courses taught by the MPES group, and use WinBUGS code developed by the group, with resulting impact in terms of commercial economic activity [i, j].

Sources to corroborate the impact

[a] Letter from NICE, Centre of Technology Evaluation: "[a] CTE_NICE_Letter.doc" Letter from the Director of Health Technology Evaluation at NICE confirming the role of the MPES group in the development of the section on "indirect comparisons and network meta-analyses" of the NICE Guide to the Methods of Technology Appraisals (2013).

[b] Guide to the methods of technology appraisal 2013 Working Party
http://publications.nice.org.uk/guide-to-the-methods-of-technology-appraisal-2013-pmg9/further-information#nice-methodology-working-party (accessed 3^rd Oct 2013)

Prof. Ades and Dr. Dias are listed as special advisors to the working group for the development of the NICE Guide to the Methods of Technology Appraisals (2013).

[c] Letter from NICE, Centre for Clinical Practice: "[c] CCP_NICE_Letter.pdf"

Letter from NICE Centre for Clinical Practice, confirming the role of the research and staff of the MPES group in supporting the development of NICE clinical guidelines, and the citation to the research of the MPES group in the NICE 2012 Guidelines Manual (2012).

[d] List of NICE Clinical Guidelines where the MPES groups research was involved: "[d] NICE CGs using MTC.doc"

In 6 published and 1 forthcoming NICE Clinical Guidelines, the MPES group were directly involved advising on and/or conducting MTC analyses, and in the development of new methods specifically for the guideline (subsequently published in peer-reviewed journals). The research of the MPES is cited in the NICE Guidelines Manual (2012).

[e] Hoaglin DC, Hawkins N, Jansen JP, Scott DA, Itzler R, Cappelleri JC, Boersma C, Thompson D, Larholt KM, Diaz M, Barrett A. Conducting Indirect-Treatment-Comparison and Network-Meta-Analysis Studies: Report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: Part 2. Value in Health 2011; 14:429-437. "[e] Hoaglin ViH 2011.pdf"

The methods and code of the MPES group are recommended by the International Society for Pharmacoeconomics and Outcomes Research, the leading society for Health Technology Assessment methods.

[f] Details of the 59 NICE Technology Appraisals (TAs) since 2009 that have used Indirect or Mixed Treatment Comparisons: "[f] NICE TAs using MTC.doc"

The methods of the MPES group are used in 46% of NICE Technology Appraisals since 2009, and the MPES group research is directly cited in 41% of those. Decisions made in NICE Technology Appraisals are mandatory, and so directly impact on which treatments are available to patients via the NHS.

[g] Letter from Chief Scientist and Vice President Strategic Initiatives, Canadian Agency for Drugs and Technology in Health (CADTH): "[g] CADTH_Letter.pdf"

The methods of the MPES group have been used and cited in health technology assessments in Canada for treatments for patients with type-2 diabetes, biologics for the treatment of rheumatoid arthritis, and treatment for atrial fibrillation. This has impacted on coverage decisions and clinical practice in Canada.

[h] Letter from Senior Director, National Evidence-based healthcare Collaborating Agency (NECA), Seoul, South Korea: "[h] NECA_Letter.pdf"

The methods of the MPES group are being used in other countries in health technology assessments for organisations similar to NICE, including South Korea.

[i] Letter from Vice President Health Economics, Oxford Outcomes: "[i] VicePresident_OO_Letter.pdf"

The research of the MPES group is contributing to the commercial success for Oxford Outcomes. A review by Oxford Outcomes identified 8 countries with guidelines on the use of MTC methods which cite the work of the MPES group.

[j] Letter from Managing Director, Global HEOR & Strategic Market Access, Mapi Consultancy: "GlobalDirector_Mapi_Letter.pdf"

The research of the MPES group and the training the MPES group has provided is acknowledged as contributing to commercial success for Mapi Consultancy.