International impact on the measurement of patient health and its use in health care decision-making
Submitting InstitutionCity University, London
Unit of AssessmentEconomics and Econometrics
Summary Impact TypePolitical
Research Subject Area(s)
Medical and Health Sciences: Public Health and Health Services
Economics: Applied Economics
Summary of the impact
Methods for valuing quality of life developed by Professor Devlin at City
University London are used internationally to help governments make
healthcare decisions. Her research focuses on a widely-used questionnaire
for measuring patient reported health, the European Quality of Life
5-Dimensions (EQ-5D). Government organisations routinely use the EQ-5D to
judge whether new medicines work and are cost-effective. Over 15 countries
are undertaking EQ-5D studies using Professor Devlin's methods to inform
decisions on pricing and provision of new medicines. These developments
have been achieved through active dissemination to the academic community
and governments and through Professor Devlin's scientific leadership of
the European Quality of Life (EuroQol) Group. The impact of this research
is highly significant in improving health and health care decision-making
and has had wide reach throughout the UK and in many other countries.
Because health budgets are limited, new medicines can be funded only by
diverting money from existing treatments. Ensuring that health budgets
produce maximum health benefits for all patients and that services are
delivered efficiently requires governments to measure health. The EQ-5D is
a widely used questionnaire for measuring patients' self-reported health
and it is applicable to a wide range of health conditions and treatments.
Professor Devlin's research focuses on how best to translate EQ-5D
answers into health valuations (value sets). To put this in context, EQ-5D
answers generate health state valuations on a scale from 1 (= full health)
to 0 (= dead) and negative values (< 0) are given to states worse than
being dead. Professor Devlin's work in this area has produced
methodological advances and has influenced health care decision-making
(e.g., judgements on the funding of new medicines and the quality of
Professor Devlin's original research highlighted some EQ-5D data quality
issues and yielded the first EQ-5D scoring system (which translates EQ-5D
data into health valuations) for New Zealand. It is still used in all
decisions about pricing and access to new drugs.
This research was further pursued in her work after joining City
University London in 2002 (Devlin et al. 2004) and identified ways
that valuation methods could be improved and how health valuations equal
to 0 (`being dead') are handled (Devlin et al 2005).
During her years at City, Devlin's research identified fundamental
theoretical considerations related to how health state valuations of worse
than being dead (negative values) should be handled. The standard method,
"Time Trade Off" (TTO), generated problematic outcomes (Buckingham and
Devlin 2006 & 2009). In 2007, her research led to Devlin being invited
to join a newly-formed EuroQol Group taskforce and to lead a research
programme to develop a new approach. This allowed the identification of
the new method `lead time TTO' (Tilling et al. 2010, Devlin et
al. 2010) which provides a simple and consistent approach.
In 2010 Professor Devlin joined the Office of Health Economics (OHE) but
has maintained links with City University London as an honorary professor
and active research collaborations with City health economists Dr Patricia
Cubí-Mollá, Professor Mireia Jofre-Bonet and Dr Victoria Serra-Sastre. At
the OHE, Devlin leads an extensive research programme to refine further
the lead time TTO method (Devlin et al. 2012) which has been
piloted in eight countries in 2011 (Devlin and Krabbe, 2013). This
programme of methodological and piloting work culminated in the EuroQol
Group (the EQ-5D copyright holder) adopting the new method for use in all
countries. Studies are underway in over 15 countries (with others planned)
to produce country-specific values to inform future healthcare
In 2012 the Department of Health (DH) commissioned research led by
Professor Devlin to obtain new EQ-5D UK values, which will be used by the
National Institute for Health and Care Excellence (NICE) in its Health
Technology Appraisal (HTA) programme; the DH's new Value-Based Pricing of
medicines; and in the NHS routine monitoring of patient outcomes in
elective surgery, long term conditions, mental health and cardiothoracic
Simultaneously, Professor Devlin has pursued a programme of work on using
EQ-5D in assessing quality of health care (Devlin and Appleby 2010) and
measuring hospital and doctor performance. In addition to this
highly-relevant methodological work (Gutacker et al. 2013; Devlin
et al. 2010; Parkin et al. 2010), she advises Government
health care organisations in Sweden, Singapore, England, China and Canada
on the use of patients' reported outcome data to monitor quality of care.
References to the research
Devlin N., Buckingham K., Tsuchiya A., Shah K., Tilling C., Wilkinson G.,
& van Hout B. (2013). A comparison of alternative variants of the lead
and lag time TTO. Health Economics 22(5), 517-532. 10.1002/hec.2819
Devlin N., Tsuchiya A., Buckingham K., & Tilling C. (2010). A uniform
Time Trade Off method for states better and worse than dead: feasibility
study of the `lead time' approach. Health Economics 20(3),
Parkin D., Rice N., & Devlin N. (2010). Statistical analysis of EQ-5D
profiles: Does the use of value sets bias inference? Medical Decision
Making 30(5), 556-565. 10.1177/0272989X09357473
Devlin N., Parkin D., & Browne J. (2010). Patient Reported Outcome
Measures in the NHS: new methods for analyzing and reporting EQ-5D data. Health
Economics 19(8), 886-905. 10.1002/hec.1608
Buckingham K. & Devlin N. (2006). A theoretical framework for TTO
valuations of health. Health Economics 15(10), 1149-1154. 10.1002/hec.1122
Devlin N., Hansen P., Kind P., & Williams A. (2003). Logical
inconsistencies in survey respondents' health state valuations — a
methodological challenge for estimating social tariffs. Health
Economics 12(7), 529-544. 10.1002/hec.741
Grants relating to the research:
2008 to 2009 "An exploration of TTO methods for states worse than dead"
EuroQol Foundation PI: Nancy Devlin; CIs: Aki Tsuchiya & Carl Tilling
(Sheffield University); Ken Buckingham (University of Otago, NZ) £35k.
Substantial funding was also provided from the Department of Health
through its Policy Research Programme (2010 to 2011 PRP 070/0065 £325k;
2012 to 2014 PRP 070/0073 £475k) and from the Medical Research Council
(2010 to 12 £480k).
The quality of the research is evidenced by its impact and relevance; the
publication of key papers in high calibre journals such as Health
Economics and Medical Decision-Making; success in winning
substantial grants to support the work across an extensive period of time;
and by the judgments of peer reviewers on the deliverables from those
grants. For example, peer reviewers on the final report on grant NIHR PRP
070/0065 unanimously rated the overall project as excellent value for
money and as having high relevance to policy or practice.
Details of the impact
The impact of Professor Devlin's work in this area includes (a) impact on
the way health outcomes are `scored' and used to inform a wide range of
healthcare; and (b) increased use of the measurement of health outcomes in
many health care systems to monitor and improve the performance of health
care providers, quality of services and patient health.
The main pathway to impact has been via the EuroQol Group, a
not-for-profit organisation promoting the measurement of patient reported
health to improve health-care decision-making globally. It holds the EQ-5D
copyright, licensed to governments and pharmaceutical companies. Professor
Devlin has held several senior roles in the EuroQol Group and since 2010
is the organisation's scientific Chair. Following Devlin's methodological
work and its international piloting, the EuroQol Group assumed her
suggested approach for use in all countries (Oppe et al., 2013)
and will replace the EQ-5D value sets currently in use, giving a maximum
impact to Devlin's research.
The EQ-5D is required by NICE in evidence submitted on new technologies.
It is used in the English NHS Patient Reported Outcome Measures (PROMs)
survey on all NHS patients before and after elective surgery and other
medical procedures and in the GP Patient Survey which assesses patients'
experiences of the access and quality of care they receive from their
local GPs, dentists and out-of-hours doctor services. Other health care
systems also use EQ-5D to measure patient health and its use expands
further as countries such as China and Brazil move toward universal
Value sets based on Devlin's research while she was at City and
subsequently are being produced in the following countries: China, Hong
Kong, Singapore, Korea, Thailand, Taiwan, Japan, United Arab Emirates
(UAE), Uruguay, Canada, US, Spain, Germany, The Netherlands, New Zealand,
Denmark, Sweden and Norway as well as in the UK. In addition to her role
as Principal Investigator on the value set studies in England, the wider
UK and UAE, Professor Devlin is also an advisor to other studies around
the world via her role on the Value Set Working Group, providing technical
support and ensuring consistently high standards.
The value sets being produced in each country have multiple uses: firstly
in health technology appraisal (HTA) decisions about new medicines. NICE
2013's Methods Review specifically refers to the new value set study led
by Professor Devlin as relevant to NICE technical appraisal for
cost-effectiveness analyses. The UK value sets are accordingly being
awaited with interest by academics, industry and decision-makers. Secondly
in the UK EQ-5D data based on the value sets will be used as evidence to
set prices for all new medicines under Value-Based Pricing which will be
introduced in 2014.
Examples of the new EQ-5D uses abroad:
- The government of Abu Dhabi is supporting a study to establish a
mapping, specific to the Middle East, of EQ-5D answers to health
valuations for use in HTA decision-making about new medicines. The
Health Authority will also require all Arab Emirate nationals to
complete the EQ-5D yearly as part of its public health screening
- Professor Devlin has been working with Alberta Health Services in
Canada, which wishes to make routine use of EQ-5D in its health care
system. This agency is co-funding a value set study.
- Professor Devlin runs EQ-5D workshops for the Health Authority of Abu
Dhabi; Alberta Health Services; PHARMAC in New Zealand; the Chinese
National Health Development Centre (introducing EQ-5D to monitor the
quality of rural hospitals); officials in the Ministry of Health in
Sweden (EQ-5D is used in patient registries); a group of Asian
countries' Government officials in Singapore (developing HTA processes
to include the EQ-5D); and health care officials in the Philippines.
Professor Devlin also works extensively with Government organisations to
support their use of EQ-5D and value sets. She contributed to the 2012
NICE methods review on measuring health outcomes; advises Monitor on how
to take into account patients' evidence when regulating the quality of
health care providers in England; and has a role in the advisory group to
the English DH's PROMs programme.
The impact of this research is highly significant in improving health and
health care decision-making and has had wide reach throughout the UK and
in many other countries.
Sources to corroborate the impact
a. The use of Devlin et al (2003)'s value set in the health technology
appraisal process of New Zealand's Pharmaceutical Management Agency
(PHARMAC) is documented in: PHARMAC
(2012) Prescription for Pharmacoeconomic Analysis: Methods for
Cost Utility Analysis. Wellington: PHARMAC.
b. The report from peer reviewers on the NIHR PRP grant 070-0065 referred
to on p.3 can be obtained upon request to the Policy Research Programme
Central Commissioning Facility (PRP CCF) of the Department of Health.
c. Documentation for the EQ-VT is available to any research team wishing
to undertake a value set study; whilst the EQ-VT user guide and related
materials are not in the public domain, they can be provided upon request
to the EuroQol Group: www.euroqol.org.
d. Examples of Nancy Devlin's work with overseas government and health
care organisations on their use of EQ-5D referred to in section 4 can be
verified as follows:
e. The EuroQol Group's adoption of the methods developed by Professor
Devlin is evidenced in a key paper setting out how such studies should be
performed internationally: Oppe M, Devlin N, van Hout B and de Charro F(2013)
An international protocol for the valuation of EQ-5D-5L
(Value in Health — forthcoming).
f. Section 4 refers to a suite of international pilot studies undertaken
in 2011 to test and further explore variants of the TTO methods developed
by Nancy Devlin. The results are about to be published in a special issue
of the European Journal of Health Economics, which may be verified
by contacting the editor in chief.
g. Professor Nancy Devlin's role as the elected Chair of the EuroQol
Group may be verified via the EuroQol Group's website: www.euroqol.org/euroqol-group/executive-committee.html.
h. The list of countries currently undertaking value set studies using
the protocol based on Professor Devlin's methods may be verified by
contacting the EuroQol Group's Value Sets Working Group.