Summary of the impact

Globally, many health research-funding organisations, public and charitable, felt the need to demonstrate to policymakers and the public how their investments in research were benefitting society. HERG's research on developing techniques for assessing the payback (or impact) from health research tackled this need. The payback stream of research itself has had significant, wide- reaching and cumulative impacts. First, internationally, health research funding bodies adopted the framework in their evaluation strategies, including to provide accountability. Second, many stakeholders made extensive use the findings of payback studies in public debate and private lobbying for public expenditure on health research. Third, governments, public research funding bodies and medical research charities, from the UK to Australia, used the findings from payback studies to inform decisions regarding the levels and distribution of health research funding, with the aim of increasing the health and economic benefits that come from investments in research.

Underpinning research

In 1993 the Director NHS R&D wanted to develop methods of showing the benefits from, and thus justifying, spending NHS resources on health research. He asked Martin Buxton, the Director of the Health Economics Research Group (HERG) at Brunel University, to explore this area. Buxton recruited Stephen Hanney to HERG to work with him on what became an extensive research programme to develop and apply the Payback Framework. They worked on this, inter alia, at HERG from 1993 to 2013; from 2000 an information scientist, Teresa Jones, joined the HERG Payback team, as did Claire Donovan in 2010 as Reader in the Assessment of Research Impact. The Payback Framework consists of a multidimensional categorisation of benefits, and a logic model to help organise assessment of benefits. The HERG team tested and applied it in various case studies [1]. Diverse organisations funded further developments and applications of the framework. HERG conducted studies independently, or, more often, worked with collaborators, principally RAND Europe (Jonathan Grant, Steve Wooding, Wija Oortwijn and others) and people associated with funding bodies. Bodies that funded studies led by, or involving, HERG include:

1) Medical research charities both in the UK (eg the Arthritis Research Campaign [2]; Asthma UK) and overseas (eg Heart and Stroke Foundation of Canada and the National Heart Foundation of Australia; and, in 2012, the National Breast Cancer Foundation - NBCF - in Australia)

2) Publically funded health research programmes both in the UK (eg the NHS Health Technology Assessment (HTA) programme [3]; Northern Ireland's health department - DHSSPS) and overseas (eg ZonMW's HTA programme in Holland [4]; a major NIH programme in the USA).

Such applications facilitated methodological developments, including increasingly sophisticated consensus scoring of levels of impact achieved (first by the Payback team alone [2], then with HTA experts [4]) and analysis of medical research charities' whole portfolios (eg Asthma UK, NBCF).

HERG conducted further methodological work, including with the WHO, on how the economic benefits from health research could be better identified [5]. Building on this, in 2007/8 Buxton led a team including RAND Europe and the Office of Health Economics (OHE) (Jon Sussex and team) in a study for the UK Evaluation Forum and funded by MRC/Wellcome Trust/Academy of Medical Sciences (AMS). This research, secured following a competitive process organised by the MRC, developed and applied new ways to assess the economic impact of UK publicly and charitably funded health research. We focussed on two major elements of the economic returns: health gains net of the health care costs of delivering them; and the GDP gains, ie the UK national income that results from the medical research and the further research stimulated by it. The extensive and original `bottom-up' research calculating the value of health gains from research in two major health care fields (cardiovascular disease (CVD) and mental health), plus the GDP gains from the same research, required publication as a major peer-reviewed report to allow presentation in sufficient detail for full public scrutiny [6]. The study showed that every GBP1.00 invested in CVD research will yield approximately GBP0.39 worth of total benefits - health gain plus GDP - every year.

References to the research

Details of the impact

HERG's payback research has produced significant and cumulative impacts, including on public debates and policy, and on the organisation of public services and medical charities. The Payback research provides new evaluation processes that have been applied by organisations nationally and internationally. The impacts were partly generated by extensive stakeholder engagement and dissemination activities by team members (see below). Many stakeholders, in turn, promoted the HERG findings showing the benefits that come from health research, and used this to support their case for continued and/or increased funding. For example, those attending the seminar in Nov 2008 at the Wellcome Trust to launch HERG's Medical Research: What's it Worth report to key stakeholders included: the Chief Executive of the MRC and the Director Generals of R&D at the DH and at the science department. In 2010 the UK Science Minister demonstrated the report's considerable importance: `The report on Medical Research, What's it Worth? is regarded by BIS Analysis teams as very comprehensive and rigorous. It was used as evidence as part of the preparations for the Spending Review and it will continue to be cited in the foreseeable future.' [1]

Examples of the detailed impact of HERG's Payback stream are organised into three sections below to demonstrate both the research's wide reach and its cumulative significance.

1) Adoption of HERG's payback framework in organisations' evaluation strategies or approaches: The wide use of the Payback Framework arises partly from the decisions of many public and charitable research funders either explicitly to request that specific impact studies be organised according to the Payback Framework and/or to include it in their strategies. Examples include:

• The main Canadian public sector, charitable, and commercial organisations funding health research combined to sponsor an expert panel including Buxton. Tasked with identifying a best method to evaluate impacts, it proposed a framework that `builds on the combined logic model and impacts approach of the "payback model"'(p.18). The President of Alberta's main health research funding body has described its adoption in provinces of Canada and in Spain. [2]
• In 2009 ICDDR,B in Bangladesh, one of the leading research centres in low-and-middle- income countries, incorporated the Payback Framework prominantly into its `Strategic Plan 2020' covering 2010-2020 and set out how it would be put into practice. [3]

2) Impact of HERG's payback work on stakeholders' political campaigns to support public funding: A wide and international range of (health) research stakeholders (including public bodies, charities and public campaigns) have quoted findings from HERG's payback studies as key evidence when advocating support for medical research. Here we give just a few of the many examples of stakeholders who cited the HERG et al (2008) report: Medical Research: What's it Worth.

• In 2010 the Spending Review was intended to make major cuts in Government expenditure. In their evidence to the Spending Review many medical research stakeholders cited the HERG 2008 report prominently in their arguments in defence of investment in medical research, eg the Association of Medical Research Charities (AMRC), specific charities, public funders. [4,1]
• The report was also cited in policy debates as key evidence in submissions to the government Spending Review by stakeholders concerned to defend investment in publicly funded research in general, including by the Commons Science and Technology Committee and by a public campaign (Science is Vital) that engaged thousands of scientists and others [5].
• More generally, diverse UK, European and international organisations used the HERG 2008 report to promote investment in medical research, including investment in research for international development; it was the first reference in a Royal Society submission on this [6].

3) Impact of the HERG payback work on policy decisions by governments and charities to sustain and increase health research funding and enhance the organisation of research systems: The findings from various payback studies are not only used in policy debates, but in a range of ways have been cited as influencing the policy decisions of governments and other research funders in relation to issues about the level and organisation of medical research funding.

• When the results of the UK Government's 2010 Spending Review were announced medical research had done comparatively well; various stakeholders identified the HERG 2008 report as a key part of the evidence that had been successfully used to influence the decision [4,7].
• In 2012 the MRC launched a research programme on understanding the link between research and economic impact and demonstrated the continuing policy-relevance of the HERG 2008 report: `Internationally there has been little new evidence that could assist funding policy in the UK since the "Medical Research: What's it Worth?" report'. [8].
• In 2011/12 HERG worked with a local team from RSM McClure Watters to apply the Payback Framework to assess the impact of health research funded by the Northern Ireland Executive. Based on that study the Executive changed its policy and announced: `Following consideration of the report, and the benefits outlined in it, the Health Minister has decided to invest £2.6 million' to become a full partner of the National Institute of Health Research (NIHR). [9].
• Asthma UK funded HERG to lead an evaluation of the impacts of their research funding. They used the analysis of how different funding streams had contributed to impacts to help revise their funding strategies, and hoped the report's publication would boost research funding. [10]
• When in April 2013 the NBCF in Australia launched the impact assessment conducted by HERG it received a high media profile, which was important to increase public understanding of the charity's achievements, provide accountability for past expenditure, and justification for future funding by supporters. On policies it also stated: `As a consequence of this evaluation process NBCF has decided to take three actions in terms of future investment in research....we have decided to substantially boost our investment in translational research.' [11]

Extensive dissemination helped generate the full range of these impacts. It included publications, invited presentations, seminars, and advice directly to stakeholders, eg on committees in Canada [2], the EU, Ireland, WHO and the UK. The resulting impact, illustrated by the key examples above, has potentially had even wider implications. The additional/better organised health research that arose (at least partially) as an impact from HERG's work might, itself, reasonably be expected on occasions to have generated some of the range of wider impacts including on health and welfare.

Sources to corroborate the impact

1) In 2010 the Science Minister, David Willetts, wrote to one of our colleagues describing how the Medical Research: What's it Worth (HERG, et al, 2008) report had been used in by BIS in their preparations for the Spending Review. A pdf copy of the letter is available from Brunel.

2) A Canadian report from the CAHS's Panel on the Return on Investments in Health Research, www.cahs-acss.ca/making-an-impact-a-preferred-framework-and-indicators-to-measure-returns-on-nvestment-in-health-research-8/ (2009) set out the impact of the Payback Framework on its recommendations. Also in a corroborative statement (available from Brunel) the chair of the Panel and current President of Alberta Innovates-Health Solutions (AIHS), wrote to Martin Buxton: `confirming the major impact of your work on the panel that led to our CAHS framework, and on its adoption by AIHS and a number of other health research funders in Canada and abroad.'

3) ICDDR,B. Strategic Plan 2020. Almost a whole page (17) of the 28 page plan is devoted to a slightly adapted version of the HERG multidimensional categorisation of benefits and a description of proposed use, although unlike in the drafts the final version does not carry references. Available: http://www.icddrb.org/what-we-do/publications/doc_download/107-strategic-plan-2020

4) Many major medical research funding organisations used the HERG 2008 report as key evidence in the 2010 Spending Review including Cancer Research UK (2010) Available: http://info.cancerresearchuk.org/prod_consump/groups/cr_common/@nre/@pol/documents/generalcontent/cr_048533.pdf The chair of the AMRC had previously chaired the Commons Science and Technology Committee until 2010. His corroborative statement (available from Brunel) highlights the significance of HERG's report: a) in AMRC's many submissions to the government in support of medical research funding and to protect the Charity Research Support Fund (CRSF), and b) in lobbying by others groups. He stated: `It is my belief that this contributed to the Treasury's decision to ring-fence the Science Budget, which includes the CRSF, for the 2010-15 financial period'

5) Many stakeholders used HERG's 2008 report as key evidence, in the policy debate prior to the 2010 Spending Review, to support research funding in general, eg the Commons Science and Technology Committee: Committee contribute to 2010 Spending Review consultation: Letter from the Chair of the Committee to the Chancellor of the Exchequer, 27 July 2010, available: http://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/news/100729-letter-to-chancellor/ and the grassroots public campaign: Science is Vital (2010) Key messages. http://scienceisvital.org.uk/2010/09/28/key-messages/

6) Organisations using the 2008 HERG report for more general campaigns for research investment include the European Commission Science Advisory Board in Health. It issued a call in The Lancet `for more and better investment in health research'; the only reference was HERG 2008. [Lancet 377:719-720]. The Royal Society used it in a submission to DFID's 2009 Consultation: "Eliminating World Poverty: Assuring our Common Future". See RS Policy Document 07/09. Available: http://royalsociety.org/uploadedFiles/Royal_Society_Content/policy/publications/2009/7876.pdf

7) The importance of HERG's 2008 report in influencing the decision to protect medical research funding was emphasised in a corroborative statement from a Deputy Director, R&D Directorate, DH (available from Brunel) that says HERG's 2008 report `was an important factor in convincing the Government to protect the science budget in the 2010 and 2013 Spending Reviews, and in 2010 to raise the health research budget in real terms...and is still referred to in Government itself.'

8) The 2012 MRC research call Understanding the link between research and economic impact highlighted the international importance of the HERG 2008 report in informing policy. Available: www.mrc.ac.uk/Fundingopportunities/Highlightnotices/Research_economicimpact/MRC008598

9) The Northern Ireland Executive press release of 5 July 2012 explained how the report on the impact of its research (based on the HERG Payback Framework) led to the Health Minister's decision to invest £2.6 million by subscribing Northern Ireland to the NIHR . Available: http://www.northernireland.gov.uk/news-dhssps-050712-we-expect-four?WT.mc_id=rss-news

10) In creating its 2011-16 Strategy, Asthma UK stated the HERG impact analysis: `has given us a unique insight into our research and provided powerful information to guide our future strategy.'(p3) Available http://www.asthma.org.uk/Handlers/Download.ashx?IDMF=392dba83-7a51-4b1a-b23c-7190df76fa30 Asthma UK also said of the article on the study: `The charity hopes that proving its impact will help generate further funding opportunities towards their search for new knowledge, treatments and, ultimately, a cure': http://www.asthma.org.uk/News/where-does-my-money-go

11) In April 2013 NBCF launched HERG's report assessing the impact of the charity's research. Pp5/6 of the speaking notes explain how the findings had already made an impact on its funding strategies. Available: http://www.nbcf.org.au/Stories/Research/HERG-Evaluation-Report.aspx