Summary of the impact

Professor Hutton's research considers the biasing effect of selection of data due to consent procedures or selective reporting, and its consequences for the validity of conclusions and reliability of results. This research has had impacts on patients directly; on health and legal professionals by informing and influencing national and international guidelines for the treatment of epilepsy used by healthcare professionals and practitioners; and has provided expert evidence to legal professionals for the conclusion of civil litigations and a General Medical Council professional misconduct trial. Hutton's research also informs ethical debate associated with the validity and robustness of study results. This work has determined guidelines for ethical conduct of research, and requirements for publications, which are significant for all biomedical researchers.

Underpinning research

Ideally, a statistical study should accurately sample from the entire population of interest, but actual statistical trials may miss some types of patients and may not report all outcomes. Therefore careful analysis of resulting biases is required to ensure the statistical integrity of conclusions.

The underpinning research analyses (a) the effects of biases caused by incomplete and often selective reporting of data; (b) subsequently related ethical issues; and (c) substantive application to the understanding of epilepsy and drugs used to treat it (e.g., Vigabatrin). The body of research was carried out at Warwick by Professor Hutton, Department of Statistics since 2000, and involved collaboration with researchers at The University of Liverpool (UoL). Specifically:

(a) Fundamental methodological research investigating how selection bias depends upon correlations in data, with a specific focus on the effect of incomplete reporting of sub-group analyses, was carried out in [1, 2, 3], and these papers include substantive medical examples. This work involved collaboration with researchers at UoL.

(b) Further research [4, 5] focuses on related ethical issues concerning the design and analysis of cluster randomised trials. For example, different methods of infection control applied to different hospitals are assessed by results of individual patients, and therefore individual consent can lead to a skewed population of patients with respect to which biases need to be assessed.

(c) Collaboration with neurologists (at UoL) who specialise in epilepsy provided the inspiration for much of the research in (a) and (b) including the impact of missing data and misclassified factors [6, 7, 8], and selection of patients into clinical trials and into follow-on studies after clinical trials [3, 9]. Part of the research was carried out under an MRC grant [11] supporting a Research Associate, Dr Hemming (Warwick) and a Clinical Research Fellow, Dr Maguire (UoL). The Warwick team developed sensitivity analyses to assess biases arising from patient self-selection in open label extension studies, which reported very different results from randomised clinical trials [9]. A further potential source of bias in meta-analysis is missing information on study or patient characteristics; Hemming and Hutton proposed a Bayesian approach to assessing such bias [10]. This method demonstrated that rates of visual field defect increased with dose and duration of Vigabatrin treatment.

In all this research, Professor Hutton took the lead on statistical and ethical issues and methods.

References to the research

11. J.L. Hutton (PI) `Models for selection bias, applied to controlled trials and observational studies of antiepileptic drugs' MRC G0400642 July 2005-Sept 2008 £199,000

Details of the impact

Hutton's research on the effects of selection bias includes development of new statistical methods, and consequently direct application of the results to particular diseases and treatments, and to the implications for good conduct and reporting of studies. The impact has therefore been in three areas:

A) Specific clinical guidance on the treatment of epilepsy
Epilepsy is a common neurological disorder affecting over 500,000 people within the UK (http://www.nhs.uk/Conditions/Epilepsy/Pages/Introduction.aspx). In around 70% of cases, seizures are successfully controlled by AEDs (anti-epileptic drugs) which are the 5^th highest category of expenditure on NHS England prescriptions. The 2004 NICE (National Institute for Health and Clinical Excellence) Guidelines for the Diagnosis and Management of Epilepsy highlighted inadequacies in care and treatment of epilepsy patients. In 2007 a major multi-centre study SANAD (Standard and New Antiepileptic Drugs), which compared the clinician's choice of drug against new AEDs in over 2,000 patients, was published. The design of this trial, led by UoL, was determined by Hutton's research. Her analyses had highlighted uncertainty with regard to interactions between drugs, and patient factors of age, type of epilepsy and seizure type related to misclassification bias [6, 8]. The significance of the SANAD trial is considerable, e.g.:

1. SANAD played a central role in the construction of the 2012 NICE Guidelines [12a], "The primary scope of the guidelines was to consider the role of antiepileptic drugs, especially given the impact of important, real-world studies such as SANAD. The role of established and newly licensed drugs has been considered using novel statistical methods allowing comparison of cost effectiveness" (Preface P3). The Guidelines are important since they are "expected to be taken into full consideration by healthcare professionals and organisations when deciding on treatments for patients" [12b]. A Consultant in Neuropsychiatry [12c] states that "SANAD is the best clinical trial and is the gold standard piece of work in relation to the treatment of epilepsy. It is referred to in the NICE guidelines for good reason. The other main strength of SANAD is that it was not constrained by the needs of the pharmaceutical industry and has the reputation of being relatively bias-free".
2. The World Health Organisation (WHO) guidelines "Evidence-based recommendations for management of epilepsy and seizures ..." cite the meta-analysis [13a]. In addition, in [13b], the SANAD trial results were explicitly highlighted when WHO deliberated whether to allow the application for a new epilepsy drug treatment regime. SANAD trial results were used in a WHO decision not to include Lamotrigine for epilepsy in their Model List of Essential Medicines.
3. SANAD determined the recommendations of the German Association of Scientific Medical Societies, for AEDs to be used to treat first seizures and epilepsy in adults [14].
4. The Scottish Intercollegiate Guidelines Network [15] cites meta-analyses (eg [8]) by Hutton as justification for its treatment recommendations.
5. Hutton was instructed [text removed for publication], as an expert witness for the claimants in a multi-party class action [text removed for publication], after which the case was concluded with a confidential out-of-court settlement.

B) Generic guidance for the conduct and reporting of biomedical research
There are many examples where Hutton's research [3, 4, 5] has been cited and used inter alia by policy makers, journal editors, and potential study participants, to provide guidance on the ethical design and conduct of cluster randomised trials, including the following examples.

1. Impacts on guidelines for statistical validity and ethics in cluster randomised trials [4,5] include:

• One of the MRC's clinical trials guidelines "Cluster Randomised Trials: Methodological and Ethical Considerations" [17] is based largely on the research in [4].
• Recommendations (from [4, 5]) are also incorporated into the Consolidated Standards of Reporting Trials (CONSORT) Design extension to cluster randomised trials [18]. CONSORT is in turn included within the International Committee of Medical Journal Editors (ICMJE) recommendations [19].
• Recommendations (from [4, 5]) are also incorporated into the Ottawa Statement on the Ethical Design and Conduct of Cluster Randomized Trials [20].

2. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [21] is an evidence-based checklist for reporting systematic reviews and meta-analysis, which has been translated into Spanish, Korean and Russian. Its recommendations are also incorporated into the ICMJE [19] to which most biomedical journals subscribe, and thus whose authors must comply with each item on a specified checklist. Hutton's work in [3] contributed to two check points related to selective reporting within studies.

3. Research led by Hutton for the National Centre for the Replacement, Refinement and Reduction of Animals in Research attracted media coverage, and contributed to reporting guidelines in Animal Research: Reporting of In Vivo Experiments (ARRIVE) [22a]. This is endorsed by many journals and ten funders, including the Wellcome Trust and three research councils [22b].

C) Evidence given in other trial cases as an expert witness
Based on her research on selection bias, ethics and meta-analysis, Hutton has made recent appearances as an expert witness in legal cases. They include:

1. The General Medical Council instructed Hutton in May 2008 in a case concerning the conduct, design, choice and reporting of outcome measures of a clinical trial for which three doctors were accused of professional misconduct [23]. The case 'collapsed because it had no sound scientific evidence to support it' [23], as a direct result of Hutton's discussions with the GMC and her report based on her research including [1,2].
2. Based on her work including [3, 7, 9], Hutton was
   [text removed for publication]
   (over 100 cases) [24a, 24b, 24c].

Sources to corroborate the impact

12a. Pharmacological Update of Clinical Guideline 20. "The Epilepsies: The diagnosis and management of the epilepsies in adults and children in primary and secondary care". Final Methods: Evidence and Recommendations January 2012. Commissioned by the National Institute for Health and Clinical Excellence.

See also the NICE Clinical Guideline 137, January 2012, pg 7: ".. a recent large multicentre trial (the SANAD trial) evaluating newer drugs in newly diagnosed epilepsy (accepting some limitations) suggested that sodium valproate should the drug of choice...It was therefore considered necessary to review new evidence regarding AEDs within an update of NICE clinical guideline 20).

12b. See: http://www.nhs.uk/NHSEngland/thenhs/healthregulators/Pages/nice.aspx

12c. Consultant in Neuropsychiatry, National Centre for Mental Health, Birmingham

13a. WHO guidelines `Evidence based recommendations for the management of epilepsy and siezures in non-specialised health settings: Standard antiepileptic drugs (phenobarbital, phenytoin, carbamazepine, valproic acid) for management of convulsive epilepsy in adults and children' http://www.who.int/mental_health/mhgap/evidence/resource/epilepsy_q7.pdf

13b. The Selection and Use of Essential Medicines. Report of the WHO Expert Committee 2009, (WHO Technical report Series: 958), ISBN 978 92 4 120958 8

14. German guidelines: Epileptic Shock and Epilepsy in Adults (AWMF No 030/041, dated September 2012). For complete document see http://www.awmf.org/leitlinien/detail/ll/030-041.html

15. Diagnosis and Management of Epilepsy in Adults — A National Clinical Guideline (Scottish Intercollegiate Guidelines Network. ISBN 1 899893 58 X, Updated October 2005. For full document see: http://www.sign.ac.uk/guidelines/fulltext/70/

16. [text removed for publication]

17. "Cluster randomised trials: Methodological and ethical considerations" MRC Clinical Trials Series, November 2002. The guidelines continue to have impact — see current MRC Additional Terms and Conditions 'MRC requires research organisations to ensure that the research undertaken under an award by the research organisation itself complies with MRC terms and conditions including MRC's ethics and best practice." See
http://www.mrc.ac.uk/Utilities/Documentrecord/index.htm?d=MRC002406.

18. See http://www.consort-statement.org/consort-statement/ Hutton is cited in Reference 248

19. http://www.icmje.org/icmje-recommendations.pdf. See p12 for PRISMA and CONSORT.

20. Ottawa statement, PLoS Med 9(11): e1001346. DOI: 10.1371/journal.pmed.1001346

21. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. Liberati A et al. PLoS Med 6(7): e1000100. DOI 10.1371/journal.pmed.1000100. Hutton is cited in references 122 and 152.

22a. Kilkenny C et al (2010). Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research, PLoS Biol 8(6): e1000412. DOI: 10.1371/journal.pbio.1000412. See Reference 5 to Hutton's research.

22b. See http://www.nc3rs.org.uk/news.asp?id=1861 for the reach of the ARRIVE guidelines and http://www.nc3rs.org.uk/news.asp?id=1798 for link to an open letter from the CEOs of the BBSRC, MRC and Wellcome Trust.

23. Gornall J. Professional conduct — Three doctors and a GMC prosecution BMJ 2008; 337:a907. Download at http://www.bmj.com/content/337/bmj.a907

24a. [text removed for publication]

24b. [text removed for publication]

24c. [text removed for publication]