Summary of the impact

Research into the Human-Computer Interaction (HCI) issues behind medical error has enabled hospitals to procure safer devices and is strategically changing attitudes. Approximately 10% of deaths due to preventable errors in hospitals are computational errors; in absolute numbers, this is higher than road fatality rates. Corresponding increased hospital stays, etc, are estimated to cost the NHS over £600m per annum. Our formal analyses and laboratory research with clinicians show error rates can be reliably reduced dramatically by software and HCI improvements (for details see reference list). Swansea's research has significant international reach across decision-makers, clinicians, manufacturers, and regulators. The US Food and Drug Administration (the FDA, the leading regulator) say research-based improvements will take decades and have therefore started to co-author papers with us to improve impact (all healthcare manufacturers watch the FDA closely).

Underpinning research

Our original research discovered that, and analysed how, interactive medical devices (e.g., drug delivery systems) ignore certain classes of critical user error, and that by managing error better, patient harm can be reduced. Clinicians make data entry slips (about 4% of keystrokes entered by nurses cause unnoticed errors [R1]), and errors are parsed badly and misinterpreted on many devices [R5]. If neither user nor device notices errors, this can lead to adverse incidents.

We developed rigorous (e.g., using PVS [R3]) and novel (e.g., using eye tracking with nurses [R1]) methods for analysis and evaluation that have transformed the understanding of user interface safety for healthcare. Our approaches refine theories and design techniques to assure safety, and ensure systems are more resilient to human error. Our analyses of systematic design variations led to techniques to reduce severe errors by factors up to 25 compared to devices in current use in hospitals. We thus developed evidence-based principles, which have been used in NHS procurement [C2] to purchase £1.5m of safer devices.

Impact is viewed in the context of preventable hospital deaths: 17% are due to numerical/calculation errors alone (under-reported, since few clinicians notice them). We found paediatric intensive care patients have a 20% chance of avoidable drug calculation errors per week. Exact figures are contentious; the BMJ estimates UK acute hospital preventable fatality rates of 14,000pa — but it is a mistake to focus too much on fatalities: extended hospitals stays and litigation costs the NHS over £600m pa, and there are social and financial impacts on patients, relatives, and on clinicians too.

High quality prior work. Thimbleby's [R4] was the foundational, peer-reviewed, international conference paper for the EPSRC Programme Grant [G1], defined by EPSRC as "recognition of highest-quality and world-leading research". It identified major shortcomings in infusion pumps for the first time, and developed a methodology for identifying similar problems.

The funding supports a team at Swansea: 3 PhDs; 2 RAs; 3 PDRFs; 2 co-Is in Computer Science, 2 in College of Medicine; and a full-time Impact Facilitator:

The UOA team consists of PI: Professor Thimbleby (since 2005); co-Is: Dr Eslambolchilar (since 2006, Associate Professor), Professor Matt Jones (since 2006); PDRFs: K. Li (since 2010), G. Neizen (since 2012), H. Tu (since 2012); RAs: A Gimblett (since 2008), P Oladimeji (since 2008); PhDs: A Cauchi (since 2010), A Lewis (since 2011), T Owen (2009 to June 2013); FT Admin: J Doyle, V Hurst.

[R5] led to a large new research stream within the EPSRC Programme Grant led by Swansea. Some papers have been officially co-authored with the FDA, the world leaders in medical regulation. Continued research has led to international best paper prizes, at ACM CHI and ACM EICS (Engineering Interactive Computer Systems) etc.

Papers have been used for course material for workshops with B-Braun (Europe's largest medical device manufacturer) in Swansea, Sheffield (B-Braun's UK base) and Germany [R5]. 3 PhD students have won best paper prizes for both doctoral work and for mainstream refereed papers. Several of our papers have been cited by the US Institute of Medicine in major reports.

References to the research

Authors in bold are in the UOA.

R1, R3 and R5 best represent the quality of the research.

In addition to the EPSRC Programme Grant mentioned in section 2 above, the research has attracted significant support from other sources:

[G1] H. Thimbleby, co-investigator (Swansea PI), EPSRC Programme Grant, EP/G059063/1, "CHI+MED: Multidisciplinary Computer-Human Interaction research for the design and safe use of interactive medical devices," £6.7M fEC with UCL, QMUL and industrial partners, 2009-2015. See www.chi-med.ac.uk — site for more details of the research, impact, results, industrial collaborations, etc.

We also obtained EPSRC Creativity@home funding, EPSRC Building Global Engagements in Research, and Impact Acceleration Account funding. We are CoIs on the eHealth Industries Innovation Centre project in Swansea's College of Medicine. Our work has led to other EPSRC grants, including EP/F059116/1, EP/G003971/1 (Platform Grant), EP/J020834/1, and a Royal Society-Leverhulme Senior Research Fellowship (2008/9).

Details of the impact

Healthcare suffers because of preventable errors. Our impact lies in showing the benefit of rigorous Computer Science and in changing attitudes. IT is a solution but also a current problem. Improved design is a cost for manufacturers given the prevailing culture of blaming clinicians instead; manufacturers bemoan "regulatory burden" and, under current legislation, might rather ignore problems. The impact of our research can be described in the following ways:

Policy debate stimulated and informed by research. We conceived of and ran 5 "Tully Meetings" (named after Prof Colin Tully, one of the founders with Thimbleby of the NHS23 group) to bring >200 top computer scientists and clinicians, including UK military, Department of Health, CfH (Connecting for Health), reporting into the Royal College of Physician's Future Hospital Commission and their work on e-prescribing [C3, C6].

Potential losses mitigated by improved risk assessment by health service. Collaborating with the NHS we showed in the largest ever study [R2] poor design of infusion pumps induces wastage of ~£1,000/pump/year. This led to retraining, and informing manufacturers of improvements. [R5] was used in requirements for a £1.5m procurement.

Public discourse stimulated and informed. We have given 120+ presentations (in REF period) at medical conferences, industrial seminars and workshops at international conferences such as ACM CHI (2010; 2011; 2013), ACM EICS (2011; 2012; 2013), BCS HCI (every year), as well as presentations to CTOs of B-Braun [C5], Boston Scientific, clinical organizations, hospital trusts, etc [C7]. Thimbleby was Clinical Human Factors Group trustee in its formative year, helped it raise funds and profile, and remains on its Advocacy Panel. The Royal Society and Gresham College invited public lectures.

Improved quality, efficiency & productivity of professional service. Drug doses rely on complex calculations. We developed calculators that eliminate some sources of error completely. A prototype was exhibited at a Royal Society Summer Science exhibition and in TECHFEST, India, etc. With a consultant anaesthetist we provided evidence nomograms have advantages over conventional methods for a number of medical calculations, including time-pressured burns resuscitation [R6, C1].

Defined best practice for professional bodies and learned societies. The Royal College of Physicians, the Royal College of Anaesthetists, the Royal College of Physicians Edinburgh, the Royal College of Pharmacists, Guild of Healthcare Pharmacists, Scottish Intensive Care Society, Central Sterilising Committee, Scottish Clinical Skills Network, etc, have had invited lectures from us. We have undertaken confidential assessments of medical devices for the NHS in respect of criminal and professional investigations.

Thimbleby has been invited on the Royal College of Physicians (RCP) Commission, and elected Honorary Fellow, the "highest honour the RCP can bestow on a non-medically qualified person" as well as Fellowship of the Edinburgh RCP; evidence of high-level recognition of the impact of this unique work.

Supporting and enabling staff to achieve impact. We appointed an Impact Facilitator with £70k for impact activities specifically in this area in August 2013.

Demonstrating reach. We consulted with Huawei on mhealth (mobile health) [C4], with B-Braun (largest European manufacturer) and consulted on future devices [C5]. A spin-out company THIS (Tech Health Interactive Solutions) (www.this.eu.com) was formed in 2013 to enable the reach and frequency of consultancy to expand.

Public engagement has been achieved through participation at major exhibitions, including venues such as the Bombay Institute of Technology's TECHFEST, the Welsh Eisteddfod, TEDx, etc, each attracting thousands of public visitors. All our workshops have patient representatives and clinicians. Other non-Computer Science lectures include International Federation of Information Processing groups, Stanford Research Institute, MIT, Cambridge, Berkeley, Oxford, Stanford. We ran workshops in China with leading medical schools (Jiao Tong; Fudan), Huawei [C4] and Smiths Medical.

Thimbleby has been invited to write for clinical journals, as well as present over 95 invited lectures/keynotes (in the REF period) at clinical conferences and training days, UK and internationally; talks were attended by ~1,800 clinicians undertaking continuing professional development (total direct audience reach of over 6,700 with ~43% clinical and manufacturers) [C7]. For building a completely new field, this is significant reach. We were invited by Tata to run a masterclass for leading software engineers. Half of the Formal Methods in Interactive Systems 2013 papers by other groups develop our work.

Our films are shown in clinical conferences, having impact beyond our control: "I presented a session at a clinical risk conference yesterday in London and used [your video] and it went down a storm [...] The feedback was excellent and I've already received requests from other people [...]" [C2].

Significance is evidenced by international recognition of the healthcare improvements, the quality of the work (including 5 best paper prizes), and by peer-reviewed evidence conducive for high impact in healthcare; our research shows that preventable deaths could be halved using modified UIs for data entry [R5]. Impact is increasing, but even if applied unchanged today we could prevent ~7,000 UK deaths/year, with additional benefits in finance and wellbeing, and additional impact for the second victims, the clinicians.

Our impact must be viewed against cultural resistance to improve system safety. Our research and evidence-based approaches to mitigate that, and consultancy and our evangelisation of the results is both direct impact and conducive to impact. We have engaged and continue to engage across the spectrum, from hospital officers and consultants to the UK and Welsh Chief Medical Officers as well as international leaders like the World Health Organisation and FDA. For example, Sir Liam Donaldson (Chief Medical Officer, 1998-2010, and current Chair of the World Alliance for Patient Safety) asked to co-author with Thimbleby a high-profile and high-impact article for the British Medical Journal specifically to raise the awareness and impact of this work.

Sources to corroborate the impact

[C1] Consultant Anaesthetist, Abertawe Bro Morgannwg University Health Board, Morriston Hospital, Swansea.

[C2] Devices Manager, Abertawe Bro Morgannwg University Health Board, Singleton Hospital, Swansea.

[C3] Director, Royal College of Physicians Health Informatics Unit, London.

[C4] Head of mHealth Solutions & Marketing at Huawei Technologies, Huawei Ltd, Basingstoke, Hampshire, UK.

[C5] Vice President Global Marketing & Sales, B-Braun AG, Melsungen, Germany.

[C6] cs.swan.ac.uk/~csharold/tully — Tully Meetings records, etc.

[C7] Spreadsheet of participants at seminars/keynotes/workshops.