Summary of the impact

The Variable Life-Adjusted Display (VLAD) is a graphical tool for monitoring clinical outcomes. It has been widely adopted by UK cardiac surgery centres, and has helped a shift in culture towards more open outcome assessment in adult cardiac surgery, which has been credited with reduced mortality rates. VLAD is also being used for a broad range of other clinical outcomes by regulatory bodies worldwide. For example, Queensland Health uses VLAD as a major part of its Patient Safety and Quality Improvement Service to monitor 34 outcomes across 64 public hospitals, and NHS Blood and Transplant uses VLAD to monitor early outcomes of all UK transplants.

Underpinning research

In the UK in the mid-1990s it was discovered that prolonged periods of poor performance by individual cardiac surgeons had been going undetected. This highlighted the need for clinical outcomes — such as the rate of mortality within 30 days after surgery — to be routinely monitored.

Researchers in UCL's Clinical Operational Research Unit (CORU) — a team dedicated to applying operational research and mathematical modelling approaches to problems in health care — collaborated with cardiothoracic surgeon Tom Treasure to develop a monitoring tool that clinicians would find useful. As part of the engagement process necessary for successful operational research, the CORU team spent months attending seminars and meetings at St George's Hospital, where Professor Treasure worked at the time, to "tune in" to how surgical teams discussed outcomes and related to data. A key challenge was how to account for differences in case-mix (e.g. different severity of patients' heart disease) between centres, so that meaningful comparisons can be made and clinicians or hospitals that undertake more risky cases are not unfairly penalised.

This research led to the development in 1997 of a novel graphical display for outcome monitoring called the Variable Life-Adjusted Display (VLAD) [1]. The VLAD is a plot of the difference between the cumulative expected mortality and the cumulative observed mortality as a function of case number (or, in later versions, time). The expected mortality takes into account the risk associated with each case, as estimated using an existing risk scoring system. For each death within 30 days the VLAD trace falls by the estimated probability of survival for that case; for each survival within 30 days it rises by that case's estimated probability of death. This simple, intuitive display was the result of the engagement process, and repeated prototyping and discussion between the CORU team and the surgical author. In addition to the incorporation of patient-to-patient differences, key to the success of VLAD has been the explicit "credit" given to clinical teams for runs of better than expected outcomes.

In the mid-2000s, CORU extended the methodology to add flexibility and aid interpretation of VLAD charts. In 2004, a collaboration with Cambridge's Papworth Hospital and Guy's and Thomas' Hospital Medical School led to the addition of graphical tools, based on exact analytical methods, which allow the user to see how likely it is that deviations from expected surgical outcomes occur by chance [2]. A method was then devised in 2005 for augmenting the basic VLAD chart with a "signalling" function based on CUSUM analysis, adding information as to whether an upwards or downwards trend in clinical outcomes constitutes a statistically significant deviation from expected performance [3].

Although originally developed to monitor outcomes in adult cardiac surgery, the VLAD technique has since been applied in many other clinical settings. The CORU team has been active in this research area; for example, in collaboration with University College Hospital (UCH), they adapted the technique for monitoring the occurrence of surgical wound infections in hospitals in 2007 [4], and implemented it at UCH for this purpose in 2011 [5]. In 2010-12, CORU also worked on outcome monitoring using VLADs after paediatric cardiac surgery, first helping to develop a dedicated risk model (known as PRAiS) to adjust for case-mix differences and then working with three UK paediatric cardiac surgery centres (Great Ormond Street Hospital, Evelina Children's Hospital in London and The Royal Hospital for Sick Children in Glasgow) to implement local routine monitoring [6].

CORU's contribution to all the research above included engagement with the clinical communities to build a shared understanding of the clinical context and the purpose of monitoring; data analysis and model development; and design and implementation of graphical tools and software.

Key UCL researchers: Jocelyn Lovegrove (Research Fellow; 1995-99), Stephen Gallivan (Senior Research Fellow to Professor, then Principal Research Fellow; 1985-2010), Chris Sherlaw-Johnson (Associate Research Assistant to Senior Research Fellow; 1990-2006), Christina Pagel (Research Fellow to Lecturer in Operational Research; 2005-current), Sonya Crowe (Research Associate to Health Foundation Improvement Science Research Fellow; 2009-current), Martin Utley (Research Fellow to Professor of Operational Research; 1996-current).

References to the research

References [1], [3] and [6] best indicate the quality of the underpinning research.

Details of the impact

Monitoring of short-term outcomes using VLADs is now conducted within many adult cardiac surgery centres in the UK and other countries, including India (Sri Jayadeva Institute of Cardiology, Bangalore; since 2010), Singapore (National University Heart Centre; since 2009), Greece (Onassis Cardiac Surgery Center, Athens; since 2011) and Sweden (Örebro and Linkoping University Hospitals; since 2010). The technique has impacted on surgical units as it allows them to analyse and compare the performance of individual surgeons and ensures that any appropriate action relating to an unexpected increase in mortality can rapidly be taken. One UK-based surgeon informed us that VLADs are "invaluable for quality assurance" within his cardiac unit and give him "great confidence in the overall performance of surgeons and the unit" [A]. In 2011, the Society for Cardiothoracic Surgery in Great Britain & Ireland (SCTS) reported a "50% reduction in risk-adjusted mortality in the United Kingdom in recent years" as a result of the collection, analysis, benchmarking and feeding back of robust data on clinical outcomes for the purposes of quality improvement [B], which is facilitated in part by the use of VLADs. The SCTS also believes that these improved processes are the cause of the reduction in recent years of damaging cardiac surgeon suspensions and restrictions of practice, as they lead to "detection of potential problems at an early stage, allowing implementation of strategies to improve outcomes before any restriction of practice or suspension may be needed." [B]

In paediatric cardiac surgery, software developed by CORU (which uses VLADs with the PRAiS risk model) was sold under licence in 2013 to all 12 UK (NHS and private) centres performing this type of surgery, and is being used by them for routine monitoring of outcomes. This use of VLADs has been incorporated by NHS England into the quality assurance checklist they developed for commissioners of paediatric surgery services [C]. The relevant national audit body, NICOR, has also purchased the software and used it in their comparative analysis of outcomes in the 10 English centres [D], which followed the suspension of paediatric cardiac surgery at Leeds General Infirmary in April 2013. Their analysis indicated that there were no `safety' problems in any of the centres [D]. Outside of cardiac surgery, VLADs have found use in the monitoring of surgical wound infection rates at University College Hospital in London, and the monitoring of mortality rates within the general adult Intensive Care Unit at Waikato Hospital in New Zealand.

NHS Blood and Transplant uses VLADs (together with CUSUM charts) on a national level to monitor early outcomes of all transplants undertaken in the UK's 23 kidney, 8 pancreas, 7 cardiothoracic and 7 liver transplantation units [E]. Each significant change in the rate of mortality or graft failure generates a signal that leads to an investigation. For example, in 2011 monitoring indicated that Royal Brompton & Harefield NHS Foundation Trust had experienced more deaths than expected following heart transplants. This prompted an external review conducted by two senior clinicians, and the eight recommendations of this review have now been implemented by the trust [F]. These included developing "a consensus approach to the management of primary graft dysfunction and failure", and making sure that "cardiothoracic retrieval surgeons at the donor operation are made aware of any need for delay so as to ensure minimised ischaemic times". In the 2011 UK Liver Transplant Audit, VLAD charts revealed that no significant deviation from expected mortality had occurred in paediatric centres since 2008, but that a significant change had occurred in January 2011 in the Newcastle adult centre, which led to that centre conducting an internal review of its service [G].

Since March 2009, VLADs have been used by the Veterans Health Administration (VHA), a component of the United States Department of Veterans Affairs and the USA's largest integrated health care system, serving over 8.3 million veterans each year. VLAD charts are incorporated into the VHA's national quality improvement project to monitor mortality on acute medical and surgical units at 127 VHA centres. Charts are updated on a quarterly basis and made available to managers or analysts at centres as part of a quarterly report package. To help these users interpret the information, the VHA prepared educational materials in November 2010 and has held several training sessions since June 2009. The VLAD chart is "well perceived by managers for its ease of use and its ability to alert users to investigate care process during a specific period." The VHA has informed UCL that it is not possible to isolate the contribution of VLAD in improving mortality since it is part of a national quality improvement program that involves other tools and improvement strategies, but that they have seen an "improvement in mortality over time and consider VLAD an important tool that signals periods needing investigation" [H].

Since 2007, VLADs have been a component of the UK Care Quality Commission's (CQC) nationwide surveillance programme, in which they are used as a presentational tool to guide interpretation. Within this programme the CQC monitors a selection of outcomes (including maternity and emergency re-admissions indicators) across all 163 acute NHS hospitals in England, in addition to adverse events in other care sectors such as adult social care and mental health. The CQC has handled over 650 alerts under this programme; in recent years 60-70% of these alerts have led to improvement plans being implemented in NHS trusts [I]. Improvement plans included those for "better management of patient fluid balance, the complete redesign of patient pathways, improved identification of early warning signs and more efficient links with primary and community care" [I]. In one case, an alert identified high mortality among patients admitted with a hip fracture. The trust reviewed their care for these patients and identified remediable problems at specific points in patients' care; to address these they developed and shared an improvement plan [I].

The enhanced approach to VLAD charting devised by Sherlaw-Johnson (reference [3] above) was adopted in 2007 by Queensland Government's Department of Health as part of their clinical governance framework; VLADs were introduced into the state's largest public and private hospitals as a major part of the Queensland Health Patient Safety and Quality Improvement Service. This was followed by a partnership between Queensland Health (QH) and the software company Opus 5K to develop the VLAD Clinical Monitoring (VLAD CM) IT system, which enabled QH to deploy VLAD charting in over 64 Queensland hospitals in October 2009, where it is currently used to monitor 34 clinical indicators [J]. On-going rigorous reviews of indicators are conducted by VLAD Indicator Review Working Groups [J, K].

The Queensland Government's VLAD Policy (2012) [J] governs the use of VLADs within QH and details the following procedure: VLAD CM disseminates monthly VLAD charts to hospitals, indicating where predetermined levels of variation in patient outcomes are exceeded and flagging issues for further review. Hospitals are required to investigate why flags have occurred and submit a response within 30 days. In 2010-11, around 1,000 VLAD charts were disseminated each month, the Queensland Health Peak Safety and Quality Committee VLAD Subcommittee reviewed 382 hospital investigation reports written in response to flags, and 300 clinical reviews by hospital staff occurred as a result of VLADs [L]. The use of VLADs has resulted in the implementation of numerous quality initiatives within Queensland hospitals, leading to improvements in areas such as discharge processes, clinician documentation and resource allocation [K].

Sources to corroborate the impact

[A] Supporting statement from a cardiac surgeon at Royal Victoria Hospital, Belfast — corroborates that cardiac surgery at this hospital is benefiting from VLADs. Available on request.

[B] Maintaining Patients' Trust: Modern Medical Professionalism 2011, available online http://www.scts.org/_userfiles/resources/634420268996790965_SCTS_Professionalism_FINAL.pdf — corroborates the SCTS's view that outcome monitoring has led to improvements.

[C] Supporting statement from Service Specialist at NHS England — corroborates the incorporation of VLADs into NHS England's quality assurance checklist. Available on request.

[D] Investigation of mortality from Paediatric Cardiac Surgery in England 2009-12, available online http://www.england.nhs.uk/wp-content/uploads/2013/04/finl-rep-mort-paed-card-surg-2009-12.pdf — corroborates the use of VLADs and PRAiS by the national audit body in their analysis.

[E] Supporting statement from the Associate Director of Statistics & Clinical Audit at NHS Blood and Transplant — corroborates the numbers of transplant centres in which VLADs are implemented. Note that this statement refers to VLAD charts as O-E charts. Evidence that these are the same thing can be found in Collett et al. (2009) The UK Scheme for Mandatory Continuous Monitoring of Early Transplant Outcome in all Kidney Transplant Centers, Transplantation, 88, 970-5 (page 971). Available on request.

[F] Royal Brompton & Harefield NHS Foundation Trust Response to NSCT External Review Report of 29^th December 2011, available online at http://www.rbht.nhs.uk/healthprofessionals/clinical-departments/transplant/ — corroborates the implementation of the recommendations by the trust.

[G] UK Liver Transplant Audit 2011 — corroborates the use of VLADs, the findings of the audit and the internal review at the Newcastle centre (e.g. see pages 9-10 and 57). Pdf available on request.

[H] Supporting statement from the Innovations and Development Coordinator at the VHA — corroborates that VLAD charts are being used by the VHA to monitor outcomes and that it finds them beneficial. Available on request.

[I] Supporting statements from the Surveillance Manager at the CQC — corroborates that VLADs are used in the surveillance programme, and corroborates the details of that programme and the improvement plans. Available on request.

[J] Queensland Government VLAD website: http://www.health.qld.gov.au/psu/vlad/default.asp — corroborates the VLAD Policy, indicators, and activity of Indicator Review Working Groups.

[K] Using the quality improvement cycle on clinical indicators — improve or remove?, K. M. Sketcher-Baker, M. C. Kamp, J. A. Connors, D. J. Martin and J. E. Collins, Med. J. Aust., 193, S104-S106 (2010) http://bit.ly/19mirG2 — corroborates the implementation of quality initiatives leading to improvements.

[L] Patient Safety: from learning to action 2012, available online http://www.health.qld.gov.au/psu/reports/docs/lta5.pdf — corroborates numbers of VLAD charts disseminated, investigation reports reviewed, and clinical reviews written. See page x (in the executive summary) and page 58.