Summary of the impact

Events in the UK NHS have shown the need for a robust understanding of hospital mortality rates.

Surrey's research produced "a unique web-enabled pattern analysis system that is specifically designed to enable clinicians and their teams to view in detail their in-house mortality patterns in the national context" (a).

Launched on a national scale in Ireland in 2013, it has already identified `mortality outliers' and been described as a `game changer' for improving service quality at national level. The tool's impact stems from its ability to translate statistical patterns into a form readily usable by health professionals to improve care quality and sharing best practice.

Underpinning research

The Problem

Comparative hospital mortality has been recognised as a key method to monitor quality of care within hospitals by identifying hospitals that appear as mortality outliers (references 3.1; 3.3). Although increasingly significant, the methods of calculating and then acting upon such data are contentious, especially in health systems that are national in scale and funded primarily from the public purse. Even if the data collected is robust, the statistical interpretation is difficult and can lead to false positive and false negatives. Such errors incur significant costs, in terms of the initiation of unnecessary or incorrect measures, anxiety caused to patients and staff, and organizational disruption (reference 3). In the Republic of Ireland to date, there has been no national system in place to compare hospital mortality across the publicly funded health service. This project has provided such a system

The Research

This project (led by Professor Simon Jones) has refined the statistical processes involved and, uniquely, developed a user-interface that enables more reliable interpretation of results by practitioners and non-statisticians. The objective was to create an intelligent infrastructure to compare hospital mortality nationally on an on-going basis. The research was based on all patients discharged from hospital in Ireland between January 2005 until the 31 December 2011. The following predictor variables were included in the model: Type of admission; Source of admission; Number of admissions in the previous 12 months; Age on admission; Gender; Medical card status; A modified version of Charlson co-morbidity score (reference 2). For each group with greater than 40 deaths per analytical cell within the time interval, a unique logistic regression equation is calculated using the predictor variables; when a group contains fewer deaths, the crude death rate is employed as logistic regression models would tend to fail to converge or result in an intercept- only model. The expected number of deaths per hospital is calculated by summing the predicted number of deaths per group for each institution. The hospital standardised mortality ratio (HSMR) is calculated and plotted against the expected number of deaths for each hospital to produce a funnel plot with 95% and 99.8% confidence intervals/control limits. Those hospitals outside the 99.8% interval/limit are referred to as "outliers", where there is a one in 500 chance that the result is due to randomness. Computations are carried out using R (version 2.12.1).

The research involved setting up a secure infrastructure to analyse over 12 million patient records from 70 hospitals and 250 medical conditions (reference 2)

Relevance

The robust statistical techniques coupled with a novel user-interface and self-generated feedback methodology provides a unique way of sharing safety-critical data with practitioners in a form that empowers action. This allows national, regional/specialty and individual hospital views of comparative mortality patterns close to "real time". The unique web-enabled pattern analysis system enables clinicians and their teams to view in detail their in-house mortality patterns in the national context and to identify areas of good practice or in need of investigation.

References to the research

Details of the impact

The research has had impact in three of the areas:

Health and Welfare.

This project extends statistical approaches to hospital mortality data by combining mathematical rigour with strong information governance and unprecedented usability for practitioners not trained in advanced statistical techniques. This translates into health and welfare impacts in the form of: rapid (i.e., close to real-time) identification of unexpected mortality patterns and the type of investigatory measures required. These impacts result from the identification of positive and negative mortality patterns: positively, it allows the identification of best practice, the sharing of which can improve outcomes for patients; negatively, by allowing the rapid investigation of excessive mortality rates, it has the potential to prevent needless deaths (reference 2). The quantification of the relationship between statistically anomalous mortality rates and needless deaths is highly contentious, but it is plausible to assert that rapid investigation is likely to be of benefit to patients, potentially in significant numbers.

Economic, commercial, organizational.

In addition to alleviating human suffering, the better understanding of hospital mortality rates has economic and organizational impacts. As reported by The Clinical Lead for Health Intelligence at the Irish Heath Service Executive (the body that commissioned this work), the "statistical model was developed appropriate for the Irish context. Because generic models are often viewed with suspicion by practitioners, a tailored model has allowed the resulting data to gain credibility on the ground, resulting in a greater willingness to engage with the process and act on its outcomes" (a). Because the results are available in close to real time and accessible to clinicians in a comprehensible form, health organizations gain from speedy adoption and from enhanced staff commitment to improvement, translating into real but not easily quantifiable saving in resources. In addition, the system has been made available to every public hospital in the country and its "drill down" functionality allows each hospital to view and extract record-level data that underpins local audits of data and care. This unique feature offers significant gains in speed, efficiency and validity of assessment. As such its impact at national level is substantial (access to the live web-site is restricted on confidentiality grounds but can be accessed if necessary for audit).

Practitioners and professional services.

As indicated above, the system's unique user interface allows unprecedented levels of involvement in data interpretation across clinical, administrative and management groups. As such, it has been claimed to be a "game changer" in developing the ways in which health professionals collaborate across institutions and professional groups to provide improved patient services (a).

Reach and Significance.

The reach of this project is significant: according to Dublin Health Services Executive (a): these tools "will reach into every public hospital in the country [Republic of Ireland]".

In terms of significance, this model and its resulting applications have the potential form major gains in the areas outlined above. Firstly, it is a new development in the Irish public health system that not only brings it in line with other developed countries (reference 1), but also goes beyond what is available elsewhere through its unique user interface. Although only in live operation since August 2013, it has already indicated cases for investigation and initial results indicate its effectiveness in engaging practitioners. As maintained in the evidence letter (a): these tools are "game changers in terms of promoting patient safety by empowering targeted quality assurance from the national to local levels".

Sources to corroborate the impact

a) Dublin Health Services Executive. (provided statement)

b) Assistant Director, Health Intelligence Unit. (contact details provided)

c) Confidential web site (access available for audit purposes only; details with University of Surrey)