Summary of the impact

In response to inadequately designed assessment systems for patients recovering from orthopaedic surgery, researchers from the University of Oxford developed a series highly reliable and sensitive patient recorded questionnaires, known as the Oxford Scores. Providing a set of standardised outcomes for appraisal and on-going monitoring of patients, the Oxford Scores enable the informed assessment of clinical outcomes. Used to predict and detect early failure of poorly performing surgical interventions, the Oxford Scores have been adopted by health providers and regulators worldwide, leading to policy and treatment guideline changes and significant improvements in the quality of life of patients.

Underpinning research

In the 1980s joint replacement surgery came into widespread use for patients suffering from osteoarthritis, or age-related degenerative joint disease. Due to the high demands of an actively aging population, rates of joint replacement surgery are now on the rise, with over 125,000 hip and knee replacements being performed annually in the UK, and over 1.2 million in the USA. As such, it is now even more important that joint replacement surgery improves the quality of life of patients. In the early 1990s it became clear that there were significant methodological deficiencies with the evaluation and reporting of orthopaedic surgical outcomes — particularly for implants and joint replacements. At the time, regular reviews of large numbers of patients for assessment of the long-term impact of surgical procedures (such as joint replacements) was rare, with the majority of patients receiving no monitoring. Clinical assessment in hospital was neither feasible nor affordable and introduced the potential for bias, due to inadequately designed assessment systems.

To combat this problem, between 1993 and 2008, Professor Andrew Carr, Professor David Murray, Professor Ray Fitzpatrick and Dr Jill Dawson of the University of Oxford's Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences and Department of Public Health designed a series of new scores for use following orthopaedic surgery, based on the then novel principle of patient reported outcomes (PROMs). Devising five 12 item questionnaires specific to patients recovering from total hip replacement¹, shoulder operations², total knee replacement³, shoulder instability⁴ and elbow surgery⁵, the group showed that their PROM questionnaires were capable of quickly, practically, reliably, and sensitively measuring clinical outcomes and important changes over time.

Designed through patient and clinician interviews, followed by refinement and testing, the Oxford Scores provide assessments of pain and function, as well as the social and psychological status of patients. The questionnaires are distributed to patients by post or deployed by various electronic platforms, making the follow-up of large study populations much more feasible and cost-effective than former clinical assessments, which require a return visit to hospital. This format also eliminates bias, as patients are able to complete the questionnaire independent of a clinical team or surgeon.

In addition to the successful measurement of clinical outcomes, this research also demonstrates a strong commitment to the involvement and engagement of patients in research. During the design of the Oxford Scores patients were involved from the very beginning of the research process, allowing the Oxford team to fully take into account real world health problems, to increase relevance, and to achieve the best possible assessment of outcome for patients.

The Oxford Scores' superiority over former assessment methods was confirmed in 2007, in a report commissioned by the UK Department of Health, produced by the London School of Hygiene and Tropical Medicine, and The Royal College of Surgeons. The Oxford Scores were ranked highest of all methods in this detailed comparative study, making them the preferred assessment tool for use in the UK and internationally⁶.

References to the research

This research was funded by the Department of Health, Oxford Regional Health Authority, The Norman Collisson Foundation, The Botnar Foundation, The Lord Nuffield Orthopaedic Centre Trust and The National Institute for Health Research (NIHR).

Details of the impact

The Oxford Scores have transformed the assessment of orthopaedic surgical outcomes worldwide and are being used in the United Kingdom and abroad to influence department of health policy and guidelines.

Clinical Use and Outcomes
In April 2009 the UK National Health Service (NHS) adopted the Oxford Scores for use by the Department of Health in all NHS Hospitals, to monitor hip and knee replacement operations⁷. In 2012 the National Joint Registry for England and Wales incorporated the use of the Oxford Shoulder Score in their national guidance for data collection⁸. A number of private providers also use the Oxford Scores to monitor quality⁹. Long-term outcome studies linking data from National Joint Registries to the Oxford Scores have demonstrated that approximately 20% of patients are dissatisfied with joint replacement surgery due to persistent pain. In addition, around 10% have some functional deficit. This is often in the absence of any technical problems with the surgery or the implant requiring revision surgery¹⁰.

In a cost-benefit study of the Oxford Knee Scores in 2009, clinicians from the South West London Elective Orthopedic Center, Surrey, UK reported that the Oxford Knee Scores had a 98% response rate from patients. The paper states: "The OKS is a short, practical, and easy to use patient-based questionnaire with good validity and a high completion rate. In our study, at 2 years, the response rate for Oxford questionnaire was 98%. It eliminates inter-observer error making it a reliable questionnaire" ¹¹.

In a large-scale independent study from Lund University Hospital in Sweden, the Oxford Scores were ranked as the best disease/site-specific PROM for assessing outcome of arthroplasty¹².

International Use
Licensed by Isis Innovation (a subsidiary of the University of Oxford, which commercialises intellectual property arising from academic research within the University), the Oxford Scores have been translated into 15 languages and are now available for use worldwide. They are currently available from Isis Innovation in: Polish, Finnish, Korean, Russian, Spanish, Chinese, Danish, French, Farsi, Japanese, Dutch, Portugese, Swedish, German and Turkish. Internationally, governments and departments of health have adopted the Oxford Scores to monitor the outcome and effectiveness of orthopaedic surgical procedures, particularly joint replacements. The use of the Oxford Scores in New Zealand and Scandinavia has allowed early identification (at 6 months post-surgery) of poorly performing implants, which subsequently require revision after 5 years. The early withdrawal of poorly performing implants significantly reduces the number of joint replacement failures and the attendant morbidity and cost¹³.

Clinical Guidance and Policy
The Oxford Hip and Knee Scores are routinely collected by the NHS, following joint replacement operations. These PROMs are co-ordinated by the Department of Health, while a number of organisations are involved in the collection, processing, analysis and reporting of PROMs data, including providers, primary care trust commissioners, the NHS information centres and contractors⁷. As a result of these routine collections, monthly and annual reports are published to inform patients, health care providers and commissioners on surgical outcomes⁷. The NHS also provides guidance on the use and interpretation of the Oxford Scores, including guides and video clips for patients and the public⁷. In addition, the Department of Health and Health Care Commissioners in the UK have adopted the Oxford Scores for use in measuring surgical outcomes¹⁴. More recently there have been moves by some health commissioners to use the Oxford Scores as a threshold for decision making regarding referral for surgery. The University of Oxford are working with health care planners and policy makers to determine how appropriate the use of the Oxford Scores will be as a decision aid for health commissioners.

Sources to corroborate the impact

7. NHS Choices. What are PROMs? (Accessed 2013) Available from
   http://www.nhs.uk/NHSEngland/thenhs/records/proms/Pages/aboutproms.aspx NHS Choices page offering advice to patients and healthcare providers on the use of PROMs.
8. National Joint Registry. National Joint Registry, launch data collection for shoulder and elbow joint replacements 2012. (Accessed 2013) Available from
   http://www.njrcentre.org.uk/njrcentre/tabid/240/Default.aspx Press Release from the National Joint Registry reporting the use of the Oxford Scores in data collection for shoulder and elbow joint replacements.
9. BUPA. Bupa calls for compulsory collection of PROMS in all private hospitals 16^th June 2008. (accessed 2013) Available from
   http://www.bupa.co.uk/about/html/pr/160608_proms_collection.html Report stating the compulsory use of PROMs as an assessment tool for private health provider Bupa.
10. Baker, P. N., van der Meulen, J. H., Lewsey, J., Gregg, P. J. The role of pain and function in determining patient satisfaction after total knee replacement. Data from the National Joint Registry for England and Wales. J Bone Joint Surg Br 89, 893-900 (2007). doi: 10.1302/0301-620X.89B7.19091 Paper reporting use of the Oxford Scores in determining patient outcomes following total knee replacement.
11. Medalla, G. A., Moonot, P., Peel, T., Kalairajah, Y. & Field, R. E. Cost-benefit comparison of the Oxford Knee score and the American Knee Society score in measuring outcome of total knee arthroplasty. J Arthroplasty 24, 652-656 (2009). doi: 10.1016/j.arth.2008.03.020 Cost-benefit study stating 98% response rate for the Oxford Knee Scores.
12. Dunbar, M. J., Robertsson, O., Ryd, L. & Lidgren, L. Appropriate questionnaires for knee arthroplasty. Results of a survey of 3600 patients from The Swedish Knee Arthroplasty Registry. J Bone Joint Surg Br 83, 339-344 (2001). doi: 10.1302/0301-620X.83B3.11134 Study comparing results from several PROMs questionnaires, including the Oxford Scores.
13. Rothwell, A. G., Hooper, G. J., Hobbs, A. & Frampton, C. M. An analysis of the Oxford hip and knee scores and their relationship to early joint revision in the New Zealand Joint Registry. J Bone Joint Surg Br 92, 413-418 (2010) doi: 10.1302/0301-620X.92B3.22913. Paper reporting use of the Oxford Scores in the early revision of hip and knee replacements in New Zealand.
14. NHS Guidance on Patient Reported Outcome Measures (PROMs)
    http://www.improvement.nhs.uk/pee/documents/DH_081179[1]PROMS.PDF (2009/2010) NHS Guidance on the collection of PROMs for the assessment of surgical outcomes, recommending the use of Oxford Hip and Knee Scores.