Summary of the impact

Research undertaken at Sheffield by Barkham and Hardy has led to a suite of self-report questionnaires assessing mental health and wellbeing. These measures are widely used as assessment and outcome measurement tools for patients, and include versions tailored for different populations. They accurately assess psychological problems and the outcomes of treatment, and enable the benchmarking of service delivery. The measures have been widely adopted nationally within the UK by statutory NHS services and voluntary sector services, as well as by university counselling services, employee assistance programmes, and private providers. The measures have been translated into 20 languages and taken up internationally. They have been instigated as a matter of policy by national government bodies and by practitioner organisations, been integrated into computerised interventions (a mode of delivery approved by NICE), provided local services with evidence for their effectiveness, and underpinned a commercial business. They support the delivery of high-quality services in the field of psychological therapies.

Underpinning research

The Clinical Outcomes in Routine Evaluation (CORE) is a self-report measure of subjective well-being, symptoms of psychopathology (e.g., anxiety, depression), social functioning, and risk to self and others. It was designed as an Outcome Measure in the evaluation of psychological interventions, and is known nationally and internationally as CORE-OM. The papers cited in Section 3 demonstrate the material and distinct contribution that research conducted at Sheffield has made to the impact of the CORE-OM and wider suite of CORE measures.

Proof-of-concept work for a CORE outcome battery was carried out by Barkham, together with Hardy and Shapiro, at the University of Sheffield MRC Social and Applied Psychology Unit. Initial funding from the Mental Health Foundation (£34k; PI: Barkham; Title: Feasibility of developing a CORE outcome battery) supported (a) empirical work to test the feasibility of developing the outcome measure and (b) conceptual work to devise a useful outcome battery. Barkham moved to University of Leeds (1995-2007) before returning to the Clinical Psychology Unit in Sheffield (2007-current).

Work on the CORE-OM continued in Sheffield during Barkham's absence. For example, Hardy and Shapiro collaborated to show that CORE was an appropriate tool for understanding the processes and outcomes of psychological therapies (Stiles et al., 2003 [R1]). This research demonstrated that CORE was a clinically meaningful measure of the benefits of sudden behavioural gains across disorders. Since returning to Sheffield, Barkham has conducted a programme of research to develop the CORE assessment battery. This includes large-scale studies validating CORE-OM as a routine measure of psychotherapy outcomes (e.g., Stiles et al., 2008, patient N = 9703 [R2]) and demonstration of the applicability of the CORE-OM to forensic populations (Perry et al., 2013 [R3]). It has also involved the development, implementation and validation of a suite of CORE measures for specific situations and populations. This includes the YP-CORE (Twigg et al., 2009 [R4]), which is a version of CORE for use with children and adolescents (11-16 age range). The structure of the latter is similar to that of the CORE-OM but with items rephrased to be more easily understood by the target age group. In addition, the CORE-10 was developed, which is a short 10-item version of the CORE-OM to be used as screening tool and outcome measure when the CORE-OM is considered too long for routine use. It can be used for the purpose of tracking session-by-session change, as required by Improving Access to Psychological Therapies (IAPT) services (e.g., Barkham et al., 2013 [R5]). Finally, the CORE-6D (Mavranezouli et al., 2011 [R6]) is a 6-item version of CORE that provides an index in terms of quality-adjusted life years (QALYs), enabling determination of the cost-effectiveness of interventions using the metric preferred by NICE.

References to the research

[plus SCOPUS citation count]

Details of the impact

This programme of research has achieved impact at four levels: (1) direct influence on clinical practice and its development, (2) informing and influencing local and national policy, (3) underpinning a commercial business, and (4) development of a novel research paradigm to inform clinical guidelines. As well as describing these high-level changes, we provide some local examples of the way in which CORE has been used in practice to enhance services and patient care.

Impact on clinical practice

Until the past decade, mental health services used a disparate array of outcome measures. Many were not validated and were not useful for developing service quality or improving patient outcomes. The CORE provided a free-to-use measure with a support and a feedback process, so that clinical data could assist services. Sheffield-based research has led to the CORE suite of measures being widely adopted within NHS primary and secondary services (approximately 150 services currently), changing the landscape of outcome measurement in the UK and internationally (where the CORE system is used in Portugal, Holland, Norway, Denmark, and Sweden). It now has an online distribution and data collation network (CORE-Net), backed by a commercial arm (CORE-IMS — see below). Service users rate CORE-OM as one of the best outcome measures available [S1].

Mainstream health service settings

Within the current REF time frame, the CORE is one of two or three measures that have been most widely adopted by psychological services in the UK, with the highest popularity among clinicians (see the report by the Centre for Health Economics, University of York [S2]). The Royal College of Psychiatrists identified that 80 out of 357 participating services were using the CORE-OM for routine assessment of outcomes from psychological therapies [S3].

Other clinical settings

The adoption of the CORE-OM has been particularly impactful in counselling services, which have used it to provide evidence of the effectiveness of their interventions (e.g., person-centred therapy). Example services that use the CORE-OM include Khiron House (www.khironhouse.com), which has a Harley Street Day Clinic and an Oxfordshire Residential Clinic, and Counselling Works (www.counsellingworks.co.uk), which offers sessions face-to-face and via the internet.

The CORE-OM has also been adopted by clinicians in other health service settings, such as practitioners working with low-vision services and nurses in Intensive Care Units. Another example is the Leeds Women's Counselling Centre, which has adopted the CORE-OM as their evaluation tool and used it to demonstrate positive and long-lasting benefits from their treatment.

There is also evidence of change in clinical provision as a direct result of implementation of CORE measures. For example, the `My Sister's Place' service for victims of domestic violence reports that it has changed several central elements of its provision as a direct result of using the CORE system. Those changes affect their assessment procedure, model of care, provision for those with more complex problems and those from more marginal groups, and testing of the value of newly introduced therapeutic approaches [S4].

University & college counselling settings

The online CORE mechanism (CORE-Net) has been tested for utility in evaluation of university and college counselling services under the auspices of the Association of University and College Counselling division of the British Association of Counselling and Psychotherapy. The system is available to all such institutions via a network called UNICORE. At present, 35 student-counselling services within the UK have implemented the CORE-OM as a routine measure.

Children and young people

The measures are increasingly being used in counselling young people. For example, YP-CORE is used at Entrust Associates (www.entrust-schoolcounselling.com) — a London-based counselling service for schools, and in child counselling at Therapeutic Education Counselling Centre (Birmingham) (www.teccservices.co.uk). Off The Record Youth Counselling (www.offtherecordcroydon.org) uses CORE-OM in their services for 11-25 year-olds.

International uptake

The CORE-OM's international uptake is evidenced by its translation into 20 languages. (www.coreims.co.uk/About_Core_Translations.html). There are official CORE-OM websites in Spanish (www.ub.edu/terdep/core/#.Ul3TvihTTGI) and Dutch (www.telepsy.nl/core), and it is also used in Portugal, Norway, Denmark, and Sweden.

Impact on policy

A number of national governments and practitioner organisations recommend use of the CORE suite of measures as a standard component of service delivery and evaluation. This includes the Scottish Government [S5]; the Welsh Government (where most local authorities direct their school counselling services to use the YP-CORE, and uptake is increasing [S6]); and the Department of Education for Northern Ireland (which requires school counsellors to use one of three routine evaluation measures, with preference expressed for the YP-CORE [S7]). The CORE-10 is one of two measures recommended by the UK Royal College of Psychiatrists (2012) for evaluating outcomes of psychological therapies with older adults [S8], and is the only generic outcome measure suggested by the Royal College of Psychiatrists (2011) for evaluation of the range of psychological therapies among adults [S9].

Commercial impact

The research has underpinned the creation of a commercial business, CORE-IMS [S10], which has enhanced the implementation of the measure and enabled the development of large data sets (as outlined below). Barkham is a company director (listed at Companies House) of the CORE System Trust.

The suite of CORE measures, their development (e.g. translation) and their distribution into new markets are managed by CORE-IMS, which awards licences for the use of two computerised outcome management systems — CORE PC and CORE Net. CORE-IMS has been established for 10 years and employs 7 people. The company reports that there are over 250 organisations across health, workplace, education, voluntary and private sector services that use CORE PC, with an estimated 3500 psychology, psychotherapy and counselling practitioner users within those organisations. Furthermore, in 2010, 40 organisations held licences for CORE Net in the UK, Portugal, Norway, Holland and Denmark. CORE IMS also distributes CORE-OM (for free), and reported that, in 2010 alone, 3000 individuals downloaded copies for their personal use.

A major provider of online treatment for depression (as recommended by NICE) is Ultrasis Ltd, which provides a package called Beating the Blues (BtBs). The CORE-OM is packaged within BtBs, which is widely used in the UK and has now been rolled out in Australia, Canada, New Zealand and the Netherlands. Ultrasis confirm delivery of BtBs to more than 100,000 people. Those CORE-OM data have been used to support the implementation of BtBs in New Zealand.

Development of a novel research paradigm to inform clinical guidelines

Widespread adoption of the suite of CORE measures has resulted in securing very large clinical data sets, resulting in the development of the Practice-Based Evidence paradigm (as outlined in the 2010 book of the same name, Barkham, Hardy & Mellor-Clark [Eds]). Traditionally, the efficacy of psychological interventions has been described by the evidence-based practice paradigm, founded on the synthesis of outcomes from a limited number of randomised control trials. Critiques of this latter approach have been concerned about the generalisation of these research trials to routine practice. Barkham's approach to adopting large data sets such as CORE-OM — collected regularly through routine clinical practice — provides a complementary evidence base for the derivation of clinical guidelines and good practice advice. The inclusion of practice-based evidence in addition to evidence-based practice has been acknowledged in the 2010 report New Ways of Working for Psychological Therapists, commissioned by the National Institute for Mental Health in England (NIMHE) National Workforce Programme.

Sources to corroborate the impact

S1. Mental Health Research Network (2012). Outcome Measurement in Mental Health: The Views of Service Users. London, UK: National Institute for Health Research. http://tinyurl.com/nuk5cgq

S2. Jacobs, R. (2009). Investigating Patient Outcome Measures in Mental Health (CHE Research Paper 48). York, UK: Centre for Health Economics, University of York. http://tinyurl.com/nq9qmsu

S3. Cape, J. et al. (NAPT Steering Group) (2011). National Audit of Psychological Therapies for Anxiety and Depression: National Report. London, UK: Royal College of Psychiatrists. http://tinyurl.com/nrhy9tq

S4. The report on the My Sister's Place changes in practice following the use of CORE-OM is available at: http://tinyurl.com/nuuaukl

S5. Reshaping Care and Mental Health Division (2011). Recommendations Following the Response to National Consultation on the Standardisation of Outcomes Measurement for Adult Psychological Therapy Across Scotland: A Discussion and Consultation Paper. Edinburgh, UK: The Scottish Government. http://tinyurl.com/ohyl4o9

S6. Rees, A. (2011). Evaluation of the Welsh School-based Counselling Strategy: Stage One Report. Cardiff, UK: Social Research Division, Welsh Government. http://tinyurl.com/nqg2e6d

S7. Independent Counselling Service for Schools (2012). Handbook. Bangor, UK: Department of Education for Northern Ireland. http://tinyurl.com/op9onvs p.23,

S8. Royal College of Psychiatrists' Faculty of the Psychiatry of Old Age (2012). Individual patient outcome measures recommended for use in older people's mental health (Occasional Paper 86). London, UK: Royal College of Psychiatrists. http://tinyurl.com/qh7qhat

S9. Hampson, M. et al. (2011). Outcome Measures Recommended for Use in Adult Psychiatry (Occasional Paper 78). London, UK: Royal College of Psychiatrists. http://tinyurl.com/o4kjp8w

S10. The work of the commercial distribution arm (CORE-IMS) is detailed on their website: www.coreims.co.uk