Assessment of Mood and Cognitive Function
Submitting InstitutionUniversity of Aberdeen
Unit of AssessmentPsychology, Psychiatry and Neuroscience
Summary Impact TypeSocietal
Research Subject Area(s)
Medical and Health Sciences: Public Health and Health Services
Economics: Applied Economics
Summary of the impact
Research carried out at the University of Aberdeen has addressed
important clinical needs in neuropsychology/ clinical psychology. The work
has provided large sample normative data for psychological tests which
clinicians can use for comparison with a patient's test score. It has also
resulted in methods enabling them to draw inferences on individual patient
performance, and methods to use existing published data for reference
purposes. All of these have been made available to clinical practitioners
in user-friendly computer programs. Together, they have impacted directly
on the quality of care for people with neurological and psychological
conditions in the UK and worldwide, as well as the training of clinical
The resulting claimed impact has been on healthcare professional
guidelines and training. Practitioners have used these research findings
and tools in the conduct of their work.
For psychologists and other clinicians, comparing a patient's score on
some form of test (for example a measure of cognitive ability or mood)
against a reference sample (often a healthy normative sample) is a
fundamental process when conducting an assessment or working towards a
diagnosis. There are three essential requirements to this process: first,
adequate normative (reference sample) data must be available; secondly,
sound inferential methods are required to compare the patient against this
reference sample; and thirdly, the methods need to be made available to
clinicians in user-friendly form.
It is on these three aspects that John Crawford, Professor of Psychology
at the University of Aberdeen since 1996, has focused a systematic
programme of research in collaboration with, primarily, Professors Paul
Garthwaite from the Open University and David Howell from the University
In research projects carried out between 1996 and 2011, Crawford et al
gathered normative data for mood scales (notably anxiety and depression)
from large, broadly representative samples of the general adult UK
population. The studies showed that demographic variables had only very
modest influences on anxiety and depression scores, and that anxiety and
depression scores are moderately correlated. Crawford et al concluded that
the normative data gathered were suitable for clinicians to assess the
rarity of a given anxiety/depression score [3.1, 3.2]. Further research
(1996-2013) focused on the large, but largely untapped, reservoir of
published data that can be used for reference purposes when assessing
individual cases: Crawford et al developed methods (mainly based on
regression, a statistical technique for estimating the relationships among
variables) that allow clinicians to make use of such data [3.3].
Other aspects of the research (1996-2013) focused on the development of
sound inferential methods. Crawford et al developed a large suite of
novel, statistically sound and convenient quantitative methods to enable
psychologists and medical practitioners to draw inferences over the
performance of an individual patient [3.4-3.6]. These methods successfully
address a number of problems clinicians may encounter, including the often
small size of reference samples and resulting systematic bias in favour of
overestimating the abnormality of scores [3.4]; the need to compare a
case's profile of test results rather than just single scores
[3.5, 3.6]; and the increasing awareness that clinicians should complement
point estimates of a patient's standing on a given scale with estimates of
the interval or stretch on the scale which the patient falls into [3.3].
Many of the quantitative methods Crawford et al developed are complex,
and there was a risk that they would not actually be used in practice. To
overcome this problem, Crawford made it an integral feature of his
research programme to create tailored computer programs that implement a
variety of psychometric and statistical methods for use in clinical
research and practice. These programs are specially designed to be
user-friendly for practitioners and made freely available to them [e.g.,
3.3 to 3.6]. The URL of the main web page for these programmes is http://homepages.abdn.ac.uk/j.crawford/pages/dept/psychom.htm.
References to the research
[3.1] Crawford, JR, Henry, JD, Crombie, C, and Taylor, EP. (2001).
Normative data for the HADS from a large non-clinical sample. British
Journal of Clinical Psychology (BJCP), 40, 429-434.
This paper (and the next) is one of a series of four papers published
in BJCP on assessment of anxiety and depression. Of more than 550 papers
published in BJCP since 2000, these papers are ranked 1, 2, 3, & 4
for citation impact in Web of Science (WoS). The above was named the
most influential article published in the period 2000-2005.
[3.2] Crawford, JR & Henry, JD (2003). The Depression Anxiety Stress
Scales: Normative data and latent structure in a large non-clinical
sample. British Journal of Clinical Psychology, 42, 111-131.
[3.3] Crawford, JR, & Garthwaite, PH. (2007). Comparing patients'
predicted test scores from a regression equation with their obtained
scores: a significance test and point estimate of abnormality with
accompanying confidence limits. Neuropsychology, 20, 259-271.
One of a series of four papers (1998-2012; three of which are
published in APA journals) developing inferential methods for the use of
regression in the individual case.
[3.4] Crawford, JR, & Howell, DC. (1998). Comparing an individual's
test score against norms derived from small samples. The Clinical
Neuropsychologist, 12, 482-486.
A foundational paper for this approach: the method has now largely
replaced the previously widespread practice of using z scores when
comparing a case to a reference sample (cited > 325 times in WoS as
of June 2013).
[3.5] Crawford, JR. & Garthwaite, PH. (2005). Testing for suspected
impairments and dissociations in single-case studies in neuropsychology:
Evaluation of alternatives using Monte Carlo simulations and revised tests
for dissociations. Neuropsychology, 19, 318-331.
This paper provided a sound classical statistical solution to the
problem of testing for a difference between a patient's scores on two
measures (e.g, verbal versus spatial memory scores etc).
[3.6] Crawford, JR, Garthwaite, PH, & Gault, CB. (2007). Estimating
the percentage of the population with abnormally low scores (or abnormally
large score differences) on standardized neuropsychological test
batteries: A generic method with applications. Neuropsychology,
This paper addressed the problem of estimating base rates of low
scores and large differences when using multiple tests. It has since
been used by the group and independent researchers to provide base rate
data for a wide number of psychological test batteries including the
HVLT-R, WJ-III, WAIS-IV, WISC-IV, D-KEFS, and RBANS. Moreover a number
of independent studies have evaluated the method for use in clinical
practice and report that it is sound.
Details of the impact
The research described has had direct impacts on clinical practice in the
UK and abroad, the development of commercially available test batteries,
and the training of neuropsychologists. More than 70 user-friendly
computer programs for clinicians developed by Crawford on the basis of his
research [e.g., 3.3 -3.6] have been made available free of charge on the
author's website. Crawford received at least 800 emails from users between
2008 and July 2013, enquiring about his quantitative methods and computer
programs, including numerous unsolicited comments testifying to their
clinical use, such as "Re-visiting your site to download .exe's to my new
computer. Thanks for all the stunning work. Most helpful"; and "I have
been finding your website and resources absolutely fantastic, and as a
clinician have recommended them to others. Thanks for all your wonderful
work, we appreciate it down under!"; and "I just wanted to take a minute
to tell you how much I appreciate the contribution you have made to the
field with your work on statistical analyses of psychometric change,
especially as it applies to neuropsychology" (representative email log
available on request — [5.7]).
The methods developed have also been used in clinical practice in the
following ways. Since 2008, the generic methods have been used in at least
500 published clinical case studies, primarily in North America and
Europe. As only a very small fraction of cases seen clinically warrant a
subsequent write-up as a case report, it is safe to assume that the
methods have been used with many more clinical cases. Furthermore, many of
the methods have been tailored to specific psychological test batteries
which would not routinely find their way into single case studies, but are
very widely used in clinical practice. These include the D-KEFS
(Delis-Kaplan Executive Function System); the WAIS-IV (Wechsler Adult
Intelligence Scale, Fourth Edition); and the RBANS (Repeatable Battery for
the Assessment of Neuropsychological Status). The take-up of the normative
data provided by the research is hard to quantify precisely, but again,
the widespread use of these data in clinically oriented research papers is
a strong indication that it is widely used. Google Scholar records over
1900 citations (as of June 2013) to the normative data for self-report
Major clinical textbooks / training manuals on assessment in clinical
neuropsychology [e.g., 5.a] provide extensive coverage of this group's
methods. Given the central role of these textbooks in clinical
neuropsychology training and practice, and their practical orientation,
these endorsements will have led to the use of the methods in clinical
As a result of his research expertise, Crawford has been called upon to
act as statistical/clinical consultant and/or author for most of the major
psychological tests used routinely in the UK. Recent examples include the
above-mentioned WAIS-IVUK, 2009; the Wechsler Memory Scale —
Fourth Edition (2009); the Rivermead Behavioural Memory Test — 3rd
Edition (2009); the Test of Premorbid Functioning (2011); The Functional
Living Scales (2012); and the Spot-the-Word Test Second Edition (2012).
These tests are used daily in clinical practice, and as they are
commercially available, the work carried out by Crawford et al has also
led to commercial impact.
Eminent clinicians in the UK and abroad have confirmed the significance
of Crawford's work for clinical practice. From the US, Gordon Chelune,
Professor of Neurology at the University of Utah, reports (March 2013)
[5.8] that the pioneering emphasis of Crawford's work has helped change
neuropsychological practice as well as the way neuropsychology is taught.
He highlights the fact that a publisher has asked him to write a book for
clinicians on the application of Crawford's methods. Jonathan Evans,
Professor of Applied Neuropsychology at the University of Glasgow,
confirms that Crawford's work has had a "very significant impact" on the
clinical neuropsychology profession and has "impacted directly on quality
of care for people with neurological and psychological conditions in the
UK and around the world" (March 2013) [ 5.9].
Since 2008, Crawford has led a number of training and continuous
professional development (CPD) events for clinical neuropsychologists and
similar practitioners based on his research expertise. These have included
annual CPD workshops on psychological assessment for the British
Psychological Society, which attract around 25 participants each year.
Feedback collected since 2009 has been extremely positive, with one
delegate speaking for many when s/he wrote, "I'm not good at stats and
quantitative issues but I actually understood this. Found it really useful
and thought provoking in terms of rethinking how I use tests and interpret
results." Another delegate wrote "Superb! I wish I had been taught this in
my clinical training". Other CPD events since 2008 have included an annual
post-qualification course for clinical neuropsychologists and
psychologists at the University of Glasgow. Delegates since 2010 (around
20 per course) have rated the event very highly at 4.7 out of a possible
5. Crawford also ran CPD workshops for the International
Neuropsychological Society (Oslo, June 2012), and the Australian Society
for the Study of Brain Impairment (Hobart, May 2013). These attracted over
Claimed impact as defined by REF guidance: clinical guidelines have
changed; professional standards, guidelines and training have been
influenced by research; professionals have used research findings in
conducting their research.
Sources to corroborate the impact
[5.1] Lezak, MD, Howieson, DB, Bigler, ED, & Tranel, D (2012).
Neuropsychological Assessment (5th ed.). New York: Oxford University
This is the standard reference work for clinical neuropsychological
assessment. Reference is made to 21 of Crawford's first author papers.
This count exceeds that of any other European psychologist.
[5.2] Atzeni, T. (2009). Statistiques appliquées aux études de cas
unique: méthodes usuelles et alternatives. Revue de Neuropsychologie
Neurosciences Cognitives et Cliniques, 1, 343-351.
This (French) review of how to make inferences concerning the
performance of a single case is, in essence, solely concerned with
Crawford and colleagues' methods; eight of the ten equations presented
are those developed by Crawford and colleagues.
[5.3] Balboni, G, & Cubelli, R. (2011). How to use psychological
tests for functional diagnosis: the case of assessment of learning
disabilities. Advances in Learning and Behavioral Disabilities,
This (Italian) guide to assessment shows that Crawford's methods are
now also having an impact in the area of learning disabilities. It
recommends five of Crawford and colleagues' methods.
[5.4] Brooks, BL, Strauss, E, Sherman, EMS, Iverson, GL, & Slick, DJ.
(2009). Developments in neuropsychological assessment: Refining
psychometric and clinical interpretive methods. Canadian Psychology,
This review provides further evidence of the impact of Crawford's work
on assessment in clinical practice. It recommends (and illustrates the
use of) five of Crawford and colleagues' methods.
[5.5] Hanson, RK, Lloyd, CD, Helmus, L, & Thornton, D. (2012).
Developing non-arbitrary metrics for risk communication: percentile ranks
for the Static-99/R and Static-2002/R sexual offender risk tools. International
Journal of Forensic Mental Health, 11, 9-23.
This recent (Canadian) paper illustrates that Crawford and colleagues'
methods are now also having an impact in the forensic area (the methods
were used with risk assessment tools).
[5.6] McIntosh, RD, & Brooks, JL. (2011). Current tests and trends in
single-case neuropsychology. Cortex, 47, 1151-1159.
This review is focused almost exclusively on reviewing and
recommending Crawford and colleagues' single case methods; it notes that
"current practice has been shaped considerably by Crawford and
colleagues' statistical refinements over the past 12 years" (p.1151) and
notes that "Crawford and colleagues' tests are now the tests of choice
for single-case comparisons" (p. 1155). It cites 15 of Crawford's
[5.7] Representative log of unsolicited feedback and comments on the
freely available computer programmes developed by Professor Crawford.
[5.8] Testimonial provided by Professor of Neurology, Department of
Neurology, University of Utah School of Medicine.
[5.9] Testimonial provided by Professor of Applied Neuropsychology,
University of Glasgow.
[5.10] Testimonial provided by Director of Critical Care and
Neurosciences Research, Murdoch Childrens Research Institute, Australia.