Summary of the impact

Garthwaite has collaborated with Professor John Crawford, a neuropsychologist at the University of Aberdeen, to develop novel statistical methods for drawing inferences on the performance of an individual patient. The methods have become the standard way of analysing single-patient studies in neuropsychology and are widely used by clinicians to compare individual patients with normative data.

The methods have also been implemented in easy-to-use software, freely accessible over the web, and have been linked to databases containing the results of large normative samples on psychological test batteries. They have been the focus of review papers for clinical practice that have strongly recommended their use.

Underpinning research

Paul Garthwaite has been Professor of Statistics at The Open University since 2000. Since then he has worked extensively with Professor John Crawford on single-patient data, publishing 24 papers on this subject since 2002. This account covers only a small part of their work.

Their early work [3.1] developed classical inferential methods for deficits; that is, cases in which a patient's measurements are so unusual that the patient cannot be part of the control population. This work gave point and interval estimates for the percentage of the normal population who would have a more extreme score than the score obtained by the patient. Assumptions underlying the inferences have been examined [3.4] and asymptotic approximations were developed [3.7] so that a classical hypothesis test could be constructed for dissociations [3.2], where a patient's measurements on two similar tasks are unusually different.

Forming an interval estimate for dissociation proved impossible using classical statistics, so Bayesian methods for inference about both deficits and dissociations were developed [3.3]. Bayesian and classical methods are equivalent for making inferences for deficits but Bayesian methods can give interval estimates for dissociations. Modifications to the Bayesian methods have been devised that give good frequentist properties, which is necessary if the methods are to be widely accepted by the neuropsychology community.

Often performance on a task is influenced by covariates such as a person's age and number of years of education, and methods have been developed that control covariates. The above methods work well with large samples but are designed to give accurate inferences with small samples, greatly increasing their usefulness. Methods have also been developed specifically to compare a patient's score with large databases [3.5, 3.6, 3.8]: complications can arise because a patient's score may be tied with many controls, in which case the usual method of breaking ties may give serious inaccuracy.

All the methods have been implemented in software that is freely available (for example, from Professor Crawford's website: http://homepages.abdn.ac.uk/j.crawford/pages/dept/) and is easy to use. The software can be run directly from the web without the need to download and save it, and a user need only type in simple summary data and the scores of the patient. Relevant programs also link to databases where the scores on common psychological tests from large control samples are stored. To use these programs a user need only specify a patient's scores and the scales on which they were measured.

The importance of this work to assessment of the individual patient has been further recognised by the award of a recent (2012-2015) MRC grant MR/J013838/1 (£480,000 FEC) for `Development of statistical methods for the analysis of single patient data'. Garthwaite is principal investigator on the grant, and Jones (The Open University) and Crawford are co-investigators.

References to the research

Details of the impact

Statistical methods for the analysis of single-patient data have largely been developed in the context of neuropsychology, so it is in this area where the impact of Garthwaite's research is greatest. Indeed, the results on deficit and dissociation found by Crawford, Garthwaite and others, and the associated software they have produced, have been used by a very large number of neuropsychologists.

1. Single-patient studies that have been reported in the literature provide the best documentation of the impact of Crawford and Garthwaite's methods. In 2012 their papers in the field received more than 140 citations and papers [3.1-3.5] have together been cited more than 500 times. Also, because only a very small fraction of patients seen clinically are of sufficient theoretical interest to warrant a subsequent write-up as a case report, it is safe to assume that the methods are being used on many more cases in practice.

The citations mainly arise from their methods being used to analyse data, rather than their work simply being discussed. Studies that have used their methods include papers in Science [5.11] and Brain [5.3, 5.6, 5.8]. Much of the work that uses the methods is aimed at influencing clinical practice. Recent examples include the work of McGibbon and Jansari [5.9], which raises the prospect of a standard test to diagnose accelerated long-term forgetting in a single clinical visit rather than requiring multiple visits. Similarly, Borchers et al. [5.4] propose guidelines for the diagnosis of optic ataxis. Their conclusions note that `[in] a first screening ... the lower C & G 0.05 threshold (13%) would be a good choice', where `C & G' refers to a method given in [3.2] that was used to determine the threshold.

2. Computer programs implementing the methods are available from the website of Crawford, who has received at least 400 emails from users between 2008 and July 2013 enquiring about these quantitative methods and programs. The emails (email logs available on request) include numerous unsolicited comments testifying to their clinical use, such as:

• `Re-visiting your site to download .exe's to my new computer. Thanks for all the stunning work. Most helpful.'
• `I have been finding your website and resources absolutely fantastic, and as a clinician have recommended them to others. Thanks for all your wonderful work, we appreciate it down under!'
• `I just wanted to take a minute to tell you how much I appreciate the contribution you have made to the field with your work on statistical analyses of psychometric change, especially as it applies to neuropsychology.'

3. Many reviews of methods for inference in the individual case have devoted considerable space to setting out the methods of Crawford, Garthwaite et al. and recommending their use [5.1, 5.2, 5.5, 5.7, 5.10]. The recommendations have been very positive. For example, McIntosh and Brooks [5.10] note that the methods developed by Crawford, Garthwaite and colleagues `are now the tests of choice for single-case comparisons' and that they `have been adopted enthusiastically by the neuropsychological community' (p. 1155).

4. The work is used by practising neuropsychologists to evaluate their patients and assess whether they have abnormal deficits or dissociations. Quantifying this uptake is hard. However, the methods are linked to important psychological test batteries, including the Wechsler Adult Intelligence Scale, the Delis-Kaplan executive function system and the Repeatable Battery for the assessment of neuropsychological data. These test batteries do not routinely find their way into single-patient studies, but they are commercially available and used daily by clinicians.

Given the ease with which the methods can be used, the large number of citations and the positive reviews and unsolicited comments, it is clear that the methods are well used in clinical practice.

Sources to corroborate the impact

Citation information comes from the ISI Web of Knowledge.

5.1 Atzeni, T. (2009) `Statistiques appliquées aux études de cas unique: méthodes usuelles et alternatives', Revue de Neuropsychologie Neurosciences Cognitives et Cliniques, vol. 1, pp. 343-51.

This (French) review of developments in how to make inferences concerning the performance of a single case is, in essence, solely concerned with Crawford, Garthwaite and colleagues' methods; seven of the ten equations presented are those developed by Crawford, Garthwaite and colleagues (two of the remaining three are basic definitional equations).

5.2 Balboni, G. and Cubelli, R. (2011) `How to use psychological tests for functional diagnosis: the case of assessment of learning disabilities', Advances in Learning and Behavioral Disabilities, vol. 24, pp. 79-92.

This (Italian) guide to assessment shows that Crawford and Garthwaite's methods are now also having an impact in the area of learning disabilities. It recommends (and illustrates the use of) four of Crawford, Garthwaite and colleagues' methods.

5.3 Bird, C.M., Castelli, F., Malik, O., Frith, U. and Husain, M. (2004) `The impact of extensive medial frontal lobe damage on `theory of mind' and cognition', Brain, vol. 127, pp. 914-28.

5.4 Borchers, S., Muller, L., Synofzik, M. and Himmelbach, M. (2013) `Guidelines and quality measures for the diagnosis of optic ataxia', Frontiers in Human Neuroscience, vol. 7, article 324.

5.5 Brooks, B.L., Strauss, E., Sherman, E.M.S., Iverson, G.L. and Slick, D.J. (2009) `Developments in neuropsychological assessment: Refining psychometric and clinical interpretive methods', Canadian Psychology, vol. 50, pp. 196-209.

This (Canadian) review provides further evidence of the impact of Crawford and colleagues' work on assessment in clinical practice. It recommends (and illustrates the use of) four of Crawford, Garthwaite and colleagues' methods.

5.6 Fotopoulou, A., Pernigo, S., Maeda, R., Rudd, A. and Kopelman, M.A. (2010) `Implicit awareness inanasognosia for hemiplegia: unconscious interference without conscious re-representation', Brain, vol. 133, pp. 3564-77.

5.7 Hanson, R.K., Lloyd, C.D., Helmus, L. and Thornton, D. (2012) `Developing non-arbitrary metrics for risk communication: percentile ranks for the Static-99/R and Static-2002/R sexual offender risk tools', International Journal of Forensic Mental Health, vol. 11, pp. 9-23.

This recent (Canadian) paper illustrates that Crawford, Garthwaite and colleagues' methods are now also having an impact in the forensic area: the methods were used to provide point and interval estimates for risk assessment tools

5.8 Maguire, A.E., Nannery, R. and Spiers, H.J. (2006) `Navigation around London by a taxi driver with bilateral hippocampal lesions', Brain, vol. 129, pp. 2894-907.

5.9 McGibbon, T. and Jansari, A.S. (2013) `Detecting the onset of accelerated long-term forgetting: Evidence from temporal lobe epilepsy', Neuropsychologia, vol. 51, pp. 114-22.

5.10 McIntosh, R.D. and Brooks, J.L. (2011) `Current tests and trends in single-case neuropsychology', Cortex, vol. 47, pp. 1151-9.

This recent (UK) review of single case methods is focused almost exclusively on reviewing and recommending Crawford and colleagues' methods. It cites 13 of Crawford, Garthwaite and colleagues' papers.

5.11 Thiebaut de Schotten, M., Urbanski, M., Duffau, H., Voue, E., Levy, R., Dubois, B. and Bartolomeo, P. (2005) `Direct evidence of parietal-frontal pathway subserving spatial awareness in humans', Science, vol. 309, pp. 2226-8.