Critical evaluation of evidence in medicine and epidemiology: impacts on higher education, the NHS and public understanding of science

Submitting Institutions

University College London,
Birkbeck College

Unit of Assessment

Biological Sciences

Summary Impact Type

Health

Research Subject Area(s)

Physical Sciences: Other Physical Sciences
Biological Sciences: Biochemistry and Cell Biology
Medical and Health Sciences: Neurosciences


Download original

PDF

Summary of the impact

Professor of Pharmacology David Colquhoun's extensive work on receptor mechanisms and ion channel function has armed him with the pharmacological, statistical, and analytical skills essential to the sophisticated evaluation of evidence in the medical sciences. Over the past ten years, he has applied these skills to assess the validity of public policies, especially on the use of alternative therapies as clinical treatments. Having built up evidence-based arguments that alternative therapies, such as homeopathy and reflexology, are ineffective at best and dangerous at worst, he was able to achieve the nation-wide closure of university courses in these subjects. He has also ensured that the homeopathy information available on the NHS Choices website is not misleading. Through his widely read blog, DC's Improbable Science, Colquhoun educates the public in skills of evidence evaluation.

Underpinning research

David Colquhoun has spent his life studying the mechanisms of how drugs act on receptors, making several key contributions to the field during his 41 years at UCL. His early theoretical work with the statistician Alan G. Hawkes, and experimental work with Bert Sakmann, led to the first solution of the classical pharmacological problem of measuring separately the affinity and efficacy of an agonist [1]. In 1996, drawing on the earlier work of Hawkes and Jalali, Colquhoun and Hawkes were able to obtain the distributions of apparent open and shut times (with exact allowance for missed events) of single-ion channels, which were necessary to apply the existing theory of single channel behaviour to experimental data recorded by apparatus that is incapable of detecting the shortest ion channel events [2]. The application of maximum likelihood fitting in this paper has been the basis of subsequent experimental work, and the distributions of apparent open and shut times are often referred to as HJC (Hawkes, Jalali, Colquhoun) distributions [3]. In working with Hawkes, Colquhoun developed great expertise in statistics, even writing a book on it (Lectures on Biostatistics by David Colquhoun, published by Oxford University Press). This statistical knowledge is critical for the proper analysis of all types of data, and allows Colquhoun to assess the results of anything from clinical trials to epidemiological studies, with a trained eye.

Colquhoun has applied the theory of Markov processes to ion channel dynamics to characterise several features of ion channel behaviour, such as open times, shut times, latency and length of activation, in response to a pulse of agonist concentration or voltage [4]. Colquhoun's application of probability theory in his work on ion channels has informed his thinking about seemingly unrelated topics. For example, in 2009, he applied Markov queuing theory to hospital bed occupancy, explaining to the general public why some queues for NHS beds are unavoidable and not necessarily indicative of insufficient resources or bad management [5].

In 2008, in collaboration with Lucia Sivilotti at UCL, Colquhoun uncovered the mechanistic reason for why partial agonists never evoke as large a response as full agonists, despite binding to as many receptors [6]. They found that the response of nicotinic receptors to partial agonists is limited by an early conformational change in the receptor that takes place before the channel opens, debunking the existing theory that a difference in the open-shut transition is responsible for the different effects of partial and full agonists. This discovery is likely to influence the design of partial agonists for therapeutic use. Colquhoun's lifelong research into the way that drugs interact with receptors thus gives him unique expertise in assessing the proposed pharmacological effects of alternative medicines, such as those used in homeopathy.

References to the research

[1] Colquhoun D. Binding, gating, affinity and efficacy: the interpretation of structure-activity relationships for agonists and of the effects of mutating receptors. Br J Pharmacol. 1998 Nov;125(5):924-47. http://dx.doi.org/10.1038/sj.bjp.0702164

 
 
 
 

[2] Colquhoun D, Hawkes AG, Srodzinski K. Joint distributions of apparent open times and shut times of single ion channels and the maximum likelihood fitting of mechanisms. Phil Trans R Soc London A. 1996 Nov;354:2555-90. http://dx.doi.org/ 10.1098/rsta.1996.0115

 
 

[3] Colquhoun D, Hatton CJ, Hawkes AG. The quality of maximum likelihood estimates of ion channel rate constants. J Physiol. 2003 Mar 15;547(Pt 3):699-728. http://dx.doi.org/10.1113/jphysiol.2002.034165

 
 
 
 

[4] Colquhoun D, Hawkes AG, Merlushkin A, Edmonds B. Properties of single ion channel currents elicited by a pulse of agonist concentration or voltage. Phil Trans R Soc London A. 1997 Jan;355:1743-86. http://dx.doi.org/10.1098/rsta.1997.0090

 
 

[5] Blog post by Colquhoun (2009): "Queuing for beds, Andrei Andreevich Markov, and why I still love the NHS" http://www.dcscience.net/?p=2369

[6] Lape R, Colquhoun D, Sivilotti LG. On the nature of partial agonism in the nicotinic receptor superfamily. Nature. 2008 Aug 7;454(7205):722-7. http://dx.doi.org/10.1038/nature07139

 
 
 
 

Example of grant: 2005-10 MRC Programme Grant to L. Sivilotti, D. Colquhoun and T. Smart Molecular neuroscience of glycine receptors (£1,260,990) G0400869

Details of the impact

1) Closing university courses in alternative therapies

The principles behind alternative therapies, such as reflexology and homeopathy, are not derived from scientific experimentation and analysis. Consequently, their teaching is neither scientifically informative nor clinically relevant. There are severe disadvantages to allowing universities to offer degrees in these subjects, including: (i) the representation of these subjects as science; (ii) the effect of promoting these therapies as forms of medicine; (iii) the increased number of "qualified" individuals who believe in — and profit from — these non-medical and sometimes dangerous treatments; and (iv) government spending on useless degrees.

Through critical investigation of these degree courses, drawing on his knowledge of pharmacology and statistics, Colquhoun produced strong evidence-based arguments to the Information Commissioner and an Information Tribunal against the virtues of offering BSc and masters courses in these topics. This effort, documented on his blog and reported in the media [A.i-v], led to the widespread closure of such courses in universities across the UK. In 2012, the Telegraph reported:
"The number of bachelor and masters degrees in subjects such as reflexology, aromatherapy, acupuncture and homoeopathy has halved since 2007, from more than 40 to 21. Many of the surviving courses are under review."
"The closures are partly the result of a campaign led by Dr David Colquhoun, professor of pharmacology at University College London..."

Degrees in naturopathy, reflexology and aromatherapy have now all vanished from Britain's universities. It is encouraging that prospective students now appear to be returning to "traditional" degrees such as physics and chemistry [A.vi]. Thus, it is already clear that the closure of university courses in alternative therapies is benefitting education. The stricter policy on what subjects should be taught at degree level means funding can be spent on more important areas of education.

2) Maintaining accurate NHS guidance on alternative therapies

The effectiveness of alternative therapies, such as homeopathy, chiropractic and acupuncture, is not supported by evidence [B.i-iii]. At best, these therapies simply do not work beyond a placebo effect, and their use in place of evidence-based medicine can therefore damage patients' health while incurring unnecessary financial costs. For these therapies to be portrayed to the general public as a serious treatment option is economically damaging and clinically counter-productive.

The Department of Health was therefore considered to have "failed the general public" (David Mattin, ex-NHS Choices editor) when, in 2012, it prevented accurate information about homeopathy from appearing on the NHS Choices website. Specifically, a new page on homeopathy appeared, replacing the previous statement that "homeopathy is not part of mainstream medicine". The new page lent unjustified credibility to homeopathy and, as documented on his blog and in the media, Colquhoun campaigned heavily for the restoration of accurate information [C.i-iii]. The campaign was quickly successful, with the NHS Choices website now clearly stating:
"A 2010 House of Commons Science and Technology Committee report on homeopathy said that homeopathic remedies perform no better than placebos, and that the principles on which homeopathy is based are 'scientifically implausible'. This is also the view of the Chief Medical Officer, Professor Dame Sally Davies." [C.iv]

A similar story is emerging around the NICE guidelines for low back pain, which currently recommend manual therapy (including chiropractic) and acupuncture — techniques which are now thought to be ineffective [B.ii-iii]. NICE received a huge backlash from the medical and scientific community, which has been documented and further publicised by Colquhoun [D.i-iii]. This effort contributed to persuading NICE to reconsider its back pain guidance (in progress now).

3) What constitutes evidence? Impact on public awareness

Colquhoun's blog also encourages the public to critically assess evidence that is presented to them. This is an important skill in a media culture, where scientific findings are inflated and oversimplified for the sake of a headline. For instance, in 2008, the Telegraph told us that a "Sausage a day can increase bowel cancer risk". This headline was based on a 2007 report from the World Cancer Research Fund and, although the epidemiological research within the report was performed well, the conclusions were arguably too strong given the evidence. By explaining the concepts of causality, randomisation and statistical significance, Colquhoun's blog posts on the subject over five years have done more than simply point out where the media have gone wrong: they arm readers with the tools they need to evaluate this type of evidence themselves [E.i-iii].

Colquhoun has written 367 blog posts which have generated 6,941 comments (all figures in this paragraph to end July 2013). He almost always replies to comments, maintaining an interactive relationship with the public, both on his blog and in newspapers. The most read post is "Acupuncture is a theatrical placebo: the end of a myth" (ref B.iii above), with over 21,000 page loads so far. The blog has had 3.3 million hits to date (605,000 in the last year), and there have been hits from almost every country in the world. The blog was featured on the Times 100 best blogs list in 2009 [F.i]. The number of page loads per day rarely falls below 1,000, but straight after a new post it rises to 3-7,000 hits per day. Colquhoun's appearances in the media (see H below) vastly increase interest in his blog. The record number of hits on one day was 24,305, which occurred on 21 April 2013, the day that The Observer published an article naming Colquhoun as a Rational Hero [F.ii].

As a result of the blog's popularity, Colquhoun has been invited to make many media appearances and write newspaper articles [G], which, in turn, encourage a wider blog readership. He also has over 8,000 followers on Twitter (@david_colquhoun). This medium provides a rather different, and highly interactive, audience for public engagement compared with the blog.

Sources to corroborate the impact

[A] Closure of university courses in alternative therapies

i. Science degrees without science. (2007). David Colquhoun, Nature, 466:373:374.

ii. Is a degree in homeopathy a sick joke? (2009). Richard Tomkins, The Financial Times.

iii. Lie back and relax: reflexology and aromatherapy degrees are dropped. (2012). Michael Hanlon, The Telegraph. http://www.telegraph.co.uk/education/universityeducation/degree-courses/8989183/Lie-back-and-relax-reflexology-and-aromatherapy-degrees-are-dropped.html

iv. Complementary medicine courses in universities: how I beat the varsity quacks. (2012). David Colquhoun, The Telegraph. http://www.telegraph.co.uk/science/science-news/9051103/Complementary-medicine-courses-in-universities-how-I-beat-the-varsity-quacks.html

v. The demise of quackademia. Progress in the last 5 years leaves Michael Driscoll and Geoffrey Petts isolated. (2012). Blog post on DC's Improbable Science.
http://www.dcscience.net/?p=4900

vi. Despite overall decline, physics applications are on the rise. (2012). Institute of Physics.
http://www.iop.org/news/12/feb/page_53743.html

[B] Ineffectiveness of alternative therapies, as discussed on DC's Improbable Science

i. Medicines that contain no medicine and other follies. (2008). National Health Executive magazine. http://www.dcscience.net/?p=675

ii. Simon Singh on chiropractic: Beware the spinal trap. (2009). http://www.dcscience.net/?p=1980

iii. Acupuncture is a theatrical placebo: the end of a myth. (2013). In Anesthesia & Analgesia. http://www.dcscience.net/?p=6060

[C] Maintaining accurate NHS Choices information on homeopathy

i. Policy-based evidence. Department of Health and Prince's Foundation censor accurate information about magic medicines. (2013). Blog post on DC's Improbable Science.
http://www.dcscience.net/?p=5778

ii. Homeopathy guidance triggers lobbying row. (2013). Sarah Boseley, The Guardian.

iii. Homeopathy charity run by Charles `cowed civil servants' into supporting the therapy. (2013). Tamara Cohen, Mail Online.

iv. Current NHS Choices web page on homeopathy:
http://www.nhs.uk/Conditions/homeopathy/Pages/Introduction.aspx?a=1

[D] Backlash against NICE recommendation of acupuncture and chiropractic for low back pain

i. NICE falls for Bait and Switch by acupuncturists and chiropractors: it has let down the public and itself. (May 2009). http://www.dcscience.net/?p=1516

ii. NICE fiasco, part 2. Rawlins should withdraw guidance and start again. (May 2009). http://www.dcscience.net/?p=1542

iii. The NICE fiasco, part 3. Too many vested interests, not enough honesty. (June 2009).http://www.dcscience.net/?p=1593

[E] What constitutes evidence? Educating the public through DC's Improbable Science. These three posts explain the problem of establishing causality, through epidemiological examples.

i. Diet and health. What can you believe: or does bacon kill you? (2009).
http://www.dcscience.net/?p=1435

ii. How big is the risk from eating red meat now? An update. (2012).
http://www.dcscience.net/?p=5164

iii. Another update. Red meat doesn't kill you, but the spin is fascinating. (2013).
http://www.dcscience.net/?p=5935

[F] Readership of DC's Improbable Science

i The blog was featured on the Times 100 best blogs list in 2009 http://thetim.es/18PmYOw

ii David Colquhoun, Twitter-addicted scourge of scientific quackery http://gu.com/p/3f63y/tw

[G] Media appearances and public engagement. The YouTube channel DrSceptic has 18 videos, including many of Colquhoun's TV interviews. It has had a collective 30,558 views.
http://www.youtube.com/DrSceptic; Colquhoun has appeared on Channel 4 News, BBC Breakfast, BBC World TV news, and the BBC News Channel; he has been a guest on Radio 5 Live, local radio stations, and on Radio 4's Today Programme three times. Selected newspaper articles: