Critical evaluation of evidence in medicine and epidemiology: impacts on higher education, the NHS and public understanding of science
Submitting Institutions
University College London,
Birkbeck CollegeUnit of Assessment
Biological SciencesSummary Impact Type
HealthResearch Subject Area(s)
Physical Sciences: Other Physical Sciences
Biological Sciences: Biochemistry and Cell Biology
Medical and Health Sciences: Neurosciences
Summary of the impact
Professor of Pharmacology David Colquhoun's extensive work on receptor
mechanisms and ion channel function has armed him with the
pharmacological, statistical, and analytical skills essential to the
sophisticated evaluation of evidence in the medical sciences. Over the
past ten years, he has applied these skills to assess the validity of
public policies, especially on the use of alternative therapies as
clinical treatments. Having built up evidence-based arguments that
alternative therapies, such as homeopathy and reflexology, are ineffective
at best and dangerous at worst, he was able to achieve the nation-wide
closure of university courses in these subjects. He has also ensured that
the homeopathy information available on the NHS Choices website is not
misleading. Through his widely read blog, DC's Improbable Science,
Colquhoun educates the public in skills of evidence evaluation.
Underpinning research
David Colquhoun has spent his life studying the mechanisms of how drugs
act on receptors, making several key contributions to the field during his
41 years at UCL. His early theoretical work with the statistician Alan G.
Hawkes, and experimental work with Bert Sakmann, led to the first solution
of the classical pharmacological problem of measuring separately the
affinity and efficacy of an agonist [1]. In 1996, drawing on the
earlier work of Hawkes and Jalali, Colquhoun and Hawkes were able to
obtain the distributions of apparent open and shut times (with exact
allowance for missed events) of single-ion channels, which were necessary
to apply the existing theory of single channel behaviour to experimental
data recorded by apparatus that is incapable of detecting the shortest ion
channel events [2]. The application of maximum likelihood fitting
in this paper has been the basis of subsequent experimental work, and the
distributions of apparent open and shut times are often referred to as HJC
(Hawkes, Jalali, Colquhoun) distributions [3]. In working with
Hawkes, Colquhoun developed great expertise in statistics, even writing a
book on it (Lectures on Biostatistics by David Colquhoun, published
by Oxford University Press). This statistical knowledge is critical for
the proper analysis of all types of data, and allows Colquhoun to assess
the results of anything from clinical trials to epidemiological studies,
with a trained eye.
Colquhoun has applied the theory of Markov processes to ion channel
dynamics to characterise several features of ion channel behaviour, such
as open times, shut times, latency and length of activation, in response
to a pulse of agonist concentration or voltage [4]. Colquhoun's
application of probability theory in his work on ion channels has informed
his thinking about seemingly unrelated topics. For example, in 2009, he
applied Markov queuing theory to hospital bed occupancy, explaining to the
general public why some queues for NHS beds are unavoidable and not
necessarily indicative of insufficient resources or bad management [5].
In 2008, in collaboration with Lucia Sivilotti at UCL, Colquhoun
uncovered the mechanistic reason for why partial agonists never evoke as
large a response as full agonists, despite binding to as many receptors [6].
They found that the response of nicotinic receptors to partial agonists is
limited by an early conformational change in the receptor that takes place
before the channel opens, debunking the existing theory that a
difference in the open-shut transition is responsible for the different
effects of partial and full agonists. This discovery is likely to
influence the design of partial agonists for therapeutic use. Colquhoun's
lifelong research into the way that drugs interact with receptors thus
gives him unique expertise in assessing the proposed pharmacological
effects of alternative medicines, such as those used in homeopathy.
References to the research
[1] Colquhoun D. Binding, gating, affinity and efficacy: the
interpretation of structure-activity relationships for agonists and of the
effects of mutating receptors. Br J Pharmacol. 1998 Nov;125(5):924-47. http://dx.doi.org/10.1038/sj.bjp.0702164
[2] Colquhoun D, Hawkes AG, Srodzinski K. Joint distributions of apparent
open times and shut times of single ion channels and the maximum
likelihood fitting of mechanisms. Phil Trans R Soc London A. 1996
Nov;354:2555-90. http://dx.doi.org/
10.1098/rsta.1996.0115
[4] Colquhoun D, Hawkes AG, Merlushkin A, Edmonds B. Properties of single
ion channel currents elicited by a pulse of agonist concentration or
voltage. Phil Trans R Soc London A. 1997 Jan;355:1743-86. http://dx.doi.org/10.1098/rsta.1997.0090
Example of grant: 2005-10 MRC Programme Grant to L.
Sivilotti, D. Colquhoun and T. Smart Molecular neuroscience of glycine
receptors (£1,260,990) G0400869
Details of the impact
1) Closing university courses in alternative therapies
The principles behind alternative therapies, such as reflexology and
homeopathy, are not derived from scientific experimentation and analysis.
Consequently, their teaching is neither scientifically informative nor
clinically relevant. There are severe disadvantages to allowing
universities to offer degrees in these subjects, including: (i) the
representation of these subjects as science; (ii) the effect of promoting
these therapies as forms of medicine; (iii) the increased number of
"qualified" individuals who believe in — and profit from — these
non-medical and sometimes dangerous treatments; and (iv) government
spending on useless degrees.
Through critical investigation of these degree courses, drawing on his
knowledge of pharmacology and statistics, Colquhoun produced strong
evidence-based arguments to the Information Commissioner and an
Information Tribunal against the virtues of offering BSc and masters
courses in these topics. This effort, documented on his blog and reported
in the media [A.i-v], led to the widespread closure of such
courses in universities across the UK. In 2012, the Telegraph reported:
"The number of bachelor and masters degrees in subjects such as
reflexology, aromatherapy, acupuncture and homoeopathy has halved since
2007, from more than 40 to 21. Many of the surviving courses are under
review."
"The closures are partly the result of a campaign led by Dr David
Colquhoun, professor of pharmacology at University College London..."
Degrees in naturopathy, reflexology and aromatherapy have now all
vanished from Britain's universities. It is encouraging that prospective
students now appear to be returning to "traditional" degrees such as
physics and chemistry [A.vi]. Thus, it is already clear that the
closure of university courses in alternative therapies is benefitting
education. The stricter policy on what subjects should be taught at degree
level means funding can be spent on more important areas of education.
2) Maintaining accurate NHS guidance on alternative therapies
The effectiveness of alternative therapies, such as homeopathy,
chiropractic and acupuncture, is not supported by evidence [B.i-iii].
At best, these therapies simply do not work beyond a placebo effect, and
their use in place of evidence-based medicine can therefore damage
patients' health while incurring unnecessary financial costs. For these
therapies to be portrayed to the general public as a serious treatment
option is economically damaging and clinically counter-productive.
The Department of Health was therefore considered to have "failed the
general public" (David Mattin, ex-NHS Choices editor) when, in 2012, it
prevented accurate information about homeopathy from appearing on the NHS
Choices website. Specifically, a new page on homeopathy appeared,
replacing the previous statement that "homeopathy is not part of
mainstream medicine". The new page lent unjustified credibility to
homeopathy and, as documented on his blog and in the media, Colquhoun
campaigned heavily for the restoration of accurate information [C.i-iii].
The campaign was quickly successful, with the NHS Choices website now
clearly stating:
"A 2010 House of Commons Science and Technology Committee report on
homeopathy said that homeopathic remedies perform no better than
placebos, and that the principles on which homeopathy is based are
'scientifically implausible'. This is also the view of the Chief Medical
Officer, Professor Dame Sally Davies." [C.iv]
A similar story is emerging around the NICE guidelines for low back pain,
which currently recommend manual therapy (including chiropractic) and
acupuncture — techniques which are now thought to be ineffective [B.ii-iii].
NICE received a huge backlash from the medical and scientific community,
which has been documented and further publicised by Colquhoun [D.i-iii].
This effort contributed to persuading NICE to reconsider its back pain
guidance (in progress now).
3) What constitutes evidence? Impact on public awareness
Colquhoun's blog also encourages the public to critically assess evidence
that is presented to them. This is an important skill in a media culture,
where scientific findings are inflated and oversimplified for the sake of
a headline. For instance, in 2008, the Telegraph told us that a "Sausage a
day can increase bowel cancer risk". This headline was based on a 2007
report from the World Cancer Research Fund and, although the
epidemiological research within the report was performed well, the
conclusions were arguably too strong given the evidence. By explaining the
concepts of causality, randomisation and statistical significance,
Colquhoun's blog posts on the subject over five years have done more than
simply point out where the media have gone wrong: they arm readers with
the tools they need to evaluate this type of evidence themselves [E.i-iii].
Colquhoun has written 367 blog posts which have generated 6,941 comments
(all figures in this paragraph to end July 2013). He almost always replies
to comments, maintaining an interactive relationship with the public, both
on his blog and in newspapers. The most read post is "Acupuncture is a
theatrical placebo: the end of a myth" (ref B.iii above), with over 21,000
page loads so far. The blog has had 3.3 million hits to date
(605,000 in the last year), and there have been hits from almost every
country in the world. The blog was featured on the Times 100 best blogs
list in 2009 [F.i]. The number of page loads per day rarely falls
below 1,000, but straight after a new post it rises to 3-7,000 hits per
day. Colquhoun's appearances in the media (see H below) vastly increase
interest in his blog. The record number of hits on one day was 24,305,
which occurred on 21 April 2013, the day that The Observer published an
article naming Colquhoun as a Rational Hero [F.ii].
As a result of the blog's popularity, Colquhoun has been invited to make
many media appearances and write newspaper articles [G], which, in
turn, encourage a wider blog readership. He also has over 8,000 followers
on Twitter (@david_colquhoun). This medium provides a rather different,
and highly interactive, audience for public engagement compared with the
blog.
Sources to corroborate the impact
[A] Closure of university courses in alternative therapies
i. Science degrees without science. (2007). David Colquhoun, Nature,
466:373:374.
ii. Is a degree in homeopathy a sick joke? (2009). Richard
Tomkins, The Financial Times.
iii. Lie back and relax: reflexology and aromatherapy degrees are
dropped. (2012). Michael Hanlon, The Telegraph. http://www.telegraph.co.uk/education/universityeducation/degree-courses/8989183/Lie-back-and-relax-reflexology-and-aromatherapy-degrees-are-dropped.html
iv. Complementary medicine courses in universities: how I beat the
varsity quacks. (2012). David Colquhoun, The Telegraph. http://www.telegraph.co.uk/science/science-news/9051103/Complementary-medicine-courses-in-universities-how-I-beat-the-varsity-quacks.html
v. The demise of quackademia. Progress in the last 5 years leaves
Michael Driscoll and Geoffrey Petts isolated. (2012). Blog post on
DC's Improbable Science.
http://www.dcscience.net/?p=4900
vi. Despite overall decline, physics applications are on the rise.
(2012). Institute of Physics.
http://www.iop.org/news/12/feb/page_53743.html
[B] Ineffectiveness of alternative therapies, as discussed on DC's
Improbable Science
i. Medicines that contain no medicine and other follies. (2008).
National Health Executive magazine. http://www.dcscience.net/?p=675
ii. Simon Singh on chiropractic: Beware the spinal trap. (2009).
http://www.dcscience.net/?p=1980
iii. Acupuncture is a theatrical placebo: the end of a myth.
(2013). In Anesthesia & Analgesia. http://www.dcscience.net/?p=6060
[C] Maintaining accurate NHS Choices information on homeopathy
i. Policy-based evidence. Department of Health and Prince's
Foundation censor accurate information about magic medicines.
(2013). Blog post on DC's Improbable Science.
http://www.dcscience.net/?p=5778
ii. Homeopathy guidance triggers lobbying row. (2013). Sarah
Boseley, The Guardian.
iii. Homeopathy charity run by Charles `cowed civil servants' into
supporting the therapy. (2013). Tamara Cohen, Mail Online.
iv. Current NHS Choices web page on homeopathy:
http://www.nhs.uk/Conditions/homeopathy/Pages/Introduction.aspx?a=1
[D] Backlash against NICE recommendation of acupuncture and
chiropractic for low back pain
i. NICE falls for Bait and Switch by acupuncturists and
chiropractors: it has let down the public and itself. (May 2009). http://www.dcscience.net/?p=1516
ii. NICE fiasco, part 2. Rawlins should withdraw guidance and start
again. (May 2009). http://www.dcscience.net/?p=1542
iii. The NICE fiasco, part 3. Too many vested interests, not enough
honesty. (June 2009).http://www.dcscience.net/?p=1593
[E] What constitutes evidence? Educating the public through DC's
Improbable Science. These three posts explain the problem of
establishing causality, through epidemiological examples.
i. Diet and health. What can you believe: or does bacon kill you?
(2009).
http://www.dcscience.net/?p=1435
ii. How big is the risk from eating red meat now? An update.
(2012).
http://www.dcscience.net/?p=5164
iii. Another update. Red meat doesn't kill you, but the spin is
fascinating. (2013).
http://www.dcscience.net/?p=5935
[F] Readership of DC's Improbable Science
i The blog was featured on the Times 100 best blogs list in 2009 http://thetim.es/18PmYOw
ii David Colquhoun, Twitter-addicted scourge of scientific quackery http://gu.com/p/3f63y/tw
[G] Media appearances and public engagement. The YouTube channel
DrSceptic has 18 videos, including many of Colquhoun's TV interviews. It
has had a collective 30,558 views.
http://www.youtube.com/DrSceptic;
Colquhoun has appeared on Channel 4 News, BBC Breakfast, BBC World TV
news, and the BBC News Channel; he has been a guest on Radio 5 Live, local
radio stations, and on Radio 4's Today Programme three times. Selected
newspaper articles: