Submitting InstitutionUniversity of Aberdeen
Unit of AssessmentPsychology, Psychiatry and Neuroscience
Summary Impact TypeSocietal
Research Subject Area(s)
Medical and Health Sciences: Clinical Sciences, Public Health and Health Services
Summary of the impact
Non-technical aspects of clinicians' performance, including cognitive
errors and lack of teamwork,
are a common cause of adverse events for patients. A series of studies at
the University of
Aberdeen identified the non-technical skills of operating theatre staff,
and developed behavioural
rating systems for their evaluation. The resulting skills frameworks for
surgeons (NOTSS) and scrub practitioners (SPLINTS) have been adopted and
these professional groups, across the UK and elsewhere in the world.
The resulting impact has been on healthcare professional standards,
guidelines and training.
Practitioners have used these research findings and tools in the conduct
of their work.
Adverse events for patients — where they are in some way damaged during
or as a result of
healthcare — are clearly situations which hospitals seek to minimise and
eradicate. Failures in
clinicians' non-technical skills (such as situation awareness, decision
making, leadership and
teamwork) resulting in human errors are a common cause of such events.
Recent concern in the
UK and the USA about unacceptable rates of surgical 'never events'
demonstrates that the
operating theatre remains a particularly susceptible domain for this type
As has been demonstrated in high-risk industries such as aviation, once
the required skills to
minimise human error have been identified, they can be improved through
structured training and
competence assessment programmes. Formative or summative assessments of
such skills can
be made by observing behaviours of individual practitioners at work (in
simulation or situ). Prior to
the University of Aberdeen work, no skill frameworks or assessment tools
for this purpose were
available for healthcare professionals — this research was designed to
address that deficit.
Professor Rhona Flin, Chair of Applied Psychology at the University of
Aberdeen since 1996, was
the Principal Investigator for three project teams based at the
University's School of Psychology,
identifying non-technical skills of operating theatre staff and designing
behavioural rating systems
for their evaluation. The research method that underpins this impact case
was based on her
earlier work with European airline pilots, followed by a set of
multidisciplinary research studies,
using task analyses, conducted at the University of Aberdeen (1999 - 2012). These projects
developed three skills frameworks with behaviour rating tools — Anaesthetists' Non-Technical
Skills System (ANTS) [3.1, 3.2], Non-Technical Skills for Surgeons (NOTSS)
[3.3, 3.4, 3.5], and
Scrub Practitioners' List of Intra-operative Non-Technical Skills
The research involved close collaboration with consultant anaesthetists,
surgeons and scrub
nurses from hospitals across Glasgow, Aberdeen, Dundee, Inverness and
Edinburgh. For each
project, an industrial psychologist was employed as the Research Fellow:
Dr Georgina Fletcher on
the ANTS project (1999-2003), Dr Steven Yule on the NOTSS project
(2004-2008), and Dr Lucy
Mitchell on the SPLINTS project (2008-2012).
In each project, the aim was to identify the key non-technical skills for
an individual practitioner
while he or she was working within a clinical team during the
intra-operative phase of anaesthesia
and surgery. The research methods used to identify the skill set included
literature reviews [3.3],
questionnaire surveys [3.2], observations of staff conducting anaesthesia
or surgery, critical
incident interviews with subject matter experts (consultant anaesthetists,
surgeons and senior
scrub nurses), and accident analyses. Refinement of the resulting data
into structured skill
frameworks with behavioural markers were produced from interviews and
focus groups with
subject matter experts, conducted in six hospitals across Scotland [3.4].
The skill sets and
associated behavioural rating systems were then tested by producing
videotaped re-enactments of
anaesthetic and surgical procedures which were shown to panels of
practitioners who used the
prototype rating tools to score observations of practitioners' cognitive
and teamwork skills.
Psychometric properties were found to be sufficiently robust [3.1, 3.5,
3.6] to justify usability testing
before releasing the tools.
References to the research
[3.1] Fletcher, G, Flin, R, McGeorge, P, Glavin, R, Maran, N, &
Patey, R. (2003). Anaesthetists'
Non-Technical Skills (ANTS): Evaluation of a behavioural marker system. British
Anaesthesia, 90 (5), 580 - 588. Reports the evaluation study for
the ANTS tool. Cited 221 times,
by clinicians using the tool or research studies e.g. in clinical
simulators, or by clinicians using the
same method to develop similar tools for other specialties.
[3.2] Flin, R, Fletcher, G, McGeorge, P, Sutherland, A & Patey, R.
(2003) Anaesthetists' attitudes
to teamwork and safety. Anaesthesia, 58, 233-242. This survey
was part of the initial task analysis
for ANTS. Cited 62 times.
[3.3] Yule, S, Flin, R, Maran, N & Paterson-Brown, S. (2006).
Non-technical skills for surgeons in
the operating room: A review of the literature. Surgery, 139,
140-149. The initial examination of the
role of non-technical skills in surgical performance. Cited 156 times.
[3.4] Yule, S, Flin, R, Paterson-Brown, S, & Maran, N. (2006).
Development of a rating system for
surgeons' non-technical skills. Medical Education, 40, 1089-1044.
The development process for
the first tool to rate individual surgeons' non-technical skills
(NOTSS). Cited 96 times.
[3.5] Yule, S, Flin, R, Maran, N, Rowley, D R, Youngson, G &
Paterson-Brown, S. (2008).
Surgeons' non-technical skills in the operating room: Reliability testing
of the NOTSS behaviour
rating system. World Journal of Surgery, 32, 548-556. The
evaluation study for the surgical
behaviour rating tool. Cited 70 times.
[3.6] Mitchell, L, Flin, R, Mitchell, J, Coutts, K. & Youngson, G.
(2012). Evaluation of the Scrub
Practitioners' List of Intraoperative Non-Technical Skills (SPLINTS)
system. International Journal of
Nursing Studies, 49, 201-211. Reports the evaluation of the
newest tool, for scrub practitioners.
Key Research Grants
• Amalberti et al 1998-2000, €750,000, EC DGVII `JARTEL' project (Flin
(AU) partner of NLR,
DLR, IMASSA, British Airways, Alitalia, DERA, SOFREAVIA). This was an
psychology research project (Framework 4) evaluating a non-technical
skills framework and
rating system (NOTECHS) for European airline pilots.
• Flin, Glavin, Maran. Anaesthetists' Non-Clinical Skills. Scottish
Council for Postgraduate
Medical and Dental Education. £156,090. 1999-2003.
• Flin, Paterson-Brown, Rowley & Maran. Surgeons' Non-Technical
Skills. Royal College of
Surgeons Edinburgh / NHS Education Scotland (NES). £124,000. 2003-2006.
• Flin, Yule & Paterson-Brown. Non-Technical Skills for Surgeons.
Royal College of
Surgeons Edinburgh. £46,000. 2006-2007.
• Flin & Mitchell. Non-Technical Skills for Scrub Nurses. NES.
• Flin, Davies, Davey et al. Scottish Patient Safety Research Network.
Council/Universities of Aberdeen, Dundee and St Andrews. Strategic
Development Grant. £1.5M. 2007-2012. One work package, based at the
Aberdeen, was devoted to non-technical skills research.
Details of the impact
This research has provided new training and assessment tools designed to
raise awareness of
human factors in patient safety and to enhance the non-technical skills of
working in the operating theatre. These tools have been adopted by
surgical and anaesthetic
professional bodies in the UK and beyond, influencing medical education,
clinical guidelines and
professional standards and training.
Since 2008, the Royal College of Surgeons Edinburgh (RCSEd) [5.1] has
delivered (with input from
University of Aberdeen psychologists), a programme of courses on surgeons'
using the NOTSS system. These are: Training the trainers online training
programme (NOTSS in a
box); NOTSS Masterclass since 2009 (high-level skills course with
international delegate list, 110
trained to date); International NOTSS workshops (e.g. Melbourne,
Philadelphia, Budapest, Ottawa,
Lisbon, Tokyo ,Kuwait — approx. 120 trained to date; workshops for Macau
and Hong Kong later in
2013); with an e-learning module for surgical trainees. Safer Operative
Surgery (SOS) courses
held across NHS institutions since 2007 were delivered by the NOTSS
faculty of the University of
Aberdeen and RCSEd. (120 delegates). These also use the ANTS and SPLINTS
The Intercollegiate Surgical Curriculum Project (ISCP) [5.2] decided to
adopt the NOTSS system
for assessment of all surgeons in training in the UK in 2012. In the USA,
SCORE (Surgical Council
on Resident Education) decided (June 2013) to incorporate NOTSS in their
curriculum for the next
academic year as a method of teaching and assessing senior residents. This
practically every surgical resident in the US will spend December 2013 on
two NOTSS modules
(cognitive skills, social skills). The American College of Surgeons
included a whole day training
workshop on NOTSS at the 2013 congress (Washington, October), led by a
Harvard surgeon [5.3].
The Royal College of Surgeons England training course `Patient Safety in
Theatre Teams', which
began in 2012, uses the ANTS, NOTSS and SPLINTS tools as core elements of
the course. [5.4]
The Royal Australasian College of Surgeons, adopted the NOTSS system in
2008 as part of their
competence framework for surgeons (Professional Standards Document). This
is included in their
updated 2011 /12 version and they have developed a Faculty of surgeons who
are running NOTSS
training courses, with 12 courses scheduled for 2012/13. To date 20
faculty have been trained to
teach this course, with 340 attending. The Australian and New Zealand
College of Anaesthetists is
developing a similar 'professionalism guide' and is intending to use ANTS
as part of it (Dr Graham,
ANZCA, personal communication June 2013).
The Royal College of Anaesthetists in London began using ANTS in its
Certificate of Competence
in training (CCT) in 2007, also in later editions. [5.5] In 2011, it
issued advice that ANTS could be
used as part of a retraining package for staff returning to work after
maternity leave. The May 2011
council meeting of the Royal College of Anaesthetists referenced the
research by discussing the
merits of attaching the correct importance to anaesthetic non-technical
skills in arranging Specialty
Trainee year 3 appointments (ST3). [5.5] ANTS is used to build a new
method of assessment in
the selection of anaesthetic trainees in South West England. [5.5]
The ANTS framework was incorporated into the UK National Advanced and
Support Courses validated by the Resuscitation Council (UK) in 2011. [5.6]
These courses are
taught to every medical graduate in the UK and many other senior
clinicians (e.g. nursing,
dentists). The European Board and Section of Anaesthesiology under the
European Union of
Medical Specialists (Union Européanne des Médicins Spécialistes) specified
ANTS as a `part of
the core competencies for postgraduate training in anaesthesiology in
Training in the use of the tools has been devised for surgeons (see
above), anaesthetists and
scrub practitioners, with workshops run since 2008 in the USA, Australia,
New Zealand, Japan,
Malaysia, Denmark, Austria, Canada, Hungary and Spain. The tools have
recognition and the frameworks are now being adopted in other countries
(NOTSS translated into
Japanese and Danish; ANTS into Korean, Japanese, German, Danish, SPLINTS
The ANTS tool was made freely available from the start, and the NOTSS and
SPLINTS tools are
available on request via the tool website. Since October 2012, over 80
requests from scrub
practitioners for SPLINTS have been received from China, Denmark [5.9],
Sweden, New Zealand, India, Korea, USA, Latvia, Germany, Italy,
Switzerland [5.10], Hong Kong,
Iran, Malaysia and Thailand, as well as the UK.
A book of collected papers from the first specialist meeting to discuss
the measurement of non-technical
skills in the operating theatre, (attended by psychologists, surgeons,
anaesthetists) was Highly Commended in the surgery category of the British
Book of the Year 2010. This knowledge transfer event was established by
the Aberdeen research
team as an annual event, and the Karolinska Institute in Stockholm hosted
it in 2013.
The development method and design framework have been adopted by other
such as emergency physicians (Flowerdew, Annals of Emergency Medicine,
2012), who have used
it to develop similar tools for their professions.
Therefore the impact, as defined by REF, was in professional practice,
methods and training; decisions by professional bodies have been
influenced by this research into
non-technical skills and professionals and practitioners have used the
research findings in
conducting their work.
Sources to corroborate the impact
[5.1] The Royal College of Surgeons in Edinburgh is using the NOTSS
system, including on their
Safer Operative Surgery Courses. http://www.rcsed.ac.uk/education/patient-safety-and-notss.aspx
[5.2] Intercollegiate Surgical Curriculum Project link to
(password required, screen shot available) and testimonial from Patient
Safety Board member/Past Vice-President of the Royal College of Surgeons
[5.3] American College of Surgeons — workshop on NOTSS http://tinyurl.com/NOTSS-ACS-2013
and Testimonial from Programme Director of General Surgery Residency of a
Harvard Medical School.
[5.4] Royal College of Surgeons England — www.rcseng.ac.uk/courses/course-search/patient-safety-in-theatre-teams.
This course uses the NOTSS, ANTS and SPLINTS tools as part of training
(Prof Baxendale, course tutor personal communication January 2013).
[5.5] Royal College of Anaesthetists — e.g. CCT in Anaesthetics 2010
Ed 2 version 1.4 chapter 5/
appendix 7; RCA Teaching and Training in the Workplace. 2011, p10.
26.8.13; Gale et al (2010), BJA, 105.
[5.6] Resuscitation Council (UK) Advanced Life Support Manual 6th
Edition & Resuscitation
Council (UK) Immediate Life Support Manual 3rd Edition — 2011 available for purchase from
"`To improve the situation the Anaesthetists Non-Technical
Skills (ANTS) system was developed. The principles used to promote good
skills in the ILS course are based on the principles of ANTS" (page 1).
[5.7] Royal Australasian College of Surgeons (2008, 2012) Surgical
Performance: A Guide by the Royal Australasian College of Surgeons.
Department of Professional
Standards. Melbourne: RACS. NOTSS training courses:
[5.8] European Union of Medical Specialists
document shows the use of ANTS in postgraduate training.)
[5.9] Translation and adaptation of NOTSS for use with surgeons in
Spanager L, et al (2012). Customization of a tool to assess Danish
surgeons´ non-technical skills
in the operating room. Danish Medical Journal 59, 1, A4526.
[5.10] Study by Swiss anaesthetists using the ANTS tool. Riem N, et al
(2012) Do technical skills
correlate with non-technical skills in crisis resource management. B.
J. Anaesthesia, 109, 5, 723-8.