Summary of the impact

Research by the University of Nottingham's Education and Technology for Health team has benefited healthcare students, professionals, users, carers and institutions both in the UK and internationally by establishing a participatory methodology for high-quality, sustainable multimedia Reusable Learning Objects (RLOs). These learning tools are now used in 50 countries globally to facilitate individual access to knowledge, enhance learning within curriculums and deliver continuing professional development, with feedback showing satisfaction of up to 100% in some nations. They are also being used to train healthcare professionals in resource-poor countries, further strengthening the University of Nottingham's role as a global education provider.

Underpinning research

Reusable Learning Objects (RLOs) can be defined as short, self-contained chunks of web-based learning, each focusing on a specific topic, with multimedia components such as audio, text, images and video. Their ability to engage healthcare users in interactive learning has earned increasing recognition in recent years. Studies by the University of Nottingham's Education and Technology for Health team have established a participatory methodology that leads to high-quality, sustainable multimedia interventions designed to satisfy the growing need to provide excellent online educational materials and products that are based on sound pedagogy and research.

Professor Heather Wharrad (Reader 2006-2012; Professor of E-Learning and Health Informatics 2012-present), Dr Richard Windle (Secondment to RLO Centre of Excellence in Teaching and Learning 2006-2011; Associate Professor 2011-present), Dr Joanne Lymn (Associate Professor 2007-present) and Dr Holly Blake (Lecturer 2006-present) have led the group's research into the design and use of health-focused RLOs for the past decade, working with a variety of institutional partners to develop 180 freely available RLOs on an open website. The nature of these RLOs has been determined by areas of difficulty experienced by healthcare students (e.g. evidence-based practice, pharmacology, statistics) and areas of the curriculum requiring regular reinforcement (e.g. practice skills, study skills), as well as by end-users in a participatory methodology that includes peer review at various stages [1]. Stakeholder input is of major importance in the success of RLOs and accompanying research, as it ensures that content is strongly aligned with need, that design is appropriate to learning style and that ownership leads to extensive use and reuse [1].

Various studies by the group have demonstrated the benefits delivered by RLOs' flexibility, accessibility and ability to allow users to go over material repeatedly. Such advantages have been shown to be of particular use to healthcare practitioners and students interweaving their learning around busy shifts and demanding courses. For example, research by Wharrad and Lymn, published in 2008, revealed how students' perceptions of their own understanding of pharmacology concepts increased substantially following the introduction of RLOs to supplement the pharmacology component of their course, with some respondents subsequently implicating RLOs in developing confidence in their prescribing abilities [2].

An independent study by Blake, published in 2010, highlighted nursing and medical students' different learning needs and the potential barriers to the effective use of RLOs, including lack of IT competence, technical difficulties and limited awareness of computer-based learning aids [3]. Research by Wharrad and Windle, published in 2011, showed a significant increase in the number of students answering chemistry exam questions correctly following the introduction of RLOs [4], while a study by Wharrad, also published in 2011, highlighted RLOs' ability to support students struggling with meta-analysis [5].

As the effectiveness of RLOs became increasingly recognised through these and other studies, reuse extended beyond the University of Nottingham to reach a global distribution of users. This opened a research strand about the reuse of RLOs as open educational resources (OERs), focusing on gaining an understanding of OER reuse patterns and resulting in the award of an Open University Support Centre for Open Resources in Education (SCORE) fellowship to Windle [6].

References to the research

Publications:

Grants:

Wharrad, HJ, and Garrud, P (University of Nottingham); Boyle, T (London Metropolitan University); and Leeder, D (University of Cambridge): Higher Education Funding Council Centre for Excellence in Teaching and Learning in Reusable Learning Objects, 2005-2010 - £3.3m + £40,000 capital funding

Wharrad, HJ, Timmons, S, Randle, J, Bath-Hextall, F, and Lymn, J: Cross-sector development and delivery of reusable learning objects to support life-long learning in health sciences and practice, Eduserv Foundation, 2005 — £152,000

Leeder, D (University of Cambridge); Wharrad, HJ, and Windle, R (University of Nottingham); and Morales, R (University of Cambridge): Sharing the LOAD — learning objects, activities, designs, JISC pedagogical design strand, 2006 - £58,000

Glazebrook, C, and Johnson, S (University of Leicester); and Beer, C, Westwater-Wood, S, Budge, H, and Wharrad, HJ (University of Nottingham): Development and evaluation of a multimedia parenting intervention to promote motor development in infants born very premature (HOPON), Action Medical Research, 2006 — £116,044

Ferguson, M (National Biomedical Research Unit for Hearing); and Wharrad, HJ, Fortnum, H, and Leighton, P (University of Nottingham): Evaluation of interactive videos for hearing-aid users, NIHR Research for Patient Benefit, 2010 — £235,000

Windle, R: One-year SCORE fellowship, awarded by JISC and Open University, `Understanding and supporting the reuse of OER with health sciences', 2011

Details of the impact

The participatory methodology established by the Education and Technology for Health team's research has led to an extensive and ever-expanding repository of evidence-based RLOs [a]. These learning tools have benefited healthcare students, professionals, users, carers and institutions both in the UK and internationally.

Although created by the team and their collaborators for use within host institutions, the RLOs were released for non-commercial reuse from the outset. This decision was made well before the creation of Creative Commons Licensing regimes for educational resources in 2007. Requiring no password or complex technology, the RLOs are now accessed across all continents and in around 50 countries. All have optional online evaluation forms that allow the gathering of information about their use, and these have been used to evidence their reach and significance throughout the impact period [b, c].

A detailed evaluation of 71 RLOs [b] analysed 13,217 respondents' forms between May 2006 and July 2013. Estimates from comparison with server logs suggests 1-2% of users return evaluations, which extrapolates to around 1,300,000 RLO users over this period. Students were the largest group (67%), with the other 33% being lecturers, nurses, doctors, users, carers and the public. 97.9% (12,954 respondents) rated the RLO as "very helpful" or "helpful" for learning a subject. Satisfaction levels ranged from 96% in African respondents to 100% in South American and Caribbean and Middle East respondents. An analysis carried out as part of Windle's SCORE fellowship showed learners were most likely to redistribute the resources, followed by educators and healthcare professionals [c].

A wealth of feedback has evidenced the RLOs' role in enhancing knowledge and confidence among students. For instance, a user from Rutgers University, New Jersey, commenting on the Volume of Distribution RLO, remarked on June 18 2011: "I have a much better understanding of what is needed to obtain therapeutic drug levels. This was the easiest online tool I found." A student of medicinal chemistry at the University of Minnesota wrote on January 5 2011: "It really makes you think and understand the material. I have been very confused after sitting through lectures with my professor. This just explained VD [volume of distribution] in about 15 minutes." Other comments have included "I could go back and listen again until I understood — you can't do this in class" and "It was really helpful seeing the different stages on screen rather just reading it from a textbook — it helps the information sink in, because you can then visualise it".

Practitioners have also provided significant positive feedback. For example, referring to the Improving Patient Safety — SBAR RLO, an honorary consultant surgeon leading a hospital-wide patient safety project at Barnet and Chase Farm Hospitals NHS Trust, North London, wrote on September 16 2010: "The group that have been looking at the teaching materials available `off the shelf' are very impressed with your e-learning module and would be keen to make extensive use of it." [d] An employee at RN Poison Control, Oregon Health Sciences University, noted on April 17 2010: "VD [volume of distribution] has always been beyond comprehension. This made it easy." An anaesthetist at the University of Sydney, commenting on the Sensitivity and Specificity RLO, remarked on February 4 2012: "I find statistics difficult to grasp and have always found so-called `easy' statistical concepts such as sensitivity and specificity difficult. Your programme, with its clear and unambiguous instruction and clear graphics, has revealed all. Yes, it is `easy' when taught well. I never thought I would ever be able to teach these concepts, but now I can — with confidence." A member of staff at a learning disability service in Sligo, referring to the Little Things Make a BIG Difference RLO, said on April 13 2010: "This learning object helped me to reflect on many aspects of supporting service users, including communication, rights, dignity, respect, equality... and the visuals helped to highlight this."

Many service users have also praised the tools' effectiveness. One parent of a child with autism, commenting on the "Hello — My Name Is Tom" Learning Disability RLO, wrote on September 6 2010: "As a parent, you can easily get lost in the heat of the moment and feel that you are the only one going through this... I would like to see a link to this site for teaching staff at all our schools."

A number of follow-up studies have demonstrated how the team's methodology has translated to the healthcare sector to provide user-designed multimedia education interventions for users and carers, including HEAR-IT (Hearing Evaluation of Auditory Rehabilitation — Interactive Tutorials), which was carried out from January 2011 to May 2012 to address the 25-30% non-use of hearing aids. Participants took part in focus groups and workshops to develop seven RLOs covering educational and practical information on the use of hearing aids and how the brain adapts. Users of the RLOs were later found to have better knowledge of hearing aids and communication and scored significantly higher on practical hearing-aid skills [e].

Since 2013 RLOs focusing on clinical skills and biology have been used in resource-deprived countries. In Juba, South Sudan, in a project headed by a consultant paediatric surgeon at Oxford's John Radcliffe Hospital, they are being used on a portable device to train students with no broadband access [f]. In Malawi, in an initiative led by an emergency department nurse from Nottingham University Hospitals NHS Trust, they are being used to educate health surveillance workers [g].

Sources to corroborate the impact

a. List of RLOs developed by University of Nottingham Health E-Learning and Media (HELM) team based on research by Education and Technology for Health team [Accessed 15 October 2013]
http://www.nottingham.ac.uk/helm/resources/learning-objects/rlo-school.aspx

b. E-Learning Object Impact Study Report, prepared by external consultant, July 2013
http://www.nottingham.ac.uk/helm/documents/elearning-object-impact-report.pdf

c. JISC SCORE Fellowship Final Report 2012
http://www.open.ac.uk/score/files/score/file/Richard%20Windle%20SCORE%20Fellowship%20Final%20Report%20Web%20Version.pdf

d. Individual beneficiary using Improving Patient Safety — SBAR RLO for staff training (honorary consultant surgeon at Barnet and Chase Farm Hospitals NHS Trust)

e. Research Lead, Habilitation for Hearing Loss, National Institute for Health Research, Nottingham Hearing Biomedical Research Unit

f. Individual beneficiary using pharmacology and skills RLOs in Juba, South Sudan, for training medical students (consultant paediatric surgeon, John Radcliffe Hospital, Oxford)

g. Individual beneficiary using biology and skills RLOs with health surveillance workers in Malawi (emergency department nurse at Nottingham University Hospitals NHS Trust)