5. Enhancing learning, teaching and assessment at university
Submitting InstitutionUniversity of Edinburgh
Unit of AssessmentEducation
Summary Impact TypeSocietal
Research Subject Area(s)
Education: Curriculum and Pedagogy, Specialist Studies In Education
Summary of the impact
Outcomes of the research conducted at the University of Edinburgh (2001
to 2007) that have had the most far-reaching impact are a strong
conceptualisation of the whole learning environment (including curricula,
teaching, learning support, and assessment and feedback) and its influence
on the quality of undergraduates' learning. What gave these outcomes added
resonance was a concern for disciplinary distinctiveness as well as more
generic features; an alertness to the pervasive implications for
day-to-day teaching-learning practices of mass 21st-century
higher education; and a focus on enhancing as well as evaluating the
The reach of the impact extends to university teachers, middle and senior
academic managers, local and national bodies with responsibilities for
surveying quality and standards and, albeit less directly, students. Staff
in at least 21 universities in 12 countries have used the Experiences of
Teaching and Learning Questionnaire (ETLQ). The National Student Survey
questionnaire was influenced by the ETLQ, and has continuing UK-wide
impact on teaching through students' retrospective ratings of their
experience. Project outputs were directed towards teaching staff through
workshops, publications and invited presentations, followed by detailed
advice on assessment and feedback of coursework.
Two main research projects underpin the impact described here:
- ESRC/TLRP Award (L139251002 for £847,560, 2001-2005), evaluated as
Very Good/Excellent. Enhancing Teaching-Learning Environments in
Undergraduate Courses. Co-directors: Entwistle, Hounsell;
Associate directors in Edinburgh: Anderson, McCune, with contributions
from Durham and Coventry
- Higher Education Academy (£30,000, 2006-2007), Innovative
Assessment Across the Disciplines (Director: Hounsell)
Key researchers, all at the University of Edinburgh, include Noel
Entwistle, Bell Professor of Education (1978-2005), now Professorial
Research Fellow (2013-2014); Dai Hounsell, (1985-2012), Professor of
Higher Education, now Professor Emeritus; Dr. Velda McCune, (1999-2008,
2011-), Deputy Director, Institute for Academic Development and Dr.
Charles Anderson (1987-), Senior Lecturer.
Enhancing Teaching-Learning Environments (ETL) was part of the
highly selective ESRC Teaching and Learning Research Programme. It grew
out of investigations into students' approaches to learning that raised
questions about how teaching could better encourage understanding. The
project was designed to involve university teachers in selected
departments — electronic engineering, biological sciences, economics and
history — with the specific aim of demonstrating how research can aid
The ETL project developed an inventory (Experiences of Teaching and
Learning Questionnaire — ETLQ) to collect students' descriptions of their
approaches to learning and the teaching and learning environment they had
experienced. Items included were based on previous research to develop a
conceptual framework through which implications for teaching practice
could be identified (Entwistle, McCune & Hounsell, 2003). During the
project, the questionnaire was administered to over 6500 students in 11
British universities, involving 90 university teachers in each stage of
The questionnaire dataset was complemented by a similarly substantial
corpus of data derived from semi-structured interviews with nearly 670
students and 90 academic staff. Results from an initial round of
data-gathering by questionnaire and interviews were discussed with the
course teams involved in the project and, where feasible, led to changes
in the way the course was taught the following year. The effects of the
changes were investigated through a second round of data-gathering.
Drawing on both sources of data, the findings of the project threw new
light on undergraduate courses as teaching-learning environments in 21st-century
mass higher education. It was suggested that the focus of courses should
be on developing the distinctive ways of thinking and practising
characteristic of the subject area concerned, rather than on lists of
intended learning outcomes. It was shown how certain aspects of the
teaching-learning environment — most notably, teaching for
understanding, staff enthusiasm and support, and
well-aligned assessment-for-learning and feedback on students'
progress and performance — could have a powerful effect on the
student experience in each of the course units investigated, including
their ways of studying and their perceived academic achievement. The
research also indicated how the key facets of an undergraduate
teaching-learning environment needed to be coherent with the main
aims of the course and congruent with each other, as well as with
the previous knowledge and experience of the students.
Later follow-up research, including a systematic review of the literature
on innovations in practices funded by the Higher Education Academy,
focused particularly on the interwoven themes of assessment-for-learning
and feedback to students, which had emerged as areas of greatest student
dissatisfaction in successive National Student Surveys in the UK, as well
as in surveys and empirical studies in Hong Kong and Australia (Hounsell
et al. 2008). What particularly interconnected this follow-up work closely
to the seminal ETL project was a focus on the interrelationships between
assessment design and feedback provision and the quality of student
learning, the crucial role of the subject/disciplinary dimension in
shaping opportunities and affordances, and the role of evidence in
understanding and enhancing practices on the ground. It has taken that
work further forward by marrying research findings and conceptual insights
to a large pool of documented initiatives grounded in specific subject and
course settings. As reported in the sections below, the fruits of this
synthesis are evident in the HEA Innovative Assessment project,
Enhancement Themes initiatives and a website.
References to the research
Entwistle, N. J. (2009) Teaching for understanding at university:
Deep approaches and distinctive ways of thinking. Basingstoke:
Palgrave Macmillan. Supplied on request.
Entwistle, N. J., McCune, V., & Hounsell, J. (2003) Investigating
ways of enhancing university teaching-learning environments: Measuring
students' approaches to studying and perceptions of teaching. In E. De
Corte, L. Verschaffel, N. Entwistle, & J. van Merriënboer (Eds.) Powerful
learning environments: Unravelling basic components and dimensions
(pp.89-108). Oxford: Pergamon. Supplied on request.
Hounsell, D. J. & Entwistle. N. J. (2006) Enhancing
teaching-learning environments in undergraduate courses. Final
Report to the ESRC on Project L139251099.
Hounsell, D., McCune, V., Hounsell, J. and Litjens, J. (2008) The quality
of guidance and feedback to students. Higher Education Research &
Development 27:1, 55-67. In REF 2 (McCune).
Hounsell, D., Balir, S. Falchikov, N., Hounsell, J., Huxham, M.
Klampfleitner, M. & Thomson, K. (2007) Innovative Assessment
Across the Disciplines: An Analytical review of the Literature.
Final report. York: Higher Education Academy (HEA).
The quality of the research is evidenced by the status of the refereed
publications, the sources of funding, the evaluation of the ESRC project,
the engagement with prominent national and international higher education
bodies, and international book publishers.
Details of the impact
The impact of the research across and beyond the UK arises from its
distinctive range and blend of outcomes and outputs. Given the quite
substantial measure of autonomy which university teachers continue to be
able to exercise in matters of curriculum, teaching-learning provision and
approaches to assessment and feedback, they represent a primary target.
They are, however, a difficult group with which to engage, not least
because of the extent to which their professional identities, allegiances
and channels of communication are allied to their chosen discipline or
subject area. Our impact strategy has therefore been pursued along a
variety of routes, combining generic and discipline-specific
dissemination, and interaction through institutional, national and
One fertile pathway to impact stems from the evolution of the influential
ETLQ survey instrument. A precondition of later impact at national level
was Entwistle's involvement (2002-03) on a sub-committee designing the
National Student Survey (NSS). His experience of developing the ETLQ and
its predecessor contributed significantly to the selection of NSS items
and the design of its rubric and format [5.1]. This has continued to have
an impact during the REF period, as the NSS has grown in significance in
terms of its capacity to highlight the importance of the student
experience and consequently the need to direct university resources
towards changing teaching practices.
The ETLQ instrument has also been adopted and used by higher education
institutions nationally and internationally. The inventory was selected
for inclusion in a peer-reviewed teaching resource bank developed for the
Association of Medical Colleges in the USA [5.2]. The University of
Helsinki has adapted versions of the questionnaire to use routinely and
annually with all students as part of their portfolios [5.3]. Departments
are provided with scale scores shown in relation with overall norms, and
use the ETLQ results to introduce and monitor innovations based on the
analyses. The scale scores are also used in counselling individual
students, and findings from the growing corpus of ETLQ data at Helsinki
are now being promulgated through articles in refereed international
journals [5.4]. In 2013, Universities in Greece and Portugal were
trialling the general use of ETLQ [5.5], while teachers have reported
using the instrument individually, or with colleagues, at universities or
colleges in Alberta, Antwerp, Cape Town, Galway, Lausanne, Granada,
Prague, Punjab and The Hague. In addition, eighteen lecturers in nine
British universities have reported using ETLQ to monitor students'
perceptions of their teaching. The ETLQ's growing accessibility on
websites and in publications increases the likelihood of other uses that
have not been reported.
A complementary avenue to impact has focused on the findings as a whole,
disseminated through a book written specifically for practising university
teachers across the subject spectrum (Entwistle, 2009), in tandem with
invited presentations to bodies concerned with improving the quality of
teaching and learning. For example, Entwistle (2008) was invited to
contribute a plenary paper for a conference organised for the Higher
Education Quality Council of Ontario, with an audience of some 75 invited
academics mainly involved in professional development [5.6]. Another nine
invited presentations and workshops have been provided in UK, Sweden and
Finland between 2008 and 2013.
Drawing on findings from the research, in the period 2008-2013 Hounsell
has given more than 40 keynote and workshop presentations to a combined
total of some 2,500 practitioners and academic managers in the UK, Sweden,
Norway, Denmark, Canada, China and Australia, and to bodies including the
Australian Learning and Teaching Council, the Australian Technology
Universities Network, and Universitas21.
His follow-up research on assessment and feedback has also entailed
pursuing more innovative modes of dissemination that combine reviews of
empirical evidence and conceptual insights with a wealth of illustrative
case-examples from across the disciplinary range that have been sourced
internationally as well as from the UK. One such instance was the
production in 2007 of a set of four guides for university teachers on
integrative approaches to assessment and feedback [5.7]. Commissioned as
part of the Enhancement Themes initiative by the Quality Assurance Agency
Scotland in partnership with Universities Scotland, the Scottish Funding
Council and the Higher Education Academy, the guides were disseminated to
staff in all Scottish universities, and there has been continuing
subsequent involvement in sector-wide enhancement [5.8] and communication
with middle and senior university managers through the Leadership
Foundation [5.9] throughout the REF period. A groundbreaking website, Enhancing
Feedback, has been created to provide reviews of over thirty
strategies for generating feedback to students and links these to more
than 200 case-examples from a wide range of subject areas. Since its
launch in 2010, there have been 27,000 visitors to the website from over
one hundred countries [5.10].
Sources to corroborate the impact
Sources to corroborate the impact are indicated in square brackets in the
text. Web pages have been archived at https://www.wiki.ed.ac.uk/display/REF2014REF3B/UoA+25
[5.1] For confirmation, contact member of the Open University team that
conducted the pilot study for the National Student Survey and can
corroborate the contribution of this research to its design.
[5.2] Directory and Repository of Educational Assessment Measures (DREAM)
[5.3] See presentation at http://www.helsinki.fi/ktl/yty/tutkimus/LEARNresearch_proj160609.pdf
that reports the use of the ETLQ.
[5.4] See for example article by Parpala, A., & Lindblom-Ylänne, S
(2012) Using a research instrument for developing quality at the
university. Quality in Higher Education 18, 313-328.
[5.5] Details of a seminar on teachers and the quality of learning to
which contacts at University of Evora, Portugal and University of
Ioannina, Greece contributed:
[5.6] This led to a further written contribution: Entwistle, N. J. (2010)
Taking stock: An overview of research findings. In J. Christensen Hughes
& J. Mighty (Eds.), Research on Teaching and Learning in Higher
Education. Montreal: McGill-Queens University Press. Supplied on
[5.7] Reports: Hounsell, D., Xu, R. and Tai, C.M. (2007) Monitoring
Students' Experiences of Assessment; Balancing Assessment of and
Assessment for Learning; Blending Assignments and Assessments for
High-Quality Learning; Managing Assessment Practices and Procedures.
Gloucester/Glasgow: Quality Assurance Agency for Higher Education.
[5.8] Report: Hounsell, D. (2011) Graduates for the 21st Century:
Integrating the Enhancement Themes. Institutional Activities.
Glasgow: QAA Scotland. (ISBN 978 1 84979 410 7).
[5.9] Paper: Hounsell, D. and Rigby, S. (2013) Leading Change in
Assessment and Feedback. Case Examples and a Guide to Action. (Series
Four Research, Commissioned Papers). London: Leadership Foundation
for Higher Education.
[5.10] Enhancing Feedback website. www.enhancingfeedback.ed.ac.uk
Data on visitors was sourced from Google Analytics.