Summary of the impact

The use of technology to enhance student learning is known to have a significant impact on achievement in all subject areas and across all stages of schooling and learning. Educational computer games and online tools help engage students — making learning enjoyable and therefore more effective. Computer scientists at Leicester are expert in the analysis of online learning tools and educational games. They have used this expertise to evaluate whether and why such games or tools work and, most importantly, how they can be improved.

The research has been used by:

• The software industry to develop new products for the e-learning and games markets.
• Schools and higher educational institutions to create tailored e-learning tools which enable better learning experiences and improved outcomes for students.
• Industry to design and improve effective e-learning modules for employee training.

Underpinning research

Technology-Enhanced Learning involves sophisticated uses of various tools, services, resources, and methods in order to attain the ultimate goal of learning enjoyably and effectively. Not only learning outcomes but also learning processes can be enriched with educational technologies, which, if appropriately designed, should give rise to excellent user experience and learning experience. The Unit is expert in the evaluation of whether the design of a digital educational game or an online tool works (or does not work), why, and how it could be improved.

While both user experience and learning experience are subjective feelings arising from interacting with a system, user experience is more associated with the quality of interface design whereas learning experience is more with the quality of content. But the distinction between the two becomes blurred when the system components are tightly coupled, especially in an adaptive computer game and learning environment where both the interface and content are dynamically changing, contingent on moment-to-moment user behaviour and performance.

Hence, developing a robust evaluation framework for such complex technology-enhanced learning systems is a major research challenge. It involves selecting, appropriating, augmenting and harmonizing existing approaches, techniques and tools as well as creating new ones to address diverse contextual needs. This research challenge has been addressed by the University of Leicester team through three EU-funded research projects:

a. 80Days (2008-2010; http://www.eightydays.eu/)

The project involved building a scientifically sound model for an assessment of knowledge and learning process which did not disrupt the flow of the gameplay. The results were embedded in a game. The team developed a four-dimensional evaluation framework based on Game usability, Gaming experience, Learning effectiveness, and Real-time interaction trajectory for adaptivity. It was grounded in theories of cognition, emotion, learning, motivation, and interaction. The team used the framework to evaluate the research project's adaptive game in several schools in England and Austria. Evaluation feedback obtained underpinned work to improve the adaptive game enhancing its enjoyment and learning effectiveness.

b. ROLE (2009-2013; http://www.role-project.eu/)

This EU-funded project investigated how learners can be supported to construct and adapt their online learning environment when migrating across contexts. Leicester's contribution was to apply the user-centred design approaches to elicit and analyse requirements from distributed and diversified users. This knowledge was then built-in to a suite of tools and resources, which were designed as part of the project and were iteratively evaluated by Leicester.

c. Go-Lab (2012-2016; http://go-lab-project.eu/)

This on-going project aims to create a portal that enables students to perform scientific experiments with online labs augmented by the pedagogical concepts of the inquiry learning cycle. Teachers can enrich their classroom activities with such labs and share best practice in a community. It also enables lab owners to promote their scientific activities. 25 Visionary Workshops and 9 Participatory Design workshops were conducted in nine and five European countries respectively, involving 728 participants (685 teachers and 43 students from schools). Leicester was responsible for collection and analysis of the data from these workshops to extract requirements for the technical team to develop the Go-Lab Portal.

The key Leicester researcher in all three projects was Dr Effie Law (Lecturer during 80-Days and ROLE and Reader during Go-Lab).

References to the research

Research grants

80days — Around an Inspiring Virtual Learning World in Eighty Days,2028 FP7-ICT-2007.4.1, €3,297,000 (Leicester: €346,000), April 2008 - September 2010. Grant holder: Effie Law

ROLE — Responsive Open Learning Environments, FP7-ICT-2007.4.3, €6,600,000 (Leicester: €497,420), Feb 2009 - Jan 2013. Grant holder: Effie Law

Go-Lab — Global online science labs for inquiry learning at schools, FC-ICT-2011-8, €9,696,582 (Leicester: € 713,048), Nov 2012-Oct 2016. Grant holder: Effie Law.

Details of the impact

Impact of ROLE project
The way in which the internet is used is changing. Once, websites were used passively to view content already created. Now users are interacting and collaborating with each other in social media dialogue within virtual communities. Examples of this so-called Web 2.0 concept include social networking sites, blogs, wikis, video sharing sites, hosted services, web applications, mashups and folksonomies.

The ROLE research project explored the place of e-learning in this new context. Its main objective has been to support teachers in developing open personal learning environments for their students. The project has created resources and tools for teachers which give them the ability to create such technology-driven learning environments for their students, which they are able to adapt to their own specialisms and needs, irrespective of individual operating systems. These resources and tools include software applications with which teachers and trainers can build their own online learning environments, and widget bundles to support a specific learning or teaching task. It has also developed a mash-up collaboration system called Graasp, which enables teachers and students to link different social media applications into their personal learning environment.

Most of the technological solutions developed as part of the ROLE project are shared with the open-source community — that is they are freely available to anyone to download. Access points have recorded more than 100,000 page views. The users/beneficiaries include:

HEIs and Schools — creating effective e-learning environments for students — The open access software and tools created by ROLE have been used by RWTH Aachen University in Germany in a Computer Science course. They created an electronic reference book, the Web 2.0 Knowledge Map, as a kind of improved Wikipedia system. The application supports students in looking up factual knowledge needed in their homework assignments to design computer software. Students can search for knowledge articles by entering topic keywords and navigate from the current article to related articles using hyperlinks. It is based on semantic net technology, where hyperlinks are not just links, but belong to predefined categories, each bearing a meaning, as a named relation. The Knowledge Map tool won the second prize in the 2010 International E-Learning Association Awards, in the category "Academic Blended Learning".

IMC AG — developing commercial products for the e-learning sector — The international company IMC AG is based in Germany and provides technology-enhanced solutions for learning and training. IMC has integrated ROLE technologies into their R&D programme. They have been used to underpin and improve their products for teachers and trainers, including an award-winning learning management system called CLIX. It has also underpinned a MOOC (Massive Open Online Course) platform called OpenCourseWorld — this is an online course aiming at large-scale interactive participation and open access via the web. The company is also working on the development of a mobile learning infrastructure for tablet- technologies. It is designed to implement the use of social media in the classroom to create a useful and efficient learning environment. POLO is still at the developmental stage but the R&D programme at IMC AG is underpinned by ROLE technologies and insight. (5.5)

Festo — enhancing employee training through more effective e-learning tools — Festo is a leading world-wide supplier of automation technology and the performance leader in industrial training and education programs. The company uses the CLIX learning management system and was one of the five main test-beds for ROLE, to evaluate the project's solutions for technology-enhanced workplace lifelong learning. (5.6)

Impact of Go-Lab project
The EU Go-Lab project — Global Online Science Labs for Inquiry Learning at School opens up remote science laboratories and their online models (online labs) for large-scale use in education. Students from around the world can perform personalized scientific experiments with online labs, and teachers can enrich their classroom activities with demonstrations and disseminate best practice in a web-based pedagogic community. It also gives the owners of the labs the chance to promote their scientific activities. As of Sept 2013, 43 students and 685 teachers have used Go- Lab, inspiring young people to enjoy and pursue careers in science — a key priority in both the UK and European Union, to support future knowledge-based economies. (5.7)

Impact of the 80-Days project
During the 80Days project, researchers worked with the games developer community to develop psycho-pedagogical and technological foundations for successful computer games in terms of educational efficacy as well as financial turnovers. (5.1, 5.2)

Economic impact on SME partners
Two game developer companies were involved with the 80-Days project — "learning games" specialists TAKOMAT GmbH, of Germany, and "serious games" experts Testaluna, of Italy. Involvement in the 80Days project enhanced both companies' know-how in designing and developing complex and high-quality learning games. Both companies use their involvement with leading-edge research has a "unique selling point" when marketing themselves.

Both companies used the 80Days demonstrator game — Lizard 3.0 (developed at Leicester) — and the scalable 80Days game concept for acquisition of other learning game projects and the search for investors and/or publishers to realize the 80Days game concept and turn it into a full game title. TAKOMAT also used the demonstrator game for the successful acquisition of a commercial serious game project — Energetika, the winner in the Best Serious Game category in the 2010 German Game Prize. (5.3)

TESTALUNA also used involvement in the research to win a contract for the development of a serious game. The company is developing two separate lines of products: one for medium-sized educational products, in association with national public bodies, and the other for small educational serious games, to be launched by free games portals. (5.4)

Awareness raising within the games developer community
The 80-Days project team from Leicester were involved in several international events which raised the awareness of techniques that support high-quality learning games to game developers. These included the Montréal International Game Summit 2008 (1,300+ participants) and Gamescom 2009 (circa 245,000 participants).

Awareness raising within the potential beneficiary community
The Leicester team contributed to a number of initiatives for raising the awareness of scientists, politicians, professionals, teachers and pupils to the role and opportunities of video games for learning experience. These included the World Cyber Games 2008 and 3^rd Youth Forum North-Rhine-Westphalia, Cologne, Germany, 07-08.11.2008. More than 5,000 people attended including schools teachers, e-learning professionals and politicians.

Sources to corroborate the impact

[1] Guido Doublet, Lars Maria Schnatmann, Daniel Schwarz (Eds.). Final plan for the use and dissemination of foregrounds (D25). Internal deliverable of the project 80Days.

[2] Final Review Report of the project 80Days (22nd October 2010). EU reviewers

[3] Corroborating source: Managing Partner and Founder at TAKOMAT GmbH, Germany

[4] Factual statement from Executive Producer at Testaluna, Italy

[5] Corroborating source: Project manager at IMC AG, Germany

[6] Corroborating source: Consultant at FESTO

[7] Post-project report to the funder (EU) on the Go-Lab project.