Summary of the impact

"Helping Engineers Learn Mathematics" (HELM) was a sector-wide developmental research project originating at Loughborough University. It impacted on:

a) Teacher awareness and understanding: in a climate of reluctant engagement and poor performance when undergraduate engineering students learn mathematics, the project raised awareness of research-based pedagogic approaches.

b) Teaching and learning practice: new modes of teaching and learning emerged (using HELM workbooks and CAA data banks) which were enthusiastically adopted across the sector.

c) Engineering education policy: policy-makers could see the value of acquiring HELM materials for staff and students to use, with or without adaptation.

Underpinning research

During the 1960s, Loughborough University (LU) began to research the teaching of mathematics to engineering students. By the mid-1990s, the so-called `mathematics problem' had become a major issue: academic staff had become increasingly concerned by the poor mathematical preparedness of incoming engineering undergraduates. The Mathematics Education Centre (MEC) was well placed to respond. Research from 1995 onwards elucidated the nature of the `mathematics problem' and evaluated innovatory approaches to tackle it, with publications documenting outcomes for practice [e.g. 3.1, 3.2, 3.3, 3.4]. The MEC's Open Learning Project (OLP) involving developmental practice, with on-going cycles of design of tasks and approaches which were trialled and evaluated, was informed by research and led to increasing knowledge in the design/developmental research process.

The OLP led to new learning resources in mathematics for engineering students: a significant outcome was the successful bid for a £250,000 HEFCE FDTL4 project known as HELM (Helping Engineers Learn Mathematics). LU researchers initiated and wrote the funding application and led a consortium of five universities (LU, Hull, Reading, Sunderland, UMIST). HELM ran from 2002-5 with further funding (£25,000) for a transferability extension in 2005-6 involving the additional universities of Leicester, Newcastle, Nottingham, Oxford Brookes, Portsmouth and Salford. Developmental research cycles continued under HELM, building on knowledge of the developmental research process gained in OLP. Through a rigorous developmental research programme, HELM transformed in quality, and expanded in scale and scope, the original OLP resources. Fifty workbooks and guides (each typically 50 pages) were created, and 6000 Computer-Aided Assessment (CAA) questions were designed.

Alongside these developmental research cycles, HELM researched the quality and fitness for purpose of the resources and their use, including academic issues and delivery modes (methods included feedback from students and staff through surveys and focus groups, reports from external evaluators and a wide network of triallists throughout the UK).

Key findings included:

(a) Students vary in their preferred modes of study and appreciate materials that suit their style [3.1, 3.2, 3.4]. As a result, HELM supports study at various levels, workbooks contain space for working, and questions incorporate hints.

(b) Case studies, based on engineering applications, were found to motivate students [3.3]. As a result, HELM workbooks include case studies and engineering examples throughout.

(c) Students value regular formative assessment, such as that provided by CAA, and are willing to try formative tests repeatedly until understanding has been developed [3.5]. HELM provides very extensive CAA materials.

(d) Students display various categorisable strategies that are indicators of correct or incorrect conceptions concerning statistical association, which have important consequences in teaching statistical topics involving independence and dependence [3.1].

(e) Few engineering students have studied mechanics before coming to university, leading to problems in its teaching and learning that HELM-style resources are able to address [3.6].

Key Researchers:

Les Mustoe (1969-2009 Senior Lecturer).
David Green (1973-2004, Reader in Mathematical & Statistical Education).
Martin Harrison (1974-2010, Senior Lecturer, then Visiting Fellow).
Joseph Ward (1977- date, Senior Lecturer).
Tony Croft (1996-date, Professor of Mathematics Education).
Carol Robinson (2002-date, Senior Lecturer).

References to the research

Papers:

All outputs listed in this section report significant, original and rigorous research. Each of outputs 1 to 5 was published in an international peer-reviewed journal, and has had lasting influence on the field. Outputs 1, 4 and 5 were published in journals ranked in the "INT1" category by the European Reference Index for the Humanities (ERIH). This is the highest category in the classification system, and represents "international publications with high visibility and influence among researchers in the various research domains in different countries, regularly cited all over the world". Outputs 2 and 3 were published in an "INT2" ranked journal (the second highest category of international journal). Output 6 was a commissioned report by the HEA's Engineering Subject Centre, informed by a supervised PhD student's research.

Major Funding:

G3.1. HEFCE FDTL4 (2002-5): £250,000 Helping Engineers Learn Mathematics project. [PI: Dr David Green]

G3.2. HEFCE FDTL4 (2005-6): £25,000 Transferability extension to the HELM project. [PIs: Dr Martin Harrison & Dr David Green]

Details of the impact

We are claiming significant impact on awareness and understanding, policy and practice.

Impact on the Mathematics Education of Engineering Undergraduates

Explicit aims of HELM (http://helm.lboro.ac.uk) were to share knowledge gained through developmental research and testing, and to promote wide use of HELM learning materials. These aims were very successfully achieved; the outcome was a comprehensive mathematics learning package for the first two years of engineering degrees. Nothing approaching this was available before or has been provided since. Unlike many projects HELM's impact has been both significant and lasting. Evidence is presented in three categories: (1) Three HELM Surveys, (2) Nine HELM Interviews, (3) BESTMATHS data analysis. These demonstrate HELM's impact has been substantial in both reach and significance in the period 2008-2013. Evidence summarised below is supported by a 25-page HELM USAGE report (5.1), two external reports (5.2, 5.3) and is corroborated by five external referees [5.4-5.8].

(1) HELM SURVEYS

Three surveys were conducted in 2012. Survey 1 targeted all UK Universities (excluding LU), and also Further Education (FE) Colleges providing Higher Education (HE). Survey 2 targeted the 58 UK respondents in 55 institutions who confirmed their use of HELM materials (49 in Universities, 9 in FE/HE Colleges). (In addition, there were a small number of overseas users, three in the Republic of Ireland.) Survey 3 targeted 17 UK respondents willing to provide additional detailed information. [5.1 sections 1, 4, 5, 6].

HESA data available (for 2011-12) for 42 UK HEIs containing staff known to be using HELM revealed that they had 48,710 FTE Engineering & Technology students, out of a sector-wide total of 102,710. [5.1 section 2].

Face-to-face interviews were undertaken with 9 academics in 5 HEIs using HELM materials. [5.1 section 3].

The data reported below may be significant underestimates: not all relevant departments could be identified or contacted, many did not respond or did not provide analysable data.

Practice

Survey 1 revealed that based on 58 responses 55 UK HE providers were confirmed active users of HELM materials and six others had HELM web links for their students. [5.1 sections1 and section 4 table 4:1].

Survey 1 also revealed that of 55 responses by staff, concerning the impact HELM had made in their institution, 27 cited `significant impact', 19 `some impact' and 7 `little or no impact', with 8 `don't know'. [5.1 section 4 tables 4:2, 4:5]

Survey 1 revealed that of 34 responses by staff who had used HELM and subsequently developed their own materials, 29 said their experience of HELM had helped them. [5.1 section 4 table 4:2].

Awareness and Understanding by Staff of HELM's Value

Survey 1 responses revealed very positive feedback on the quality of the materials and benefits to staff and students. Respondents cited `improved performance' or `improved retention' as significant impacts. When asked why they continued using HELM many said because the `quality is very high'. Some cited great saving in staff time (for developing materials). 100% of 59 respondents (including four not using HELM currently) said they would (or did) recommend HELM to others. [5.1 section 4 tables 4:2, 4:4].

Policy

Survey 2 revealed policy decisions in the use of HELM resources:

(a) Years 1-2 engineering undergraduates were the main HELM users,

(b) Students in many other disciplines used HELM - from Acoustics to Sport Coaching,

(c) Students across a wide range of levels used HELM - from Access to MSc.

[5.1 section 5].

Practice

Survey 3, covering 17 institutions, showed that almost all 34 core Workbooks were each being used by 60 to 180 students per institution. As a typical student studies five or six workbooks per year, this equates to about 725 students per institution per year, i.e. about 12,000 in total. [5.1 section 6].

(2) HELM INTERVIEWS

Interviews in 2012 with academics at five universities provided additional insights on HELM's significant and enduring impact. Comments recorded include:

• "HELM CAA questions are much appreciated.... I see an increasing role for HELM in our new Department." [Mechanical Engineering Principal Lecturer].
• "When HELM was adopted results went up dramatically.... feedback was tremendous... retention improved." [Aeronautical Engineering Senior Lecturer].
• "HELM is very helpful to PhD students acting as tutors." [Learning Development Officer]
• "Our aim is to implement HELM's CAA regime widely across the School & University." [Civil Engineering Senior Lecturer, responsible for University online support]
• "HELM saves a great deal of time when preparing a new course." [Mathematics HoD].

[5.1 section 3].

(3) BESTMATHS ANALYSIS

Policy

In 2002 the Royal Academy of Engineering made the policy decision to ask LU to develop and run a distance learning project based upon HELM's workbooks and CAA regime, for gap year students. The `BESTMATHS' programme has run each year since 2002 (rebranded as LUMEN (Loughborough University Mathematics for ENgineers) in 2010).

Practice

Data for academic years 2008/9 to 2011/12 show that in those four years 738 of the most able future engineers and scientists undertook this distance learning programme, with 90% successfully completing the course and receiving certificates. Participants have gone on to a range of universities, the five commonest being Cambridge, Imperial, Bristol, Durham and Bath. [5.1 section 7 table 7:1].

Awareness and Understanding

Feedback from students has been very positive, typical examples being: "The exercises are challenging, which is good. A brilliant opportunity to learn advanced mathematics." [Imperial]; "I have been impressed ... the material is easy to understand." [Cambridge]; " I am really enjoying the maths and find the workbooks brilliant." [Dundee]. [5.1 section 7 table 7:3].

Sources to corroborate the impact

The following sources of corroboration can be made available at request:

5.1. Report 1: HELM USAGE — available from Loughborough University.

5.2. Report 2: Michael Grove (July 2006) Investigating the Community-wide Impact of the FDTL4 Projects in Mathematics & Statistics. Report for the HEA Maths Stats & OR Network.

5.3. Report 3: HELM Transferability Project Final report:
SYMPOSIUM PAPERS, 28 June 2006 (FDTL4 project 48/02).
Within this: PAPER by Drumm, I. pp 51-59. C2.

5.4. Contact 1: Letter from the Director of the National HE STEM Programme, confirming the profound and lasting impact of OLP and HELM.

5.5. Contact 2: (Queen's University, Belfast) Letter confirming the substantial and continuing impact at QUB and elsewhere in Northern Ireland.

5.6. Contact 3: (UWE, Bristol) Letter from Principal lecturer confirming the value to her department of HELM materials at UWE.

5.7. Contact 4: (UWE, Bristol) Letter from Senior lecturer and drop-in coordinator confirming the value of HELM materials at UWE.

5.8. Contact 5: (Cambridge) Email from Engineering Admissions tutor confirming the value of LUMEN materials (which are based on HELM) for Cambridge undergraduates undertaking a gap year.