Summary of the impact

Robertson's research in learning via game-authoring demonstrated sustainable success in bringing programming into schools. This subsequently catalysed (i) creation of a whole new suite of qualifications by the Scottish Qualifications Authority; (ii) inclusion of the requirement for games development experience in the new Scottish Technologies Curriculum; (iii) the development of `Computing at Schools Scotland' in terms of its successful annual conference and provision of specialist CPD for Computing teachers. Having so far influenced the education of over 500,000 Scottish pupils, Robertson's work is also cited in the key document underpinning Obama's programme for STEM training in the US via game-authoring.

Underpinning research

Schmidt (of Google CEO), in his 2011 McTaggart address, said the UK was "throwing away your great computer heritage" by failing to teach programming in schools. This problem is well-known among Computer Science and related professionals and is a frequent complaint among industry recruiters.

As stated later in the Royal Society Report , "Shut Down or Restart" (2012) http://royalsociety.org/education/policy/computing-in-schools/report/:

"The current delivery of Computing education in many UK schools is highly unsatisfactory [...] mainly because:

a .the current national curriculum in ICT can be very broadly interpreted and may be reduced to the lowest level where non specialist teachers have to deliver it

b. there is a shortage of teachers who are able to teach beyond basic digital literacy

c. there is a lack of continuing professional development for teachers of Computing

d. features of school infrastructure inhibit effective teaching of Computing."

Robertson's work, beginning in 2003, addressed these key barriers. She developed an approach to bringing computing into the classroom that was designed to be engaging and easy to get to grips with for teachers of arbitrary backgrounds, while engaging pupils, and being maximally flexible in terms of how lesson plans might be organized. This showed that sustained computing experience via game-authoring could benefit attainment and skills elsewhere in the curriculum. In this way, Schools and teachers could accept the `start-up' costs, and naturally sustain the associated teaching hours on the basis of demonstrable improvement in their pupils' overall attainment.

Robertson won an EPSRC grant to explore these ideas (EP/D064546/2: Supporting creativity in computer game authoring) which operated at HWCS between 10/06 and12/08, staffed by Cathrin Howells (recruited from the Scottish national agency, Learning & Teaching Scotland). The idea of this project was to develop educational software to support aspects of the creative process for game making (particularly ideas generation and peer evaluation). The research outcomes of this project were findings indicating how the use of the software in classrooms facilitated a set of "successful learner" skills embedded in the Scottish Curriculum for Excellence [1], and an in depth study of gender differences around the creative process of game making [2]. Robertson won a follow on EPSRC `Partnership in Public Engagement' project EP/G062641/1, entitled `Making Games in Schools' (MGIS), which operated at HWCS between 05/2009 and 10/2010. The idea of this was to transfer the `Adventure Author' (AA) software (initially developed as part of the "Supporting creativity in computer game authoring' project), along with pedagogical findings relating to supporting the creative process, into classrooms in a sustainable way.. The research outcomes of this project were a methodology for embedding innovative technology in schools through training teachers, and a deeper understanding of how children's attitudes towards computer science are influenced by taking part in a game making project [3,4]. In parallel, Robertson regularly engaged teachers and other education professionals, particularly regarding the potential benefits of learning-via-game authoring. For example in May 2007 she led a game- authoring workshop (organised by http://www.educationscotland.gov.uk/ - then `Learning Teaching Scotland' - LTS), and in September 2007 she presented the latest findings from EP/D064546/2 at the LTS-organised `Scottish Learning Festival'. She also regularly engaged in the national debate via her blog and other press (e.g.:http://www.tes.co.uk/article.aoo cspx?storycode=2625661, from paragraph 20).

References to the research

Grants
EP/D064546/2 Supporting Creativity In Computer Game Authoring 1 October 2006 - 31 December 2008 £203,775

EP/G062641/1 Making Games in Schools 1 May 2009 - 30 April 2011 £99,613

Details of the impact

Impact in the classroom 2009-13
As of June 2013, a total of 2,566 school pupils in 14 Scottish schools have taken lessons using Adventure Author, spending 40,325 hours with the software. This is supported by associated lesson plans which allow fully flexible delivery. The majority of this activity represents permanent adoption by schools beyond Robertson's MGIS project'; That is, impact was achieved initially as part of MGIS, but schools have now adopted the materials and software in their normal routine. This approx. 16 hours of learning per pupil replaced previous planned national curriculum activity in their timetables, bringing programming into these schools for the first time. (see section 5). This contrasts sharply (see comprehensive survey in the 2012 Royal Society Report), with previously an absence of computational thinking taught in Scottish schools, with occasional exceptions that tend to be one-off events or optional extra-curricular activities. Schools currently teaching with Adventure Author plan to continue in the 2013/14 school year and beyond, in particular, since it will make it much easier for them to serve the needs of the new curriculum.

Teachers' evaluations have reported deep levels of engagement and commitment from the learners. Teachers were repeatedly surprised by how well the children did, and how enthused, engaged, creative and excited they were, working together, solving problems and sharing their expertise. Unexpected children came to the fore - the less able, the disaffected - and the overall quality of both game-based and written work is mentioned, with teachers noting the strong connection between the language work and the game software, e.g. the support for logical thinking. A HMI inspection at St Augustine's High School in September 2009 cited MGIS in their report, as an example of good practice in interdisciplinary learning "Linking learning across subjects through fantasy game modelling".

Further Impact in the classroom via impact on the Scottish Curriculum
Robertson's research provided hard evidence which was a key influence in the creation of a whole suite of new National Progression Awards (NPAs) in Games Development (w.e.f. academic year 2010/11 - http://www.sqa.org.uk/files_ccc/G9RP44,%20G9RR45%20and%20G9RT46.pdf) Further, Robertson's work with teachers commissioned by Learning and Teaching Scotland in the consultative period for the Curriculum for Excellence Technology Outcomes (between 2007 and 2009) contributed to the inclusion of games experience as a mandatory requirement for all pupils from Primary to year 3 of senior school inclusive, and, in particular, games development experience for all pupils in years 2 and 3 of senior school (ages 12 and 13), similarly w.e.f. 2010/11 (see Curriculum document in section 5). As a result of the latter, by Summer 2013, over 500,000 Scottish pupils will have benefited, with approx. 200,000 gaining games development experience. Meanwhile, in 2012/13, 307 pupils sat the new computer games development NPAs.

Creating games is also recommended as a mechanism for learning throughout the support notes for teachers in the new National 4 and 5 qualifications (approximately equivalent to GCSE and being offered for the first time in 2013/14) and for the Higher and Advanced Higher Qualifications in Computing Science (taken in the 5^th and 6^th year of senior school).

Further Impact on Scottish Computing Educators
One outcome of Robertson's MGIS project was her organisation (in 2010) of "Look to the Future", a conference for Scottish Computing Educators, partly supported with a $10k award from Google (see REF3a part (b)), in which Robertson involved representatives from what later became `Computing at Schools Scotland' (CASS - http://www.casscotland.org.uk/), then a fledgling organization. The organisation and success of this event had a significant influence on the growth of CASS. CASS has since grown rapidly, organised two further conferences for Scottish Computing Educators, now has ~350 members (over half of the 600 Computing teachers in Scotland), and has won £400k from the Scottish Government to support continuing professional development.

International Influence. Educators worldwide follow the Adventure Author blog (27% of page views are from the US), which itself led to two Australian teachers collaborating with Scottish teachers in the MGIS project. Robertson's views on computing education are published regularly in the Communications of the ACM, feeding international policy debate.

Robertson's paper on educational game authoring (ref. [1] in section 3) informed the US National STEM Video Game Challenge, launched by President Obama in September 2010 — http://stemchallenge.org/about/why-games/. This was part of a strategic policy " ... to motivate STEM learning among America's youth by tapping into students' natural passion for playing and making video games." [1] was cited twice in the description of the challenge as evidence that making games is a highly engaging way to learn STEM skills. In addition, influence on teacher training worldwide is evident in [1]'s adoption by the ACM Computer Science Teachers Association reading list.

Sources to corroborate the impact

http://royalsociety.org/education/policy/computing-in-schools/report/
(Royal Society report on the state of UK CS education)

http://cacm.acm.org/blogs/blog-cacm/152857-girls-cant-program-in-their-heads-gender-and-games-in-the-computing-classroom/fulltext
(Blog article written for Communications of the ACM blog by Dr Robertson about results of this project. ACM is the worlds' most prestigious organisation for computing professionals.)

http://www.educationscotland.gov.uk/Images/StAugustinesRCHighScIns20091116_tcm4-701107.pdf
(HMI report for St Augustine's School which mentions Making Games in Schools.)

ACM Computer Science Teachers Association: http://csta.acm.org/Curriculum/sub/CurrFiles/K-8CSReadingList_Research.pdf

A corroborating statement is available from Nesta's Digital Education Manager for Scotland (also co-Chair of Computing at School Scotland), This concerns the key importance of Robertson's work in (i) creation of New Qualifications in Games Development (National Progression Awards), (ii) the design of mandatory requirements in the new Curriculum for Excellence for all school pupils to have experience of computer games development (iii) the role of Robertson's work in shaping the annual conference of Computing at School Scotland (CASS), and in CASS' provision of CPD opportunities for Computing teachers in Scotland.

A corroborating statement is available from a Qualifications Manager at the Scottish Qualifications Authority (SQA). This further confirms how Robertson's work contributed to the creation of a whole suite of new National Progression awards and further confirms how it influenced the new requirements for all pupils to gain experience in games development. This statement also confirms the recommendations re games development in teachers' support notes relating to new National Qualifications in Computer Science.

A corroborating statement is available from a Policy Manager at the Scottish Qualifications Authority (SQA). This further discusses the key importance of Robertson's work in leading to the recommendations for learning via games development appearing throughout the support notes for teachers in the new National 4 and 5 qualifications (being offered for the first time in the 2013/14 academic year) and for the new Higher and Advanced Higher Qualifications in Computing Science (which will be offered for the first time in 2014/15 and 2015/16 respectively).

Recommendations for learning via games development appearing throughout the support notes for teachers in the new National qualifications in Computing Science are evident here: http://www.sqa.org.uk/files_ccc/CfE_CourseUnitSupportNotes_N5_Technologies_ComputingScience.pdf. ... as well as similar documents for other Levels.

The requirements for games and games development experience are expressed as required outcomes TCH 0-09a—TCH 3-09a in the Curriculum document pertaining to Technologies: http://www.educationscotland.gov.uk/Images/technologies_experiences_outcomes_tcm4-539894.pdf

Scottish School roll statistics, by stage and year, are available from the Scottish Government via: http://www.scotland.gov.uk/Topics/Statistics/Browse/School-Education/ ; e.g.: in 2012, 370,839 pupils were in primary schools, and 158,567 were in the first three years of secondary school.