Summary of the impact

Educational effectiveness and improvement research by the University of Southampton School of Education has contributed significantly to the design and implementation of educational policy and practice at both national and international levels. Impact has been predominantly in the area of policy, but the School's ground-breaking research has also shown the effects of (and practice within) `good' schools and has pioneered novel approaches to school improvement, school organisation and the use of data in schools. The Educational Effectiveness and Improvement Group has helped establish the International Congress for School Effectiveness and Improvement (ICSEI) and given it a global reach; its research has directly informed policy implementation through academy chains, schools and local authorities in the UK generally and in Wales in particular, and internationally in the US, China, Sweden, Cyprus and Chile. The School's worldwide reach is among the most widespread in Education.

Underpinning research

Context: In the last two decades, the school effectiveness and improvement movement, organised internationally through ICSEI (co-founded by Reynolds and upon whose board Muijs and Downey serve) and in the US by American Education Research Association (AERA)'s School Effectiveness & School Improvement Special Interest Group (chaired by Muijs) has successfully argued for and demonstrated the importance of schools in determining children's educational and social achievement. In that period, David Reynolds, Daniel Muijs, Anthony Kelly and Chris Downey have together generated some of the best known and highly regarded research in this area.

Reynolds's work in the period 1993-2013 on the characteristics of effective schools and added value, and on effective ways of helping schools improve, has produced in the period 2008-2013 clear blueprints for school leadership, school culture and school expectations, and how these can be affected by external improvement programmes.

Muijs's work in the period 1993-2013 has focused on the (historically neglected) area of effective teachers and their behaviours, attributes and structures, moving later into the design and evaluation of professional development programmes based directly on his research findings. His work also focuses on networking and collaboration between schools, Federations, Best Practice Networks, Schools Facing Challenging Circumstances, and Extended Schools.

Kelly and Downey's work in this period has analysed how pupil performance data can be utilised to improve schools by making school management more `intelligent' about the functioning of classrooms. Their research has improved the professional use of data in schools and led to more improvement-oriented practitioners at the level of both school and Local Authority. It has also been used in the area of equity.

The appointment of Reynolds and Muijs in 2010 to join Kelly (appointed 2001) and Downey (appointed RA in 2006 and lecturer in 2008) represented a major investment by the University of Southampton in expanding research and building impact in educational effectiveness and improvement; specifically, to promote the dissemination to practitioners and policy-makers of the research conducted by the Educational Effectiveness and Improvement Group, and to improve `policy and practice reach' and impact on children's life-chances nationally and internationally.

Research funding for Reynolds, Muijs, Kelly and Downey in the period 1993-2013 has totalled approx. £6m in grants (Reynolds — £2m; Muijs — £2.5m; Kelly & Downey — £1.5m) and has been widely disseminated in books and through peer-reviewed journals papers in, for example, British Educational Research Journal, School Effectiveness and School Improvement, the Oxford Review of Education, Educational Research, and the Journal of Education Policy.

The underpinning research: In the last three years alone, the group's innovative research programme in this field has included: Kelly and Downey's projects with local authorities on the use of pupil-level data to inform the practice of educational planning, and a national survey (funded by CfBT) of teachers' attitudes to the use of attainment and progress data, which has been cited in government policy documents [3.1]; Downey, Muijs and Kelly's (2010-11) project (funded by Fischer Family Trust) on trends in pupil attainment / pupil progress observed in the National Pupil Database, and on the take-up of the EBacc qualification [3.2]; Kelly's work for the States of Jersey (2012-13) on school effectiveness using innovative `capability' approaches and new metrics for gauging equity to guide schooling policy on the island [3.3]; Reynolds's project (2010-12) on High Reliability Schools and Within School Variation [3.4], using novel, high-effectiveness methods of school improvement modelled on `failure-free' organisations; Muijs's project (2008-10), on the impact of the government's `TeachFirst' initiative (funded by Goldman Sachs) focusing in an innovative way on the impact of pedagogy in schools in challenging circumstances [3.5], and his work on modelling collaborative improvement and school-to-school partnerships in Hampshire schools [3.6]; and Reynolds and Kelly's (2013) research for the Royal Society's `Vision Initiative' on accountability and its effect on STEM subjects. Clearly, the Educational Effectiveness and Improvement Group's research, coming as a result of close collaborations between the 4 colleagues and supported by some 6 research assistants at any given time, represents arguably the most intensive and successful `education effectiveness' research grouping in the UK.

References to the research

Selection of co-ordinated grants awarded (figures rounded)

1. Kelly & Downey, 2006 - 2008, Use of data to inform school improvement, ESRC, £96,000

2. Kelly (=P.I.) et al., 2007 - 2008, Value for money in schools, Audit Commission, £20,000

3. Downey & Kelly, 2009 - 2010 Investigating post-16 progression and learner engagement, Dorset Children's Services, £34,000

4. Kelly & Downey, 2009, Data dictatorship and data democracy: understanding professional attitudes to the use of pupil performance data in schools, CfBT Education Trust, £28,000

5. Downey, 2009 - 2011, Evaluation of the Poole Schools Project, Poole Borough Council & Bournemouth and Poole Primary Care Trust, £20,000

6. Muijs, 2008 - 2010, Evaluation of the maximum impact programme for Teach First, Goldman Sachs Foundation, £98,000

7. Reynolds, 2010 - 2011, The Within-School Variation Project, TDA, £19,500

8. Downey, Muijs & Kelly, 2010 - 2011, Analysis and reporting of trends in pupil attainment and progress observed in the National Pupil Database, Fischer Family Trust (FFT), £52,000

9. Muijs, 2011 - 2012, Evaluation of S2S partnerships, Southampton City Council, £6,000

10. Kelly, 2012 - 2013, Equity in Pupil Attainment on Jersey, States of Jersey & Jersey Community Relations Trust, £77,000 [incl. RA salary]

11. Kelly (=P.I.) et al., 2013 - 2015, Engaging university researchers with secondary school students and teachers, RCUK, £187,000

12. Reynolds, & Kelly, 2013, Accountability and the meaning of success in education systems, The Royal Society, £13,500

Details of the impact

Reynolds is one of the world's leading school effectiveness researchers and is currently (2011-present) the Senior Educational Policy Adviser to the Welsh Government, responsible for applying his research to the Welsh (and from Wales to the international) policy-making and practitioner community with a view to raising Wales's showing in PISA [5.1]. Reynolds is also, through his membership of the Board of E-ACT and as Chair of its Educational Committee, using his research to impact directly on 31 schools with 20,000 children and 3,000 staff [5.2]. In the US, his High-Reliability School research, which has been disseminated in the leading American practitioner journal Phi Delta Kappan and at multiple practitioner and policy-maker conferences, has led to the current construction of a US national cadre of `High-Reliability School Districts' [5.2].

Muijs's research on the effectiveness of `TeachFirst' significantly and directly influenced the decision to expand that programme. The DfE document, `The Case for Change', informed the 2010 White Paper and cited his research, as a result of which the government decided to "more than double the size of TeachFirst and extend it across the country and into primary schools" [5.3]. Muijs's research on the impact of collaboration between schools was also incorporated into, and cited by, the NCSL's `Guide to Federations', produced for all headteachers in the UK [5.3]. In Europe, he serves on the committee that advises (and reports directly to) the Education Minister of the Flemish Government on reforming teacher training [5.4].

Kelly and Downey's research has made a direct contribution to changing the practice of professionals by encouraging the use of data by local authorities and has been regularly cited in policy-making circles. The research, described by one LA as "invaluable", enabled Dorset, for example, to develop tools to survey children's well-being, saving the Authority more than £100,000 in direct costs (based on the rates that were charged at the time for the Pupil Attitudes to School and Self (PASS) survey from `W3 Insights') [5.5]. Kelly was called as an expert witness before the House of Commons Select Committee on Education (10/11/2010) and also serves on the Parliamentary Group on School Governance since its inception in 2011, both of which have materially affected the way schools are governed and inspected in England and Wales. Kelly's research has also been cited in Parliamentary Debates (e.g. in January 2013, Lord Clancarty cited Kelly when he opened the debate in the House of Lords opposing the government's EBacc Certificate proposals, which Minister Gove subsequently withdrew) [5.6].

Downey, Muijs and Kelly's FFT-funded study developed new quantitative metrics for optimising data-use by practitioners, which has had a direct effect on the content of FFT's service to schools and LAs throughout England and Wales [5.7]. Kelly's recent project (2012-13) on Jersey has drawn on this theoretical work on equity measurement, and the marrying of school effectiveness and Sen's (Nobel prize-winning) `capability' approaches, and is impacting on schooling on the island.

Findings by Kelly and Downey about the extent to which data can be used as a driver for reform in schools was cited in the DFE's `Impact of Research' policy review document by Goldacre and Plant (2013), the effect of which has been to steer government policy towards greater reliance on robust longitudinal and RCT-trialled practice [5.8].

The group's extensive programme of dissemination to practitioner and policy-maker audiences through multiple conference presentations is exemplified by the influential (`State of the Art') reviews by Muijs and Reynolds commissioned by ICSEI in 2011 and 2012. ICSEI is one of the few international organisations in education that links policy-makers, practitioners and researchers (in Cyprus in 2011, 43% of the 500+ delegates attending the annual conference were leading practitioners and high-ranking policy-makers; in Malmo in 2012, the figure was 46% of 600+ attendees). In 2011, Reynolds presented his research to 1,250 practitioners at the Chinese National Association for School Effectiveness and Improvement organised by Shenyang Normal University; and in 2013, Muijs presented his research to more than 1,500 headteachers in Chile and the Chilean Minister of Education. All locations show evidence of take-up of `effective schools' practices and the contribution "had a great impact on teacher training and on the formation of educational policy". [5.9].

Sources to corroborate the impact

5.1 See letter dated 19 Mar 2013 from Minister for Education and Skills, Welsh Government.

5.2 See letter from Director General of E-Act and see McRel Education summit for innovative education at: http://www.mcrel.org/products-and-services

5.3 See Department for Education (2010) The Importance of Teaching. The Schools White Paper 2010. Available online at: https://www.education.gov.uk/publications/eOrderingDownload/CM-7980.pdf and see National College for Leadership of Schools and Children's Services (2010). A National College Guide to Federations. Nottingham: NCSL.

5.4 See clip from TV news in Belgium at:
http://www.deredactie.be/cm/vrtnieuws/videozone/programmas/journaal/2.30294;
and from the two broadsheet newspapers that exist in Flanders:
http://www.standaard.be/cnt/dmf20131005_00776315;
http://www.demorgen.be/dm/nl/1344/Onderwijs/article/detail/1717443/2013/10/05/Smet-richt-6-beleidsgroepen-voor-lerarenopleidingen-op.dhtml;
and a longer piece from the main news magazine and from the Flemish equivalent of the TES: http://www.knack.be/nieuws/belgie/strengere-criteria-voor-aspirant-leraars/article-normal-109907.html; http://www.klasse.be/leraren/38054/lerarenopleiding-krijgt-slecht-rapport/#.UmElPVBwqSo (all last accessed on 18/10/2013)

5.5 Corroborating contact: Former Head of Learning and School Improvement, Learning and School Improvement Service, Dorset County Council.

5.6 For verification, see House of Commons Select Committee on Education: The role & future of Ofsted, 10/11/10. Available online at:
http://www.publications.parliament.uk/pa/cm201011/cmselect/cmeduc/570/10111001.htm and see Debate on Education: English Baccalaureate Certificate. Opened by the Earl of Clancarty on 14 January 2013. Available online at:
http://www.parliamentlive.tv/Main/Player.aspx?meetingId=12184 [starts @ 4:50]

5.7 Fischer Family trust (FFT) is an independent, not-for-profit charity. Currently, all LAs in England and Wales subscribe to FFT, and virtually all schools in England and Wales access FFT data either online or using FFT databases.

5.8 For verification, see Goldacre, B. and Plant, R. (2013) Impact of Research. Department for Education Analytical Review — Executive Summary (London: DfE) p.2.

5.9 For verification, please contact Professor Kyriakides, University of Cyprus, School of Education.