Summary of the impact

The education and life chances of children are closely related to the performance of the school they attend. Researchers at the IEPR have been involved in research on school performance for some years now, and the research outlined below has made significant conceptual and empirical contributions to current debates about the effects of policy changes on secondary school performance. This is demonstrated by citations in evidence to the House of Commons Select Committees and national media. The main impact of our research has been on policy-makers, and those individuals whose lives have been affected by the work of policy-makers.

Underpinning research

(2003-2004) Davies, P.; Adnett, N.; Telhaj, S.; Coe, R.; and Hutton, D., Within school competition and pupil achievement (Report to the ESRC)
This research project was led by Staffordshire University. It investigated competitive processes in schooling through an analysis of data on pupil enrolments and attainment at subject level within secondary schools in England. The data collected from schools, which participated in data monitoring schemes managed by the University of Durham, were analysed. These data provided actual attainment relative to predicted attainment at subject level at the end of Key Stage 4 and Key Stage 5. Analysis of these quantitative data involved the examination of how pupils' subject choices at the ages of 14 and 16 were a consequence of previous teacher/department performance; if a relationship between subject enrolment and attainment persisted over time, and varied according to pupil type and institution; and whether an extension of pupil choice affected average attainment.

(2008) Pugh, G.; Mangan, J.; and Gray, J., Resources and attainment at Key Stage 4: Estimates from a dynamic methodology (Report to the DCSF)
This study was led by Staffordshire University and we worked in collaboration with the University of Cambridge. The research evaluated the performance effects of increasing expenditure at Key Stage 4 by exploiting the time series dimension of school-level datasets through dynamic panel analysis. The findings of this quantitative study provided not only the general effect on pupil outcomes of increased funding, but also on the particular effects on pupil outcomes in different types of schools.

References to the research

The outputs in this section are based on externally-funded research projects. The quality of the research is evident in the fact that the project proposals were sent out for peer review by the funders. Moreover, these outputs were published in high impact refereed academic journals which adhere to the practice of blind peer review by at least two reviewers.

Details of the impact

The conceptual contributions reflect pivotal contributions on positional competition in schooling markets and the presence of co-opetition in schooling markets. The empirical studies investigate the relationship between resources and school performance; changes in school performance over time and the nature and effects of within-school competition.

(2004) Davies, P.; Adnett, N.; Telhaj, S.; Coe, R.; and Hutton, D., Within school competition and pupil achievement (ESRC)
http://www.esrc.ac.uk/my-esrc/grants/RES-000-22-0090/outputs/Read/80a4e8cc-f43f-4487-81c7-90f683d0a48a
This project was widely cited in national debates on resourcing and governance of UK schools:
national newspapers (e.g. Guardian), BBC (Radio 4), THES, and international interest from Australia, which has led to further research linked to education policy. It was cited in House of Commons, Public Administration Select Committee, Written Evidence, 9 December 2004, HC-49-11, p.18.

The following IEPR Working Paper is based on this project:

• Telhaj, S.; Hutton, D.; Davies, P.; Adnett, N.; and Coe, R. (2004) Competition within schools: Representativeness of Yellis sample schools in a study of subject enrolment of 14-16 year olds, http://www.staffs.ac.uk/schools/business/iepr/docs/Working-paper11.pdf

The paper examines the issue of sample representativeness which is critical in impact evaluations. Biases can arise from a sample which does not represent the participant population, or a non-participant sample which does not represent the participant population. Representativeness expresses the degree to which sample data accurately and precisely represent a characteristic of a population's parameter variations at a sampling point. Representativeness is a qualitative parameter which is most concerned with the proper design of the sampling programme. The representativeness criterion is best satisfied by making certain that sampling locations are selected properly and a sufficient number of samples are collected:

The Belgian Healthcare Knowledge Centre is a federal institution that produces studies and reports to advise and educate policy-makers when deciding on healthcare and health insurance. We were sent an email by them which stated that one of their on-going projects in this field aims at assessing the representativeness of a sample of the Belgian hospitalisations at a national level. They informed us that their methodology refers to the above paper, and asked for permission to cite this Working Paper as a reference.

(2008) Pugh, G.; Mangan, J.; and Gray, J.; Resources and attainment at Key Stage 4: Estimates from a dynamic methodology (DCSF)
The above report was referred to in Parliamentary Questions in the Summer of 2009. It was considered widely in the media: Radio 4 Today programme (07/09/07), five national newspapers and the Times Educational Supplement during that week. The following paper was published from the above project:

• Pugh, G, Mangan, J. and Gray, J. (2011) Do increased resources increase educational attainment during a period of rising expenditure? Evidence from English secondary schools using a dynamic panel analysis, British Educational Research Journal, 37(1): 163-189.

An earlier version of this paper was presented in a keynote symposium at British Education Research Association conference, where the discussant was a senior representative from the Government Economic Service. The published paper was later commended at the British Educational Research Association conference in 2012. Furthermore, it was the instigator of contact which led to a project between the research team and the Department of Education, New South Wales, Australia, to assess the impact of school expenditure on performance.

Sources to corroborate the impact

The ESRC Report and the two related articles (1, 2, 3 & 4 in section 3 above)

1. Davies, P.; Adnett, N.; Telhaj, S.; Coe, R.; and Hutton, D. (2004) Within school competition and pupil achievement http://www.esrc.ac.uk/my-esrc/grants/RES-000-22-0090/outputs/Read/ 80a4e8cc-f43f-4487-81c7-90f683d0a48a
2. House of Commons, Public Administration Select Committee, Written Evidence, 9 December 2004, HC-49-11, p.18.

The DCSF Report and the related article (5 in section 3 above)

3. Pugh, G.; Mangan, J.; and Gray, J. (2008) Resources and attainment at Key Stage 4: Estimates from a dynamic methodology
   https://www.education.gov.uk/publications/standard/publicationDetail/Page1/DCSF-RR056
4. Impact in Belgium: Belgian Healthcare Knowledge Centre
5. Impact in Australia: Manager, Statistical Performance Reporting, Finance Directorate, NSW Department of Education and Communities, Sydney, NSW 2000, Australia (Testimonial available)
6. The commendation at the British Educational Research Association Conference in 2012 was in conjunction with the publisher (Routledge). (Certificate and letter from Chair of Editorial Board available).
7. The Times Educational Supplement, Friday 7^th September 2007, p18 (copy available).