Summary of the impact

As a consequence of research carried out at the University of Stirling as set out in this case study, monitoring and evaluation of sport for development programmes has been enhanced:

• Organisations (NGOs/charities) in Africa and India have developed more systematic programmes for the monitoring and evaluation of their work;
• Project workers have acquired the knowledge and skills to be able to use quantitative and qualitative research techniques to design, collect, analyse and report evaluation findings;
• Monitoring and evaluation has become an integral component of sport-for-development work in the organisations that participated in the research;
• UK funding and sponsoring organisations have adapted approaches to supporting sport-for-development programmes and overseas organisations to include monitoring and evaluation.

Underpinning research

The research set out below was commissioned by a partnership consisting of International Development though Sport (IDS), Comic Relief and UK Sport. The aim of the research was to assess what contribution sport makes to the personal development and well being of disadvantaged children and young people.

Eight overseas organisations were involved in the study: six in Africa (EMIMA, Tanzania; KCCC; Uganda; The Kids League, Uganda; SCORE, South Africa; Don Bosco, Liberia; and YMCA Senegal, Senegal) and two in India (Magic Bus and Praajak Railway Children). All of the projects used sport and other activities as tools for the personal development of children and young people. The organisations worked with different groups of children and young people in different contexts with the main areas being: (i) personal development (self-efficacy and self-esteem); (ii) gender equality and women's empowerment; (iii) HIV and AIDS; and (iv) peer leader development.

This research was designed to build the capacity of these organisations to be able to undertake their own monitoring and evaluation and to be able to move away from having an external agency undertake evaluation on their behalf. The research provided a framework for the participating organisations to design their own monitoring and evaluation projects and provided the appropriate advice, support and training to develop the knowledge and skills necessary for staff to become proficient in undertaking monitoring and evaluation. Training workshops were delivered and included the following:

• Using logic models in programme design and monitoring and evaluation;
• Questionnaire design and administration;
• SPSS (Statistical Package for the Social Sciences) — Data file design, data entry and data processing;
• SPSS (Statistical Package for the Social Sciences) — Statistical analysis and chart building (Excel);
• Interviewing techniques;
• Interpreting and reporting evaluation findings.

As part of the training and support provided by this research, each organisation was provided with copies of the Sport-in-Development: A Monitoring and Evaluation Manual (Coalter 2006). This tool provided a foundation on which each organisation built their own monitoring and evaluation project.

Further on-going support was provided by way of telephone and email exchanges between the organisations and the evaluation team. Detailed comments on questionnaire design and content, on processing data using SPSS and reporting was provided to the nominated monitoring and evaluation project workers. In addition, visits were made to the project where advice and support was provided on-site, which was tailored to meet the needs of the organisation.

The University of Stirling Research Team

• Professor Fred Coalter, Chair of Sports Policy, School of Sport, University of Stirling — Fred was the Principal Investigator for the evaluation and was responsible for all phases of the evaluation (now retired).
• John Taylor, Research Fellow, School of Sport, University of Stirling — John supported the research organisations' design and analysis stages of their evaluations, provided training in the use of SPSS and contributed the content and production final report.

References to the research

Grant awarded to: Professor Fred Coalter
Title: Comic Relief/UK Sport joint venture
Sponsor: International Development through Sport (IDS) / UK Sport / Comic Relief
Period of grant: 01/12/2006-01/12/2009
Value of grant: £51,000

Details of the impact

The research impacted on the work of the African and Indian organisations delivering sports-for-development projects and the work of the funding and sponsoring bodies in the UK (IDS, Comic Relief and UK Sport).

African and Indian organisations

The study provided the organisations under investigation with a monitoring and evaluation framework and associated tools that strengthened their programmes of activity. This framework is set out in Coalter (2006) and is available for download via the sportanddev.org website, which is an international repository for work in this field. By adopting more robust systems of monitoring and evaluation, for which they received training and support, the organisations became more capable of assessing the quality and impact of their work. It also led to the strengthening of the design of their programmes, and the setting of appropriate outcomes. It has also led to improvement in other areas as noted by IDS/UK Sport and Comic Relief as they explained that "although challenging for some of the partners, others have taken the monitoring and evaluation skills acquired through the process into other areas of their work."

The research allowed project workers to assess the impact of their work on children and young people more thoroughly, which enabled them to amend or adjust their programmes to further improve their outcomes. Subsequently, several of the organisations involved in the research have appointed workers with responsibility for monitoring and evaluating of programmes.

Some of the organisations have continued to develop monitoring and evaluation programmes. For example, Magic Bus continues to expand its monitoring and evaluation function and is currently supporting others in the sector through further training opportunities based on the training the organisation received through this project. In December 2011 it delivered a three day workshop in `M&E and Project Cycle Management of S4D Programmes', designed to help delegates to understand and prepare M&E frameworks, enhance programme design and management and understand logic models.

Funding and Sponsoring Bodies in the UK

The research resulted in changes in the way that IDS supports partners and how it approaches research. In its summary document about the study, IDS stated:

"IDS has refined its strategy towards developing and initiating research projects and continues to aspire to be an all round `better partner'.
The impact study findings have alerted us to the importance of re-evaluating traditional assumptions about the nature and extent of expected impacts and the definition and measurement of success. We will support our partners to evaluate their interventions focusing on accurate beneficiary targeting, better understanding participant needs and the wider context." (IDS, 2010, p23)

This was re-iterated by the International Programme Manager, IDS/UK Sport when she said that "For IDS and UK Sport, it has informed our advice to partners on programme design and our programme strategy going forward."

This was further emphasised by IDS/UK Sport and Comic Relief as they explained that: "As a result of this project, our own understanding of the role of sport has progressed immensely. It has informed our own programmes and will continue to do so in the future."

The study has also helped raise Comic Relief's awareness of the importance of monitoring and evaluation:

"We acknowledge the need to support organisations to take the time to develop and fully understand their theory of change to allow for better setting and monitoring of outcomes for children and young people. We will assess the relevance of the `deficit model' across other programme areas and continue to challenge organisations in their assumptions about the groups they are working with and the issues they are addressing through their programmes We will support further learning and exchange between the sport for development and wider development sectors, so that organisations can learn from each other's good practice and understanding about how change is brought about." (IDS, 2010, p24)

The Sport for Change Manager for Comic Relief also explained how the research "has helped guide the design of our Sport for Change strategy and the assessment of proposals."

Sources to corroborate the impact

Written statements to corroborate the impact which are quoted from in the case study have been provided by the Sport for Change Manager at Comic Relief and the International Programme Manager for IDS/UK Sport. Contact details have been provided with the REF submission.
See also http://www.uksport.gov.uk/pages/research-and-publications/