Summary of the impact

There is a growing demand for evidence of the impact that non governmental and private businesses are having. Research findings have led to a range of practical and policy developments related to encouraging organisations to measure their impact and use it both for their own development and to access more resources. This research has resulted in social impact measurement being introduced to many organisations and an improvement in the tools used. Research insights into how social impact can best be measured have led to changes in the practices of charities such as Citizens Advice and the use of a measurement tool by over 200 smaller organisations.

Underpinning research

Research on social accounting and social audit built on previous work that started in 1999 and explored the approaches to measuring social impacts, the indicators used, and the experiences of organisations looking to measure such impact (Lyon et al, 2002; Evans and Syrett, 2007). Evans and Syrett (2007) reported on a major international comparative project to explore how social impact can be demonstrated in the social economy, with a focus on less tangible aspects such as social capital. This research showed how a social accounting methodology that examines the impacts of each organisation can demonstrate impacts on jobs and services but also in terms of tackling empowerment, democratic participation and social exclusion. Further work by Lyon and Evans went beyond the social economy to examine the impact of small firms in deprived areas and the development of a set of indicators for measurement (Lyon et al, 2002; Lyon and Bertotti, 2007).

Social impact measurement research was developed through collaborations involving Lyon within the ESRC funded Third Sector Research Centre (TSRC) (2008-2013). Lyon co-authored studies with other TSRC academics based at Southampton and Birmingham Universities. Research on how charities and enterprises are measuring their impact explored the ways in which social enterprises and charities measured their impact and the motivations that drive them (Lyon and Arvidson, 2011; Arvidson and Lyon, 2013). This was the first study to explore the perceptions of organisations rather than focusing only on the types of tools being promoted to measure impact. Further research examined particular tools for social impact with recommendations for improvements to social accounting methods with a particular emphasis on the Social Return on Investment methodology that has widespread use within the sector (Arvidson, Lyon, McKay and Moro, 2013). The research concluded that organisations have considerable discretion on how they use these tools and there are considerable dangers of comparing the impact results of one organisation with another. However, these approaches continue to be a useful learning tool.

Previous research reflected theoretically on the nature of impact measurement as an institutional form or focused on the use and promotion of particular tools, rather than adopt critical perspectives on how the tools are used, misused and adapted (Arvidson and Lyon, 2013). Research at Middlesex examined the experiences of third sector organisations in measuring impact and their use of impact evidence, building on critical analysis and initial empirical work to demonstrate the central importance of these factors. Following this, we supported the think-tank New Philanthropy Capital in conducting a large survey on this topic, that demonstrated the different motivations for measuring impact (for learning and marketing) as well as for satisfying funder requirements.

References to the research

The quality of these research outputs is assured by all research having undergone peer review in both high ranking refereed journals and well regarded edited volumes. There has been quality assurance too from project steering groups. All ESRC funded Third Sector Research Centre activity is overseen by a steering group chaired by Prof Ken Peattie (Cardiff University) and comprising leading academics (including international scholars), representatives of government bodies and key social enterprise and third sector bodies. All TSRC working paper series publications underwent a peer preview process by two established academics.

Research funding for Middlesex University

The Contribution of Social Capital In the Social Economy to Local Economic Development in Western Europe (CONSCISE), Awarded by: European Commission (Framework V), 1999-2002, (£510,000).

Measuring Enterprise Impacts Upon Deprived Areas. (DTI Small Business Service) 2002 (£28,000)

The Contribution of Social Enterprises to Rural Economies, Department for Food, Environment and Rural Affairs (DEFRA) 2003 (£27,134)

Social Impact Stream of the ESRC/OTS Third Sector Research Centre, Led by Birmingham University, 2008-13

Social Impact Measurement - Public And Third Sector Commissioning, Procurement, Social Impact Measurement And Social Enterprise Development, funded by EEDA, 2009-2010 (£21,055)

Details of the impact

MU research on how organisations can and are measuring impact has led to our involvement in a range of programmes that have stimulated the uptake of social impact measurement and the improvement of existing tools. These include:

2000-2007: Building on initial work, we provided consultancy to the Department of Trade and Industry (DTI) and DEFRA relating to the impact of social enterprise. An influential piece of MU work on the impact of enterprise in deprived areas for DTI was used extensively in the design and initial documents of the Local Enterprise Growth Initiative.

2008-9: We were asked to evaluate the Social Enterprise Investment Fund (a £100m programme funded by the Department of Health) and research used to influence policy.

2010-11: There was a growing appetite from organisations to use MU research themselves, with training being provided to over 200 organisations based on tools developed from the research findings. More intensive engagement with Citizen's Advice used the research to successfully develop their strategies.

2012-13: The work on supporting charities and social ventures to measure their impact has continued, with development of web based tools and research being used in a two year Knowledge Transfer Partnership on growth strategies for a charity, The Holy Cross Centre Trust.

Influencing Policy

The research on Social Return on Investment (initially published as a Third Sector Research Centre working paper in 2010 and later published as Arvidson, Lyon, McKay and Moro, 2013) was used to advise a range of funders and policy managers about the types of impact evaluation tools that should be promoted and the limitations of these approaches. The Social Return on Investment approach had been selected as the single approach to be promoted by the Cabinet Office's Office of the Third Sector (later renamed the Office for Civil Society). MU was on the advisory panel of the Cabinet Office's Measuring Social Value Project and advised the Cabinet Office on what approaches are most suitable. A senior official of the Office for Civil Society reported: `we found the meeting very useful and timely in terms of our thinking on the next steps for Social Return on Investment and how it can feed into the government priorities'. The research was quoted in a key document from the Cabinet Office on Social Investment (1; 2).

Subsequent to the briefing based on this research (Arvidson et al, 2013) and further studies by DEMOS, there was a policy change that replaced the emphasis on promoting SROI as a single approach to measuring impact and recognised the need for a range of approaches. This research received much attention in the media with articles in the Guardian online and in the Third Sector trade press (3; 4). The briefing paper on the subject was downloaded over 3,500 times. It also led to further policy briefings with the Big Lottery Fund (24/10/2010), Department for International Development (5/11/2012) and advice given to Big Society Capital (13/3/2013) (5). Further research on impact tools and their use by commissioners was used to brief the Cabinet Office (19/8/2013). Research on how charities are using Impact Measurement was also drawn on by the Greater London Authority/NESTA in setting up Project Oracle - a programme to build the evidence base on what works in youth policy (6).

Influencing Practice

Results from the study of how organisations measure social impact (Lyon and Arvidson, 2011) were used to initiate a KTP that developed ways to support organisations to select an appropriate means to measure their impact (7). This resulted in 14 courses run by the KTP partner (The Guild), attended by over 200 organisations, and 1,064 organisations have placed orders for the guide. The research led to MU winning a competitive contract (£12,000) to develop impact assessment approaches for Citizens Advice and an address to their AGM. MU also won a competitive tender commissioned to develop a website for measuring impact for the Big Lottery Fund and Plunkett Foundation (8).This targeted many thousands of smaller organisations and social enterprises.

This research also laid the basis for the Inspiring Impact programme led by New Philanthropy Capital, which brought together impact measurement specialists across the UK to discuss forms of shared measurement. MU research was quoted extensively in the launch document for this programme (9). The research was also used by New Philanthropy Capital in developing the Data Lab Project and Middlesex University was subsequently funded to provide further support in setting up units in different government departments to allow charities to use public sector administrative data to assess the impact of interventionism (10).

This work has led to a number of other impact assessment studies for specific organisations that have built on our analysis of impact measurement approaches and have been used to shape the future direction of the organisations. They include: Evaluation of the School for Social Entrepreneurs programmes (2013-15: £104,000); Evaluation of the New Philanthropy Capital Data Lab Project (2013-16: £54,000); Right to Request Programme, commissioned by the SSE and Department of Health (2011: £5,000); Social Enterprise Investment Fund Evaluation, commissioned by the Department of Health with Birmingham University, Oxford University and Shared Intelligence Ltd. (2009-2011; £400,000, with £53,000 to MU); Evaluation of Innovation Exchange Brokerage Model, for Innovation Exchange, funded by OTS, Cabinet Office (2009-10; £25,000).

The social impact measurement agenda is currently changing fast with growing demands from funders and public sector commissioners. The impact of the Middlesex University research in this area is expected to grow considerably with invitations to join new partnerships and policy makers looking to draw on our work in developing more appropriate approaches to identifying and reporting the impact of organisations.

Sources to corroborate the impact

1. Briefing paper on Social Impact and SROI (Arvidson, Lyon, McKay and Moro, 2010) quoted in: Cabinet Office (2011) Growing the Social Investment Market: A vision and strategy https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/61185/404970_S ocialInvestmentMarket_acc.pdf
2. Office for Civil Society, Cabinet Office (factual statement available)
3. Third Sector Magazine (leading weekly paper for practitioners) published article based on Lyon and Arvidson (2011). http://www.thirdsector.co.uk/Management/article/1105066/research-finds- funders-put-pressure-charities-measure-social-impact
4. Arvidson and Lyon (2013) quoted in main cover story of Charity Times http://www.charitytimes.com/ct/April_May_2013_Cover_Story_Measuring_Impact.php
5. Social Impact Team, Big Society Capital.
6. Project Oracle set up by NESTA and GLA to promote Evidence based policy, referenced Lyon and Arvidson (2011) http://www.nesta.org.uk/library/documents/DevProjectOracle_v8.pdf
7. The Guild, (Knowledge Transfer Partnership partner and providers of support for organisations wanting to measure their impact) (factual statement available).
8. Website developed for National Lottery Funded programmes on Local Food to help organisations measure their impact
   http://www.makinglocalfoodwork.co.uk/Exploringyourimpact.cfm
9. Inspiring Impact programme, funded by the Cabinet Office brought together the social impact measurement community in UK and the launch document referenced Lyon and Arvidson (2011) extensively. http://inspiringimpact.files.wordpress.com/2012/06/inspiring_impact.pdf
10. Head of Measurement and Evaluation, New Philanthropy Capital, (Leading body supporting organisations to measure their impact) (factual statement available)