Summary of the impact

Research conducted by Members of the Centre for Local and Regional Government Research (CLRGR) in Cardiff Business School (CBS) concerning the use of targets, performance indicators and external inspection to assess council performance and drive improvement in local services, has had a direct, significant and on-going impact on government policies in England and Scotland. In England, the research informed the Labour Government's decision to reduce the number of national performance indicators. It was also used by the Conservative Party in developing its 2010 manifesto commitment to reduce local government inspections and informed the local government policies implemented by the Coalition Government. In Scotland, the research directly influenced the design of a new inspection methodology introduced by Audit Scotland in 2009.

Underpinning research

Researchers from the CLRGR undertook five inter-related studies of the use of targets, performance indicators, external inspection and peer review to assess council performance and drive improvements in local services.

In 2003, the Office of the Deputy Prime Minister¹ (ODPM) commissioned CLRGR to conduct a meta-evaluation of the Local Government Modernisation Agenda (£1.85m)^3.1. Researchers gathered empirical evidence through surveys of senior local authority managers and in-depth, longitudinal analyses of twelve case study councils. They also analysed data from a wide range of secondary sources including inspection reports, statutory performance indicators and national surveys of public satisfaction with and confidence in local government services^3.2. They concluded that national targets and external inspections imposed by central government departments had encouraged improvements in local services, but were costly and producing diminishing marginal returns. The government's reforms had improved service quality and user satisfaction, but had damaged staff morale and failed to improve productivity. The researchers recommended a reduction in the intensity of top down inspection and monitoring and argued for greater investment in self-assessment from within the local government sector.

In 2007, the CLRGR was awarded £45,000 by the Local Government Improvement and Development Agency (IDeA) to undertake a follow up study to develop more detailed proposals for a new approach to performance improvement. Their report analysed the rate of improvement in services between 2000 and 2006. It concluded that there was a continuing role for external inspectors in safeguarding the quality of services for vulnerable groups, but recommended that in future there should be greater use of self-assessment and peer review to secure improvements.

In a concurrent study, funded by the Economic and Social Research Council (ESRC) between 2006 and 2009, the researchers compared approaches to assessing local government performance in England, Scotland and Wales^{3.3 & 3.4}. They gathered evidence from an analysis of performance indicators and elite interviews with senior policy makers, inspectors and senior local government officers. They found very significant differences between the approaches adopted in the three countries, leading to different impacts on local government performance.

In 2008, the CLRGR was commissioned by Audit Scotland to undertake a detailed evaluation of the way in which it assessed the performance of Scottish authorities^3.5. They conducted a comprehensive survey of senior managers in all Scottish authorities and detailed case studies of the inspection process in six councils, plus interviews with senior auditors and civil servants. The researchers showed that the assessment methodology (known as `Best Value Audit') had facilitated improvements in councils' performance but needed to be more flexible and place greater emphasis on assessing service outcomes and identifying good practice.

In 2012-13, CLRGR researchers were commissioned to evaluate peer challenge, a key component of the Coalition Government's policies for local government and fire services. They concluded this approach has been broadly successful in promoting improvement.

The five studies were led by Steve Martin (Prof 00-present) and involved seven other members of the CLRGR: Rachel Ashworth (Lecturer 01-04, Senior Lecturer 04-08, Reader 08-present), Alex Chen (Research Fellow 03-07), James Downe (Senior Research Associate 03-13, Reader 13-present), Tom Entwistle (Lecturer 05-08, Senior Lecturer 08-12, Reader 12-present), Clive Grace (Honorary Research Fellow 05-present), all faculty at CBS, and Richard Cowell (Lecturer 01-06, Senior Lecturer 06-11, Reader 11-present) and Gareth Enticott (Research Fellow 03-12, Senior Lecturer 12-present), both from the Cardiff School of Planning and Geography. The study of the Local Government Modernisation Agenda was led by Martin and undertaken by colleagues from the CLRGR and sub-contractors at Birmingham and DeMontfort Universities and Ipsos MORI. The studies of comparative approaches to performance assessment and of the Best Value Audit process in Scotland were undertaken in collaboration with the University of Edinburgh, but again these were initiated and led by Martin and his colleagues at the CLRGR. In all cases the research contracts were awarded to Cardiff which acted as the lead institution, undertook the bulk of the research and retained the majority of the research funding.

References to the research

Grant: S.J. Martin, J.D. Downe, C.L. Grace with S.M. Nutley (University of Edinburgh) (10/06 to 01/09) Comparing for Improvement: the development and impact of public services audit and inspection, Economic and Social Research Council, Ref. RES-166-25-0034, £97,000.

This programme of research has resulted in ten papers in peer reviewed journals and more than 20 commissioned reports written for government departments and agencies. A paper on the Best Value Audit process^3.5, published in Public Money and Management, was shortlisted for the 2009 UK Public Administration Consortium prize.

Details of the impact

Governments around the world are striving to find ways of driving up the quality and efficiency of public services. The local government reforms pursued by the Labour Government between 1997 and 2010 are widely seen as one of the most ambitious reform programmes to date and have attracted considerable attention in the UK and internationally. By evaluating their impact and similar initiatives by the Scottish Government, the studies conducted by CLRGR researchers have provided an evidence-base for future policy making and had a direct and significant impact on the way in which governments in England and Scotland assess the performance of local authorities.

In the first interim report from the meta-evaluation, published in 2005 (`Meta-evaluation of the Local Government Modernisation Agenda: Progress report on service improvement in local government'), CLRGR researchers concluded that the assessment process in England needed to be more targeted and place less of a burden on local authorities. This finding was reiterated in the final report, published in 2009 (`The state of local services: performance improvement in local government'). [text removed for publication]. The then Labour Government announced two major policy changes in the 2008 white paper `Communities in Control: Real People, Real Power''. Firstly, the number of "centrally-imposed performance indicators" was reduced "from 1200 to 198" and secondly, "the Best Value Performance Indicators and Performance Assessment Framework indicators" in England were replaced by a new approach, the Comprehensive Area Assessment<sup>5.2</sup>. The white paper was presented to parliament in July 2008 and the new policy took effect from 2009 [text removed for publication]. In the same year, the Conservative Party's Policy Green Paper, `Control Shift: returning power to local communities', took a direct quotation from the 2005 report to evidence low satisfaction rates among the public with regard to local authority performance, using this to support its manifesto pledge to reduce the level of top down inspection of local authorities^5.3. [text removed for publication].

CLRGR's 2008 report for IDeA came at a time when the Local Government Association (LGA) favoured a strong self-improvement approach to assessment. The report (`Getting better all the time?') thus "both enhanced what was already in train and provided some good data and analysis for driving the arguments further"<sup>5.4</sup>. The IDeA particularly valued CLRGR's experience "across the UK jurisdictions"<sup>5.4</sup>. By providing "a strong analytic framework", the research "became the under pinning for the next stage of work and policies that were already in train across local government"<sup>5.4</sup>. One example of this was the launch of the LGA's sector-led approach to improvement; in this case, the CLRGR's report for IDeA was used to provide background for the consultation on this change (`Setting the Pace: Developing a Framework for Sector Led Help', 2009)^5.5. These reforms have now been implemented throughout English local government.

In Scotland, the recommendations of the CLRGR's evaluation of the Best Value Audit process (presented in a report entitled `Decisive Moment: The independent review of the Best Value Audit Process') were accepted in full by the Accounts Commission shortly after publication in 2007, and by 2010 were being used to "underpin the Best Value improvement process and the design of the second round of best value audits (known as BV2)"^5.6. The introduction of BV2 has helped Audit Scotland "to deliver a more proportionate and better targeted best value audit than was the case in the first cycle of audits" with the report being "an important factor in driving a more integrated approach to audit and inspection ... drawing on effective self-evaluation within public bodies"^5.7. A particularly important development, identified by Audit Scotland as something which would not otherwise have been done, was the introduction of peer involvement. This element of the process was subsequently introduced to the Fire and Rescue Best Value audit work and was something that Audit Scotland feels added to the "credibility" of the audit teams and has contributed to the "increased transparency" of how audit judgements are reached^5.7. Reflecting on the long-term implications of those changes, the research findings "helped to create momentum around improved joint working with [their] scrutiny partners, with an outcome-based approach to audit and inspection which has continued to this day"^5.7. A study published in 2008 and commissioned by Scottish Ministers also made use of the 2007 report in an assessment of the work of the 32 local authorities in Scotland to promote equality between disabled and non-disabled people^5.8.

The 2012 evaluation of peer challenge was cited by the Government in its written response to the pre-legislative scrutiny report by the Draft Local Audit Bill Committee: "Cardiff Business School has completed an interim evaluation of the Local Government Association's peer challenges. It found that the peer challenge teams had the right mix of skills and experience and councils welcomed a challenge that was tailored to their needs. The Government therefore does not see the need for a further review of sector-led improvement" (2013: 17).

The research described has achieved considerable reach because of the way in which it was developed, with findings being debated with senior civil servants, auditors and local government officers throughout. Activities included policy briefings with civil servants, presentations to: the Society of Local Authority Chief Executives Annual Conference (2008); civil servants attending the DCLG's Analysis Week (2009); the ESRC Public Services Programme Policy Maker's Briefing (2009); the Audit Commission's Annual Research Conference (2009); the Audit Commission's Board (2010); the New Local Government Network Seminar on Future of Performance Assessment (2010); and chairing the DCLG workshop on `Localism, Value-for-Money and the New Policy Agenda' (2010). The research featured in the lead article in an edition of SOLACE Foundation Imprint (2008) (`Getting so much better all the time'), which was distributed to all senior local authority managers, and in articles in the Local Government Chronicle. The research was highlighted in an article which appeared in the Guardian newspaper in February 2008 (`A Marriage of Convenience'), demonstrating recognition by the media of its relevance to the general public.

The reach of the research was further extended both indirectly, through the researchers' involvement with think tanks, and directly, through advisory roles which brought them into regular contact with policy makers; these provided an important means of feeding the recommendations into the policy making process. Martin was a Director of the IDeA (2003-2010) and a member of the steering committee of the Society of Local Authority Chief Executive's Foundation (2008-2010). He was also a member of the board of the New Local Government Network (2007-present), the leading think tank on local government policy and acted as an advisor to a report it published which used CLRGR's work (`Through the Looking Glass: Putting Citizens at the Heart of Assessment' July 2010)^5.9. Downe and Martin were appointed to the UK Government's Expert Panel on Local Governance in 2009, and Martin gave evidence to the Communities and Local Government Select Committee enquiry, Audit and Inspection of Local Government (2011)^5.10.

The UK is seen as being at the forefront of approaches to assessing the performance of local government and the research has therefore also attracted a high level of international interest. Senior government officials and local government managers from across the world, regularly visit CBS to learn first-hand about the research. Martin was invited to speak about the research at the Swedish Local Authorities Research Network Annual Conference at Lund University in 2008. This international perspective has been developed through a follow-on project funded by an ESRC Knowledge Exchange Opportunities grant and aimed at the cross-fertilisation of ideas between countries (2012-13). A programme of events engaged very senior policy makers from across the UK, other European countries, North America, Asia and Australia, with three major international conferences in London, Canada and Australia.

Sources to corroborate the impact

1. [text removed for publication]
2. DCLG (2008) Communities in Control: Real People, Real Power, White Paper. Available at: http://www.official-documents.gov.uk/document/cm74/7427/7427.pdf (see 1.41 & 3.27 for policy changes which reflect CLRGR's recommendations). Also see Evidence Annex for citations `Bovaird & Downe forthcoming' on p.39, 57 and 69.
3. Conservative Party (2009) Control Shift: returning power to local communities. See citation 62 on p.18, a direct quotation from the 2005 report for the ODPM.
4. Statement: Former Chief Executive, IDeA. Now Chief Executive, Community Service Volunteers. Corroborating the impact of the IDeA commissioned study on policy.
5. The IDeA (2009) Setting the Pace: Developing a Framework for Sector Led Help. Available at: http://webarchive.nationalarchives.gov.uk/20090608235722/http:/www.idea.gov.uk/idk/aio/9711172. See p.8, para.7 — reference to CLRGR report `Getting better all the time?'.
6. Accounts Commission (2010) Accounts Commission response to the evaluation of the BV2 Pathfinder Process. Available at: http://www.audit-scotland.gov.uk/docs/best_value/2010/bv2_pathfinder_evaluation_response.pdf. Sets out the changes made in response to recommendations. See para. 2&3 on use of CLRGR report.
7. Statement: Auditor General and Accountable Officer, Audit Scotland. Corroborating the impact of the study on Audit Scotland's approach to assessment.
8. Scottish Government (2008) Reporting on Progress towards Equality of Opportunity between Disabled Persons and Other Persons made by Public Authorities in Scotland: The Scottish Ministers' Duties: Finance and Sustainable Growth. Available at: http://www.scotland.gov.uk/Resource/Doc/251630/0073696.pdf. See para.7.84 — reference to the 2005 report for the ODPM.
9. New Local Government Network (2010) Through the Looking Glass: Putting Citizens at the Heart of Assessment. Corroborating use of research by NLGN think tank. Available in part at: http://www.nlgn.org.uk/public/2010/through-the-looking-glass-putting-citizens-at-the-heart-of-the-assessment-process/, or in full on request.
10. House of Commons Communities and Local Government Select Committee (2011) Audit and inspection of local authorities, Fourth Report of Session 2010-12, HC 763. Record of Martin's oral evidence available at:
    http://www.publications.parliament.uk/pa/cm201012/cmselect/cmcomloc/763/11020702.htm

All documents and web pages were saved as pdf on or before 08.08.13 and are available on request from the HEI.

---

¹ In 2006, the ODPM became the Department for Communities and Local Government (DCLG).