Summary of the impact

This case study demonstrates how extensive University of Manchester (UoM) research over nearly two decades has led to a step change in policy monitoring practices, through the development of innovative indicator methodologies that have strong analytical, learning and spatial emphases. The key impact was the direct translation of a UoM research report into the Office of the Deputy Prime Minister's (ODPM) official spatial planning monitoring guidance, with all 394 English local planning authorities required to comply with evidence-based plan-making. This work continues to shape the policy debates and practices of the UK Coalition Government, with impact extending internationally, influencing policy, debate and practice within: the EC, the UN, the World Bank, Australia and China (Shantou).

Underpinning research

The impact is based on on-going research carried out at UoM since 1995, with the first major publication in 1998. Core researchers (current roles) include: Professor Cecilia Wong (1993-2000, 2006-); Dr Mark Baker (Reader, 1995-97, 2000-); Dr Stephen Hincks (Lecturer, 2007, 2009-); and Richard Kingston (Senior Lecturer, 2003-).

Since the demise of the social indicators movement in the late 1970s, research interest in policy usage and the methodological development of indicators has stagnated. The previous studies were largely concerned with the monitoring of national conditions, which are often different from those that inform spatially-oriented urban and regional policies. UoM research has filled this gap, developing innovative methodologies to improve the technical and statistical measures of indicators (with spatial and dynamic dimensions); integrating a learning and analytical approach of policy monitoring; improving the understanding of policy concepts; linking the analysis to different socio-political contexts; using visual interpretation; and engaging key stakeholders in the process to develop a comprehensive framework.

Wong's ESRC Fellowship (1995-98) at UoM laid down the foundation for this area of research for the next two decades. The findings [B][D][E] that emerged from the Fellowship:

• Showed the importance of user perspectives and contextual interpretation of indicators.
• Demonstrated the value of using principal component analysis to develop composite indices.
• Empirically tested the importance of different factors for local economic development.
• Presented the visualisation of indicator values via mapping analysis.

Early policy impact from the findings included [D] being cited prominently in the 2002 `Towards a National Regional Benchmarking System' report to the Australian Local Government Association (devoting a full section `1.4.4 Cecilia Wong on LED Indicators'). This paper was also cited in `The Changing Dynamics of Urban America' for the US based `CEOs for Cities' (Harvard University, 2004). Wong was appointed as an expert Member of the Office of the Deputy Prime Minister (ODPM) `Planning Research Network' (2003-06) and was also an advisor to the UK Government on matters of data and policy monitoring. Finally, the methods emerging from the ESRC Fellowship were applied in two related Town and City Indicators projects commissioned by the ODPM, disseminated at the Urban Summit (2002) [B], and further developed in the spatial planning field [A][C]. A number of influential reports have been produced by the team (2005-2012):

• ODPM `Local Development Framework Monitoring: A Good Practice Guide' (2005) & `Regional Spatial Strategy and Local Development Framework Core Output Indicators — Update 2/2008' (2008)
• RTPI/DCLG `Outcome Indicators Framework Report' (2007) & `Measuring the Outcomes of Spatial Planning in England' Official Report (2008)
• RTPI — `A Map for England: Spatial Expression of Government Policies and Programmes (and its Map Compendium)' (2012)

ODPM: The ESRC research was further developed and applied in monitoring spatial planning policy. As part of the 2004 Spatial Planning Reform in England, the team developed an evidence- based approach [D] for the ODPM to monitor complex spatial planning policy. Six core design principles were established: (1) the structure-performance model; (2) the objectives-targets- indicators approach; (3) a nested hierarchy of indicators; (4) a framework of indicators; (5) the use of analytical indicator bundles; and (6) analytical principles. The report was directly translated into the ODPM's official guidance that all English local planning authorities had to comply with up until 2011 (Wong & Kidd at the University of Liverpool, and Baker at UoM).

RTPI/DCLG: The ODPM research was followed up an additional research study for the Department for Communities and Local Government (DCLG) and the Royal Town Planning Institute (RTPI). In order to shift the policy monitoring ethos, the research [A] emphasised the need to innovate the monitoring framework by (1) moving from outputs to outcomes; (2) demonstrating the value of bundling indicators to yield meaningful policy intelligence; (3) showing the importance of spatiality; and (4) developing a fully integrated spatial planning outcome framework. The framework also showed the value of (5) adopting a collaborative, reflexive and double loop learning approach — questioning the goal whilst solving the problem - on complex spatial policy monitoring (Wong, Baker, Kingston, Hincks and Rae at UoM; & Watkins and Ferrari at the University of Sheffield).

RTPI: In 2011 the UoM team was commissioned by the RTPI to develop `A Map for England', to inform the debate over the Government's preparation of the `National Planning Policy Framework'. This piece of research focuses on identifying the spatial synergies and conflicts of government policies, programmes and initiatives as well as monitoring whether the existing and planned policies and programmes are in conflict with the core contextual indicators that spatial planning is operating within. The use of map overlays to present such tensions and conflicts is seen as innovative, and achieves the objective of creating dynamic policy intelligence that is open source, and allows individual users to exercise their own reflexive knowledge to interpret the maps in order to inform their own policy thinking (Wong, Baker and Hincks at UoM).

References to the research

(all references available upon request — AUR)

Details of the impact

Pathways: Building on the early impact of Wong's ESRC research, further studies were carried out for government departments — ODPM/DCLG - leading to her appointment to advisory roles of government and agencies and other international collaboration. Wong's reputation as a quantitative planner led to various expert advisory roles: Expert Member of the DCLG's Housing and Planning Panel (2007-10) and Expert Panel Member of the European Commission Directorate General for Regional Policy `Urban Audit Analysis II' (2008-09). More recently, she has been invited to advise the UN-Habitat BLP Partner (Middle East) and to establish a monitoring framework for master planning in Shantou, China. Taken together, UoM research on indicators and policy monitoring methodology has led to a step change in policy monitoring practices in Britain, as well as shaping professional debates, thinking and practices internationally (World Bank, EC, Australia, Africa, UN-Habitat Middle East, and China).

Impact 1: Official planning monitoring practice in England. When preparing their `Annual Monitoring Reports' (AMRs) all local planning authorities in England were required to comply with the official ODPM `LDF Monitoring: A Good Practice Guide' — from publication (2005) until April 2011. The first page of the guidance stated clearly: "This practice guidance is based on work undertaken by Cecilia Wong, Mark Baker and Sue Kidd". This has had a direct and profound impact on shaping policy monitoring practice and ethos in England, constituting an evidence-based approach for spatial planning, and contributing to the wholesale spatial planning reform of the 2004 Spatial Planning Act. The impact of this guidance in shifting the mind-set of planning authorities in terms of how they used indicators and the AMR to inform policy-making was ascertained in a subsequent survey (in the 2007 Outcome Indicators Framework Report). Of the 186 local authorities (out of 394) that responded, 79.3% found the combination of core output and other indicators in the framework useful; 89.6% found the AMR process useful in the overall plan-making process; and 100% agreed that it was important to collect indicators to measure the outcomes of planning policy.

Impact 2: An integrative, collaborative and reflexive framework to measure policy outcomes. The 2008 follow-up research on `Spatial Planning Outcomes Indicators', for the RTPI and the OPDM offered further new thinking towards the development of a `Strategic Planning Outcome' (SPO) monitoring framework. Findings were directly used by the ODPM to update the 2005 guidance (ODPM Update 2/2008). Moreover, the SPO framework has further shifted the mind-set and practice of spatial policy monitoring in new directions by offering an integrative framework that moved away from the traditional linear monitoring ethos, towards collaborative, reflexive and double loop learning — to `question the goal whilst solving the problem' — approach for monitoring complex spatial policies. As highlighted by the then Communities Secretary, Hazel Blears: "..... The Outcome Indicators Project gives councils a new more rounded evaluation tool to consider. It builds on existing monitoring structures that already help councils plan, monitor and assess the way they plan for sustainable communities". The then RTPI Secretary General adds, "... by adopting the measures set out in the Outcome Indicators Project, local authorities and regional planners will be able to gain a much clearer picture of the overall effect their planning policy is having on communities, the economy and the environment" [1].

Impact 3: Continuous influence on policy thinking and debates during the Coalition government. Our research continues to operate at the forefront of influencing the Coalition government's policy thinking. During 2011 and 2012, UoM has been contacted by staff of the Department of Business, Innovation and Skills (BIS) on the monitoring of spatial planning and the development of the `Assisted Area Map' in relation to the use of spatial indicators and policy monitoring. BIS opened a consultation on 31/7/2013, commenting that: `Government has a long- standing working relationship with the Centre for Urban Policy Studies, and within my own policy area, you provided sound advice in relation to developing the forthcoming 2014-2020 Assisted Areas Map' [2]. UoM researchers were also commissioned by the Royal Town Planning Institute (RTPI) to apply spatial analysis to develop `A Map for England' (www.mapforengland.co.uk) to provide the evidence base for the planning of infrastructure and services and enable decision makers at the local level to make more informed policy choices [3]. Following major media reports on BBC Radio 4's `Today' and `You and Yours' programmes, the map findings were presented on the RTPI pilot website, and subsequently drawn upon by politicians to illustrate their points within the `Growth and Infrastructure Public Bill Committee' [4]. The BIS Minister Michael Fallon and Shadow Planning Minister Dr Roberta Blackman-Woods referred to the maps, showing the relationship between superfast broadband coverage and National Park boundaries, and commenting that National Parks have some of the worst access to (and availability of) superfast broadband in England; with National Parks in the North of England particularly poorly served, as the availability of superfast broadband varies between and within National Parks. Blackman-Woods also made the point that there is clearly an issue that needs to be addressed in the East of England which is both a growth area (and an internationally renowned centre for research) and an area that receives a low rating by Ofcom for superfast broadband availability. The RTPI President Colin Haylock noted that: "with a Map for England policy makers could make better judgments about how individual policy proposals interact with and affect development of the country as a whole. Such an initiative would also increase consistency in the appraisal of policy, improve security and resilience, and provide a better understanding of sectoral issues that might complement or conflict with each other" [3].

Impact 4: International arena: key references, advisory roles and professional practices. The World Bank includes Wong's Urban Studies paper [D] as a key reference under its `Local Economic Development (LED): a Primer' [5] which continues to impact on shaping policy and practice across the world. Similarly, [B] is referenced by the LED Network of Africa (LEDNA) [6]. More specifically, Wong's 2006 Book [B] has an impact on shaping the policy discourse in Australia over the development of sustainability indicators. The `Sustainability Indicators' report, prepared by the Cairns and Far North Environment Centre for its community forum in 2008 [7], made key recommendations to State and Local Government about the role and context of sustainability indicators, by extensively citing direct quotations from [B] nine times, and using the five broad principles established in the book as a benchmark for their activities. In the EU context, principal component analysis applied in [D] was also used in `Urban Audit Analysis II' (2010) and Wong was the Expert Panel Member to oversee the preparation of the report and made comments over its methodology in 2008/09 [8]. More recently, Wong was invited by the UN-Habitat BLP Partner at the Middle East to join an Expert Group Meeting (10-11<sup>th</sup> May 2013) to evaluate and revise the current `City Prosperity Index' (CPI). In the invitation letter, the Director of MERC-BLP specifically mentions that: "We are aware that you have conducted extensive studies and research in the field of Urban Prosperity, and that you have prepared and published a research paper under the title of `The Relationship Between Quality of Life and Local Economic Development: An Empirical Study of Local Authority Areas in England' [E]" [9].

Wong and Baker are currently involved in an international consultancy team led by Nanjing University (China) to develop an innovative indicator framework to monitor the progress of the new `Shantou Master Plan'. Learning from the experience of developing the SPO in England, the broad principles of developing `an integrative, collaborative, reflexive and flexible' indicators framework with a map-based visual interface were presented, in April 2013, to the international expert panel and senior planners and politicians of the Shantou Government. The project's director commented that "adapting certain key principles you developed for monitoring spatial planning policies and outcomes in England to apply to the complex administrative and governing contexts of Shantou are very helpful. In particular, your ideas of adopting a gradual, incremental approach to policy monitoring, with a more focused and flexible spatial framework for indicators analysis that emphasises on the communicative aspects of policy learning are seen as an innovative practice in China and make an important contribution to our on-going work on the master plan" [10]. This on- going engagement further demonstrates the continuing impact of UoM research in shaping international practices of policy monitoring.

Sources to corroborate the impact

(all claims referenced in the text)

[1] (2008) RTPI `Press Release: Minister welcomes new way for councils to evaluate planning policies' (10^th July)

[2] Testimonial from Assistant Director, Economic Geography, Department of Business, Innovation and Skills (12^th April 2013)

[3] (2012) RTPI `Press Release: RTPI launches Map for England debate' (22^nd March)

[4] (2012) RTPI `News: MPs draw on Map for England data in debate' (5^th December)

[5] (2006) Swinburn, G. et al (eds.) Local Economic Development: A Primer-Developing and Implementing Local Economic Strategies and Action Plans, The World Bank (p.63)

[6] LEDNA (Local Economic Development Network of Africa) Web Link

[7] (2008) Cairns and Far North Environment Centre `Sustainability Indicators: From Theory to Practice', report prepared by the CFNEC

[8] (2010) Urban Audit II Project: RWI `Second State of European Cities Report, Research Project for the European Commission', DG Regional Policy (pp. 9-10, 22)

[9] Email Invitation from Director, MERC-BLP (The Middle East Regional Center of the Best Practices and Local Leadership Programme), (25^th Feb 2013)

[10] Testimonial from Director, Shantou Master Plan Project (23^rd April 2013)