Summary of the impact

Southampton statisticians have made a valuable contribution to government policy formulation across the UK and further afield to areas of North America and Europe. Novel methods for delivering more accurate estimates of socio-economic indicators at neighbourhood level have given local authorities, national government agencies and MPs the tools to implement more effective policies designed to assist the poorest communities and strengthen community cohesion. The UK's Office for National Statistics (ONS) has described Southampton's contribution as `a breakthrough', while the Mexican government agency, CONEVAL, regards this work as `the most prestigious' of its kind.

Underpinning research

Surveys are often designed to provide estimates of social and economic indicators at a population level. For example the Labour Force Survey, conducted by ONS, is a large household survey which provides the official measures of employment and unemployment in the UK. Policymakers at a local level, for example local authorities or MPs serving their constituents, require a much more detailed picture than is provided by national estimates. However, obtaining accurate values for small geographical areas — small area estimation — is problematic as the survey sample size is usually too small to yield reliable direct estimates. Statistics are instead based on `indirect' estimates that use information from other areas with similar characteristics to the region under the microscope.

Statisticians at the University of Southampton have been developing novel statistical methodologies of small area estimation since 2001 to produce more accurate estimates of key social and economic indicators for more effective local policymaking. Indicators that have been a particular focus of the research are: (a) average household income and average business turnover; (b) complex poverty indicators (head count ratio, poverty gap and poverty severity); (c) proportions of unemployed, employed and inactive individuals and corresponding measures of precision quantified by mean squared error (MSE) estimators.

The first phase (2001-2004) of this research was carried out as part of the EU's EURAREA (Enhancing Small Area Estimation Techniques to meet European Needs) project. The aim of the project was to provide National Statistical Institutes across Europe with a basis for deciding whether, and how, to apply small area estimation techniques in the production of official statistics. Professor Ray Chambers (1995-2006) and Dr Ayoub Saei (Research Fellow, 2002-2006), led the development of techniques for deriving point and MSE estimates of labour force activity at small area level [3.1]. Local authorities in the UK had previously found accurate unemployment information almost impossible to obtain; the unreliability of estimates resulted in only about a quarter of the annual estimates of unemployment in 1999/2000 qualifying for publication.

From 2003 to 2006, funded by an ESRC grant, Chambers and Dr Nikos Tzavidis (Research Fellow, 2003-2005; Senior Lecturer, 2010-present) developed multi-quantile (M-quantile) models for small area estimation of averages and totals [3.2, 3.3]. Tzavidis was subsequently PI on a project that extended this work to methodologies for estimating poverty indicators and distribution functions at small area level. This latter strand of research, from 2008 to 2011, was part of the EU's SAMPLE (Small Area Methodologies for Poverty and Living Condition Estimates) project, which identified new indicators and models for inequality and poverty [3.4-3.6].

The research carried out in the course of the ESRC-funded work and the SAMPLE project resulted in the development of innovative statistical methodologies in small area estimation. Chambers and Tzavidis' 2006 paper [3.2], published in Biometrika, was key to the development of this field and is still regularly cited (117 citations to 31/10/13 — Google Scholar). The Office for National Statistics (ONS) was able to build on the methodology to publish improved unemployment figures at local authority level, first as experimental statistics and then as official national statistics.

All new methodologies were developed with users' needs in mind and supported by easy-to-use software. This work led to significant international interest in Europe (e.g. Eurostat EURAREA Project, Portugal, the Ukraine) and outside (e.g. Australian Bureau of Statistics, Statistics New Zealand, Korea). In particular, this approach, coupled with contracts with key non-academic organisations such as the ONS, Mexico's National Council for the Evaluation of Social Development Policy (CONEVAL) and the Netherlands' Centraal Bureau voor de Statistiek, has widened the influence of Southampton's pioneering techniques.

References to the research

Publications:

(*) These references best indicate the quality of the underpinning research.

Grants:

3.G1 EURAREA (Enhancing Small Area Estimation Techniques to meet European Needs), Framework Programme 5, 2001-2004, Professor Ray Chambers [€1,833,781]

3.G2 Multi-Quantile Models for Small Area Estimation, ESRC, 2003-2006, Professor Ray Chambers [£164,465.93]

3.G3 SAMPLE (Small Area Methodologies for Poverty and Living Condition Estimates), Framework Programme 7, 2008-2011, Dr Nikos Tzavidis (PI for University of Southampton) [€778,000]

3.G4 Office for National Statistics, Methodology Contract (contract number PU-10/0141), 2010-2015, University of Southampton [minimum amount: £135,000 x 5 years]

3.G5 Netherlands Central Bureau of Statistics, Methodology Contract, 2010-2011 [€99,900]

Details of the impact

Decision makers tasked with devising and implementing effective and inclusive socio-economic policies need as much information as possible. Novel methodologies of small area estimation developed by researchers at Southampton have provided policymakers, both in the UK and internationally, with reliable data ranging from average household income to unemployment figures right down to neighbourhood level. In particular, the Southampton research has allowed small-area estimates of employment indicators and income indicators to be derived from the Labour Force Survey, the largest household survey in the UK, and the Family Resources Survey, respectively.

Southampton's long-time collaboration with the ONS has proved essential in providing this range of disaggregated data to the Department for Work and Pensions (DWP) and local authorities around the UK. Dr Alan Taylor, Head of the ONS' Small Area Team, has described Southampton's work in this field as `a recognised breakthrough' [5.1]. He said: `In my experience great research can often lead to academic papers but not get translated into outputs. Our collaboration [with Southampton] has led to significant improvements in ONS small area estimates of income and unemployment and [has] led the world in model-based estimation.'

The ONS' own National Accounts Team and Labour Market Division have relied upon the estimates to answer enquiries from local authorities, policy advisers, government departments and academics. The ONS has reported that demand for small area estimates has been `strong' [5.1] and that users have drawn on them for a variety of socio-economic purposes. An analysis conducted by the ONS in 2012 concluded: `Without these estimates valuable insights into the differences between small area geographies would be lost... Users would need to look around for an alternative and less suitable source. In particular, a number of parliamentary questions would have to be answered at higher geographies.' [5.1]

Southampton's success in producing sets of local authority-level estimates of labour force activity has proved invaluable for MPs in serving the needs of their constituents. A letter [5.2] written to the ONS in 2005 by the Chief Librarian for the House of Commons, John Pullinger, reflects the significance of these data, obtained using Southampton-designed methodologies. Pullinger stressed the importance of making a common set of key labour force indicators — for example Jobseekers Allowance claimants as a proportion of the constituency's working age population — available to serving MPs, something which is only possible as a result of this research. The model-based estimates of mean income developed by Southampton researchers helped the ONS plug a gap in its national statistics provision enabling the ONS to supply the DWP with small area income estimates.

Local authorities are the main users of these estimates. The Greater London Authority used small area estimates in its 2010 Focus on London report, Income and Spending at Home [5.3], which compared income at both individual and household level across the capital. This was a key input in the Mayor of London's Outer London Commission Report [5.4], which analysed the challenges and opportunities facing the outer London economy. This in turn was an important source of information for developing new policies in housing and transport proposed in the new London plan (2011), the strategic development plan [5.5] for the capital up to 2031.

Another local authority, the London Borough of Newham, used the estimates in their Information Management System allowing members of the public to summarise and map data via data interrogation tools on the borough's website. Newham further used the data to answer queries from members of the public and from other local authority officers. Users, including the DWP, stated that without these estimates valuable insights into the differences between small area geographies would be lost [5.1].

The impact of Southampton's research has extended overseas. In 2009, the Mexican government agency, CONEVAL, sought the University's help in producing estimates of income and social rights deprivation in Mexican municipalities. Methodologies developed by the Southampton team since 2001 have allowed CONEVAL to measure access to health, education and food at neighbourhood level as part of its brief to define, identify and measure poverty across the country. Dr Aparicio Jimenez, CONEVAL's Poverty Analysis Director, said: `With this research, the poorest municipalities in Mexico can be identified. With this valuable information, the federal government, the National Congress, federal states and municipalities will be able to design more effective social programmes.' CONEVAL described Southampton's research as `the most prestigious' of its kind [5.6].

A year later Southampton's modelling methods enabled the Netherlands' Centraal Bureau voor de Statistiek (CBS) to deliver more accurate estimates of business turnover, which have guided policy changes at a national level [5.7]. CBS reported that Southampton's methodologies proved superior to other small area estimation approaches `in several situations' [5.8].

Sources to corroborate the impact

5.1 Head of Small Area Team, Office for National Statistics.

5.2 Letter by Librarian of House of Commons to Director, Labour Market Division, Office for National Statistics).

5.3 http://data.london.gov.uk/documents/FocusOnLondon2010-income-and-spending.pdf
(The Head of the Small Area Team at ONS can confirm the use of the Southampton research in producing this data)

5.4 http://static.london.gov.uk/olc/docs/final-report.pdf
(This document makes substantial use of the data in 5.3)

5.5 http://www.london.gov.uk/priorities/planning/london-plan
(This document builds on 5.4 and also uses the data in 5.3)

5.6 Letter/Email about the value of Southampton/CONEVAL collaboration (2012) by the Director of Poverty Analysis, CONEVAL, Mexico.

5.7 CBS discussion paper: Small Area Estimation of Turnover of the Structural Business Survey (2012): http://www.cbs.nl/NR/rdonlyres/2FC2BE74-1AF0-462A-9C76-9C836EA07655/0/201203x10pub.pdf

5.8 The Researcher of Netherland Centraal Bureau voor de Statistiek (CBS), Division of Methodology and Quality can corroborate the use of the code developed by Southampton for implementing the methods.