Summary of the impact

The European Social Survey (ESS), established in 2001 and located at City University London, is a major multinational social survey involving the participation of 36 countries. It is designed to monitor change and explain the interaction between Europe's changing institutions and the attitudes, beliefs and behaviour patterns of its diverse populations. The impact of the ESS is extensive nationally and internationally particularly through improving survey methodology and through the subsequent adoption of these standards and practices by other national and international survey programmes. This has led to higher standards of measurement in policy-oriented surveys and commercial survey practice. The ESS was awarded the Descartes Prize for `Excellence in scientific collaborative research' in 2005 in recognition of its world-leading quality. In 2014 the ESS will become the first UK-hosted European Research Infrastructure Consortium (ESS ERIC) when it is launched on 31^st January at the Royal Society in London by Commissioner Geoghegan-Quinn of the European Commission Directorate-General (Research and Innovation) and the UK Minister for Universities and Science, the Rt. Hon. David Willetts MP.

Research undertaken at City University London in collaboration with the University of Sussex led to a new methodological approach being developed through interaction with survey researchers within the framework of the European Social Survey (ESS), supported by European funding. The new methodological tool enhances understanding and analysis of media claims by governments and policy-makers. The work undertaken has addressed the need for interpretation of attitude questions across different countries and variations in responses to cross-national studies.

The impact was achieved through 1) the design of new methodologies for the coding of media claims by governments and policy-makers; 2) providing contextual data on media events; 3) the provision of guidance manuals and training across eight European countries for coding and archiving data for media analysis; and 4) the development of a media events framework for the analysis and understanding by policy-makers of `normal' and `exceptional' findings of public opinion surveys.

Underpinning research

The project directed by Statham (Professor at the University of Sussex) and Tumber (at City University London since 1988, now Professor) was undertaken as part of the overall European Social Survey (ESS), a multinational survey on the long-term changes in attitudes, beliefs and behaviour patterns across Europe. The research was aimed at understanding the ways in which media claims on news are interpreted and it classified and developed new methodological tools for the cross-national study of news.

Initially, Statham and Tumber outlined an overall description of the ESS providing a rationale for event data collection. This was necessary to examine the impact of specific events on survey responses nationally. The analysis revealed the lack of a methodical and systematic basis for the retrieval of information. This led to the development of a systematic media-based approach and a new methodology. The research included pilot studies, coordination from several European countries, workshops and conferences. These activities, directed by Statham and Tumber, built on the previous three rounds of the ESS. The media landscapes of five European countries were studied to cover the different cultural and national dynamics along the North/South and East/West axes of Europe; and the range of old and new democracies. Statham and Tumber's research benefited from the cooperation of eight different countries for the data collection phase between 2010 and 2011: Spain, Portugal, the UK, the Netherlands, Lithuania, Poland, Slovakia and Greece.

An extensive literature review was conducted between April and December 2006 to map European national newspapers and existing methods for comparative news content analysis. Limitations and existing variations in the conceptualisation and characterisation of media claims were analysed. Simultaneously, a pilot study was conducted (between 10^th September 2006 and 6^th January 2007) to test a coding scheme for media events. The UK, Germany, Poland, Spain and Ireland were included in this study. Successive versions of research papers were presented by the research team in February and June 2007 addressing ways to develop new methodological tools to analyse news content in cross-national studies.

Background, guidelines, coding scheme and a codebook based on the work of Statham and Tumber are now available online. The new scheme for coding and analysing media claims is being applied in the 6^th and 7^th rounds of the ESS.

References to the research

Statham P., & Tumber H. (2007). News Events Analysis: Measuring the Political Climate. Paper presented at ESS National Coordinators meeting, 20^th February 2007, Mannheim, Germany.

This presented outputs of the pilot study and feedback on the national variations in the interpretation of media events in social surveys. References to this paper appear on the Eurpol website, www.eurpolcom.eu/research_projects_ess.cfm where outputs by Statham and Tumber are described as providing a `firm long-term foundation' for the ESSi project through `an extensive review of existing state of art methods, approaches and literature on news content and event analysis'.

Reference 1 is in a peer-reviewed journal. The book by Professor Sir Roger Jowell (founder of the ESS) et al is published by Sage, which is recognised academically as a high quality publisher of journals and books. The citations to `News Events Analysis: ...' indicate that this is an important point of reference.

Details of the impact

The beneficiaries of the European Social Survey Infrastructure (ESSi project, 2006 to 2011) are located internationally and especially in the European Community (EC). An undirected form of measuring this impact is provided by the ESS Data Archive team which produces data user statistics. In September 2012 there were 50,255 registered users from 226 countries (ESS User Survey, 2012). The total number of users (non-registered and registered) and those who access the reports and online materials is even higher. 90% of them are based in Europe and 6 of the countries directly involved in the ESSi exercises are in the top 10 countries in terms of user numbers. According to the ESS User Survey (2012), students are the main users and beneficiaries of the ESSi new methodology and the provision of data and methodological guidelines are most likely used as learning materials in universities. However, 11% of all users are outside the academic sector in NGOs, governments, private enterprises or other types of activities (ESS User Survey, 2012). As a result, the research has significant reach, providing material to support work on media and public opinion surveys to a wide range of non-academic users. According to the ESS website, the guidelines for coding were downloaded 882 times and the training page 805 times by 21^st June 2012. Following the launch of the new ESS website on 10^th September 2013 there had been 371 visits as at 26^th November 2013.

The research contributed directly to the development of new methods for cross-national studies and improved the ESS coding of news. The new way of understanding media claims allows a categorisation of content and provides a score that creates a more powerful tool for measuring the strength and direction of news (ESS, 2012, `Rationales for Changes and Improvements'). This approach also allows national centres to estimate more accurately the time and resources required for data coding. These improvements benefit a variety of users of survey data, including government bodies and social research consultancies working on media studies and especially those analysing cross-national public opinion surveys. The guidelines also provide national agencies working on ESS with an international framework for collecting, archiving and analysing media claims. The minimum number of sources to be used and the time period for the data collection are also specified in the new methodological framework (Harrison et al. 2012, p3).

The European Commission, in its European Communities publication series, has publicly acknowledged City University London's role in the ESSi 'improving social measurement in Europe' and building up 'a network of specialists to refine methods of event-reporting across nations as a necessary backdrop to measuring attitude change among people in different EU countries' (European Commission, 2009). City's research identified existing problems in the data collection among different countries and created a framework for coding media claims that has already been applied and will be used again in the next round of the ESS (ESS, Round 7 Specification for ESS ERIC Member and Observer countries, 2013, p27).

To improve the effectiveness of the ways in which the ESS measured the influence of media events on social opinions and attitudes, a descriptive tool was devised to evaluate the potential impact of news on the public. The process engaged directly with non-academic beneficiaries through several consultations and workshops with representatives of the different countries. The biennial European Survey Research Association (ESRA) conference in Prague in 2007 and in Warsaw in 2009 enabled discussions of ideas with other experts on the ESS. Similarly, the symposium organised by Statham and Tumber at City University London (31^st March 2009) constituted a valuable opportunity for external experts on methodology to give additional advice, comments and pointers on the project. The 2011 Lausanne Conference, in which our team presented a paper, included presentations by leading non-academic international researchers including governmental organisations (Office for National Statistics, Australian Bureau of Statistics, Singapore's Ministry of Social and Family Development, Colombia's Institute for Education Evaluation) and social research institutes and consultancies (Natcen (Britain's leading independent social research institute), Ipsos, TNS, Gallup).

A database identifying national differences in the `political climate' is also being produced as part of the research process that started in 2006 and it was applied across different rounds of the ESS. The software format of the dataset allows consultants, policy-makers and other non-academic researchers to access this baseline for research and facilitates the coordination of cross-national and wider research programmes.

The improved methodological tools are being used by ESS country coordinators in rounds 6 and 7.

Sources to corroborate the impact

A description of the ESSi project goals can be found online on www.eurpolcom.eu/research_projects_ess.cfm. The events organised by City University London in the context of this project are in part described at: www.eurpolcom.eu/events_activities.cfm.

Sessions on event reporting have been organised at the NA4 work programme symposium and at three biennial international conferences of the European Survey Research Association. The team's participation is stated online in the ESRA 2007 Prague conference programme: www.europeansurveyresearch.org/sites/default/files/files/esra-sessions%5B1%5D.pdf; the 2009 Warsaw conference:www.europeansurveyresearch.org/sites/default/files/program.pdf; and the 2011 Lausanne conference: www.europeansurveyresearch.org/conferences/lausanne-2011.

Details of the programme are also available on www.europeansurveyresearch.org/sites/default/files/ESRA_Conference_2011_Programme_Book_1.pdf.

Evidence of City University London's key role in the development of news analysis for the ESS can be found in the 5th ESSi National Coordinators programme, available at:
www.europeansocialsurvey.org/docs/about/ESS5_end_of_grant_report.pdf. Dr Eric Harrison of City University London presented the advances in Mannheim, Germany in February 2009.

Official recognition of the importance of media claims analysis for the ESS and the future development of contextual and event analysis can be found on the ESS website: http://ess.nsd.uib.no/ess/contextual/ and the EC's website: www.ec.europa.eu/research/infrastructures/pdf/essi.pdf. The new framework and coding for media claims are being applied in the new rounds of the ESS and the importance of reviewing the survey methodology has been indicated (ESS Round 7 Specification for ESS ERIC Member and Observer countries, 2013, p27).

The codebook and the guidelines are publicly available on the ESS website: www.europeansocialsurvey.org/methodology/sqpcoding.html.

Fernee, H, Stoop, I and Harrison, E (2012) `Coding media claims in the European Social Survey, Round 6. What happened when the data were collected?' Available online: www.europeansocialsurvey.org/docs/round6/methods/ESS6_media_claims_guidelines.pdf.

Harrison & Stoop (Netherlands Institute of Social Research) presented the paper `When is an event an event? The contribution of media reporting to the survey context' at the July 2011 Lausanne European Survey Research Association conference. The paper analyses the impact of media claims on the public opinion:
www.europeansurveyresearch.org/sites/default/files/ESRA_Conference_2011_Programme_Book_1.pdf.

ESS Specification for Participating Countries Rounds 6-7: www.europeansocialsurvey.org.

Rt Hon. David Willetts, MP: Letter to ESS ERIC Steering Committee Members (15^th June 2011)