Summary of the impact

The impact of this research has been achieved through developing evidence-based recommendations for personal advisers in Jobcentre Plus — the UK's one-stop service for administering state benefits and helping claimants into work. By opening the `black box' of adviser- claimant interviews for the first time, the study produced the following key impacts:

1. Policymakers in the Department for Work and Pensions (DWP) and on the Social Security Advisory Committee (SSAC) gained an evidence-based understanding of a key area over which they have policy control;
2. Consequently, DWP policymakers and Jobcentre Plus managers made policy changes with respect to adviser-claimant interviews;
3. Through these policy changes and our training workshops, recommendations from our study have helped improve the service delivered by advisers to benefits claimants.

Beneficiaries were those claiming state benefits, Jobcentre Plus advisers and managers, and DWP and other Government policymakers.

Underpinning research

This research was conducted at the University of York from April 2007 to September 2009, and was commissioned by the UK Government's Department for Work and Pensions (DWP). DWP funding (£366,015) supported a research team at York that combined twin sets of expertise:

• Professor Paul Drew and Dr Merran Toerien (then Professor and RCUK Research Fellow, respectively, in Sociology) brought methodological expertise in conversation analysis (CA);
• Professor Roy Sainsbury and Ms Annie Irvine (Research Director and Research Fellow, respectively, in the Social Policy Research Unit) brought substantive knowledge of Welfare and Employment policy and research.

When commissioning the research, a senior civil servant (DWP, Labour Market Strategy Unit) stated that the University of York was chosen because of its reputation as a world-leader in applied conversation analytic research. Drew is one of the most highly cited researchers in his field and is renowned for his work on interaction in organisations. This study built directly on his impressive track record, developed with colleagues at York since the mid-1970s.

The focus for this study was adviser-claimant interviews conducted in Jobcentre Plus, the aim of which is to help benefits claimants (where possible) to progress towards employment. Civil servants at the DWP believed there was a critical gap in the evidence available to policymakers responsible for this interview system: although advisers are considered central to the Government's goal of providing claimants with personalised support, the interviews themselves have remained a `black box'. Thus policy and, crucially, adviser training, have been derived from theory rather than an evidence-based understanding of how these interviews work in practice. We addressed this gap by conducting the first and only study to collect video recordings of advisory interviews in situ in Jobcentre Plus.

We recorded over 200 interviews, including, for comparative purposes, a sample conducted by private sector advisers. The aim was to identify those techniques used by advisers that were most effective in helping claimants progress towards work. We used the fine-grained, highly specialised methods of conversation analysis (CA) to examine not only what was said in the recordings, but precisely how it was said. CA is a qualitative, systematic and technical approach. Analysis involves comparing all instances of key activities conducted in the recorded interactions in order to identify patterns. For instance, we compared the ways in which advisers asked lone parents about their work-related plans. Typically, advisers asked if they were `looking for work at the moment'; but sometimes they asked if they would be `looking for work in the future'. By examining claimants' responses, we showed that the first form of words routinely shut down the discussion (claimants simply said `no', since they were not required to find work until their child was older). By contrast, the second form of words routinely opened up discussion about steps that claimants could take to prepare for work. This was far more effective, then, for addressing the core aim of these interviews: to be work-focused. Through this form of detailed, comparative analysis we showed that advisers were most effective when they were — to put it in non-technical terms — collaborative, directive, proactive, positive, and challenging in their interactions with claimants. For each of these characteristics we provided concrete examples showing how to enact them in practice and what the typical consequences were for the unfolding interaction.

The study was innovative in two main ways. Through our focus on actual practice, we:

1. Addressed the question of `what works' directly. By contrast, previous research has relied almost exclusively on indirect and retrospective evidence, including research interviews with claimants and quantitative outcome measures (e.g. job entry). Such studies cannot illuminate what is effective about the precise ways in which advisers manage the interaction with claimants. Yet this is crucial, as it is the only form of support over which advisers have direct control.
2. Produced effective practice recommendations that are concrete and detailed, by showing, for example, the difference small changes in wording can make. By contrast, other recommendations in this area have tended to be very general, based on theories of communication (e.g. ask `open questions'), rather than on specific evidence of what works with claimants in practice.

References to the research

Findings from this research have undergone rigorous peer review for academic publication, including for the high-ranking international Journal of Social Policy. They were also circulated to stakeholders through regular Working Papers and our final report, which all underwent extensive review within the DWP. The report is publicly available on the DWP's website.

Final Study Report

Academic publications

DWP Internal Working Papers (available on request)

DWP Working Paper No.1 The Aims and Methodology of the Study: Conversation Analysis and Work Focused Interviews in JCP and EZ Offices (May 2008); DWP Working Paper No.2 Initial WFIs with Incapacity Benefits Customers (Pathways to Work) (May 2008); DWP Working Paper No.3 Initial and Review WFIs with Lone Parent Customers (October 2008); DWP Working Paper No.4 A Comparison Between WFIs in JobCentre Plus and Employment Zone Offices (November 2008); DWP Working Paper No.5 New Jobseeker Interviews with JSA 18-24 and 25+ Customers (March 2009).

Grant details

Title	A study of language and communication between advisers and claimants in Work Focused Interviews
Awarded to	Professors Paul Drew & Roy Sainsbury
Period	April 2007-September 2009
Sponsor	Department for Work and Pensions (contract no. PRO 2198)
Value	£366,015

Details of the impact

This study was commissioned by the Department for Work and Pensions (DWP) to feed directly into policy, training and practice. On the basis of our research, we have brought about changes within DWP and Jobcentre Plus. The following impacts can be highlighted:

4.1 Through regular presentations, Working Papers, and our final report, our research substantially increased policymakers' understanding of a key area — advisory interviews - over which they have policy control (see 5.1, below). As a civil servant based in the Jobcentre Plus Strategic Planning Division commented in June 2009 (see 5.2, below): "Your papers have proved very useful... as we develop our strategy story. I think much of what your papers tell us is about how we approach and deal with our future learning routeway. Our Learning and Development colleagues will probably be adamant that they encourage open questioning as part of their training techniques etc., but your research offers so much more and I think we need to take that on board".

This same civil servant (now a DWP manager for the Fraud National Account Team) more recently commented that the study's impact has travelled beyond its initial remit: "The skills and experiences I picked up from [this] work and the video evidence she [Toerien] shared from the joint workshops we held, have helped me, four years later, to consider how I can better equip my current team, who focus on interviews where benefit fraud is the primary driver, to be more effective and successful in asking the right questions and then responding to the customer in often tense and demanding interview situations" (April 2013).

The study's reach has also extended beyond its immediate users (in Jobcentre Plus and DWP) to the Citizens' Advice Bureau (who cite our research in their 2010 report, "Fair welfare: supporting claimants into work") and to the senior committee advising Government on social security policy - the Social Security Advisory Committee (SSAC). In March 2010, we were invited to present our findings to SSAC, allowing them to see for the first time how advisory interviews are actually conducted. Thanked by the Chair for "an open discussion with Members", we showed clips from our recordings which led the committee to reflect on how the interviews might be improved. The meeting minutes can be provided for audit purposes (see 5.3, below).

The relevance of our research has continued to be recognised following the 2010 change in Government and subsequent changes to the benefits system. In March 2013, we were invited to present our findings at the Behaviour Change and Adviser Effectiveness Workshop, organised by the DWP (see 5.1). There were over 200 attendees, including senior policymakers and advisers. As a result, we have subsequently met with the Principal Psychologist at the DWP, to consult on how our work might contribute to a `behavioural framework' (still under development) for DWP at both policy and delivery levels (see 5.2).

4.2 DWP's approach to evaluating advisory practices in Jobcentre Plus has changed as a result of our study. This is evident in two subsequent pieces of research. First, on the basis of our interim results, DWP commissioned us to assess — using the same methodology — whether there was evidence of age-based discrimination in advisory interviews (DWP report no.634, HMSO; see 5.4). Second, citing our final report, an evaluation by the Department for Business, Innovation and Skills used our criteria for assessing the effectiveness of adviser techniques and styles. The evaluation was commissioned specifically to shape Government reforms of the welfare-to-work programme (see 5.5).

4.3 The evidence base generated by our study has contributed directly to policy changes concerning the content and conduct of advisory interviews. This is evidenced by two announcements made in the official DWP Press Releases (June 2010) for the main study report and the related report on age-based discrimination (see 4.2, above). The former states that: "Since the research was conducted in 2007 and 2008, Jobcentre Plus has introduced a range of materials for Advisers and their Managers which draws on findings from the study" (see 5.6, below). The latter states that: "As a result of this research more training is being introduced to help advisers understand the detailed issues faced by some people over the age of 50" (see 5.7, below). A senior civil servant based in the DWP (Labour Market Strategy Unit) also noted that: "The findings from this research have already helped inform decision-making around the moving/removal of a claimant segmentation tool which was shown to be impeding the Work Focused Interviews between advisers and some claimants. Their research results have deepened our understanding of what makes an effective adviser" (April 2013, see 5.2). Since DWP policy determines what advisers are expected to cover with claimants on the frontline, such changes impact advisers' day- to-day practice and, hence, the quality of support received by claimants.

4.4 Our findings have been translated directly into an evidence-based training package for advisers. At the invitation of the DWP — and part-funded by them — we held a two-day workshop at York in June 2013 (see 5.1). The workshop offered intensive, tailored, interactive training for fifteen advisers with the goal of enabling them to implement our findings effectively when talking with claimants. Feedback showed the value of using recordings of real interviews in training: "Brought home how much we need to simplify communications instead of overloading claimant — particularly at first interview" (anonymous feedback from participant). Run as a pilot for a forthcoming DWP-led randomised controlled trial of different approaches to adviser training, this workshop may be rolled out across Jobcentre Plus nationally (depending on the results of the trial). A senior civil servant based in the DWP (Labour Market Strategy Unit) noted: "This research has had both impacts in the immediate and potentially long term performance gains. We are now using the results of this research to develop and test a CA informed adviser training programme. The results of this research have the potential to change the whole adviser training approach" (April 2013, see 5.2).

In sum, our research has had a significant impact beyond academia, with beneficiaries including those on the frontline (advisers), those expected to obtain personalised support from advisers (claimants), those who manage Jobcentre Plus and DWP, and those with responsibility for UK social security policy. The evidence presented in this case study shows not only impact to date, but the strong potential for ongoing impact, as our findings continue to be used to improve the conduct of adviser-claimant interviews in Jobcentre Plus.

Sources to corroborate the impact

The following items are either available in the public domain (see websites listed below) or can be made available for audit purposes on request. For each, we have highlighted which key impact it corroborates (i.e. 4.1 - 4.4 above).

5.1 Senior civil servant, DWP (see 2; 4.3; 4.4) Working papers, and handouts and slides for presentations conducted throughout and after the study (see 4.1).

5.2 Email correspondence with DWP employees (see 4.1).

5.3 Minutes of Social Security Advisory Committee (SSAC) for 3rd March 2010 (see 4.1).

5.4 Irvine, A., Sainsbury, R., Drew, P. and Toerien, M. (2010). An exploratory comparison of the interactions between advisers and younger and older clients during work focused interviews, DWP Research Report No. 634. (HMSO publication, 130 pages). Available at:
http://webarchive.nationalarchives.gov.uk/20130314010347/http://research.dwp.gov.uk/asd/asd5/rports 2009-2010/rrep634.pdf (see 4.2).

5.5 Department for Business Innovation and Skills, Research Paper No.43 (2011) Identifying Claimants' Needs: Research into the Capability of Jobcentre Plus Advisors. (A.Bellis, J.Oakley, M.Sigala & S.Dewson) (direct use of our findings on page 21, which are drawn on in section 5.1 of the paper) (see 4.2).

5.6 Official DWP Press Release for the main study report available at: https://www.gov.uk/government/news/publication-of-dwp-research-report-633-a-study-of-language-and-communication-between-advisers-and-claimants-in-work-focused-interviews (see 4.3)

5.7 Official DWP Press Release for the follow-up study of 50+ claimants available at: https://www.gov.uk/government/news/publication-of-dwp-research-report-634-an-exploratory-comparison-of-the-interactions-between-advisers-and-younger-and-older-clients-during-work-focused-interviews (see 4.3).

5.8 Anonymous feedback on Adviser workshop — available to the panel on request (see 4.4).