Summary Impact Type

Research Subject Area(s)

Download original

Summary of the impact

The impact of this research has been achieved through developing
evidence-based recommendations for personal advisers in Jobcentre Plus —
the UK's one-stop service for administering state benefits and helping
claimants into work. By opening the `black box' of adviser- claimant
interviews for the first time, the study produced the following key
impacts:

Policymakers in the Department for Work and Pensions (DWP) and on the
Social Security Advisory Committee (SSAC) gained an evidence-based
understanding of a key area over which they have policy control;

Through these policy changes and our training workshops,
recommendations from our study have helped improve the service
delivered by advisers to benefits claimants.

Beneficiaries were those claiming state benefits, Jobcentre Plus advisers
and managers, and DWP and other Government policymakers.

Underpinning research

This research was conducted at the University of York from April 2007 to
September 2009, and was commissioned by the UK Government's Department for
Work and Pensions (DWP). DWP funding (£366,015) supported a research team
at York that combined twin sets of expertise:

Professor Roy Sainsbury and Ms Annie Irvine (Research Director and
Research Fellow, respectively, in the Social Policy Research Unit)
brought substantive knowledge of Welfare and Employment policy and
research.

When commissioning the research, a senior civil servant (DWP, Labour
Market Strategy Unit) stated that the University of York was chosen
because of its reputation as a world-leader in applied conversation
analytic research. Drew is one of the most highly cited researchers in his
field and is renowned for his work on interaction in organisations. This
study built directly on his impressive track record, developed with
colleagues at York since the mid-1970s.

The focus for this study was adviser-claimant interviews conducted in
Jobcentre Plus, the aim of which is to help benefits claimants (where
possible) to progress towards employment. Civil servants at the DWP
believed there was a critical gap in the evidence available to
policymakers responsible for this interview system: although advisers are
considered central to the Government's goal of providing claimants with
personalised support, the interviews themselves have remained a `black
box'. Thus policy and, crucially, adviser training, have been derived from
theory rather than an evidence-based understanding of how these interviews
work in practice. We addressed this gap by conducting the first and only
study to collect video recordings of advisory interviews in situ
in Jobcentre Plus.

We recorded over 200 interviews, including, for comparative purposes, a
sample conducted by private sector advisers. The aim was to identify those
techniques used by advisers that were most effective in helping claimants
progress towards work. We used the fine-grained, highly specialised
methods of conversation analysis (CA) to examine not only what was said in
the recordings, but precisely how it was said. CA is a qualitative,
systematic and technical approach. Analysis involves comparing all
instances of key activities conducted in the recorded interactions in
order to identify patterns. For instance, we compared the ways in which
advisers asked lone parents about their work-related plans. Typically,
advisers asked if they were `looking for work at the moment'; but
sometimes they asked if they would be `looking for work in the future'.
By examining claimants' responses, we showed that the first form of words
routinely shut down the discussion (claimants simply said `no', since they
were not required to find work until their child was older). By contrast,
the second form of words routinely opened up discussion about steps that
claimants could take to prepare for work. This was far more effective,
then, for addressing the core aim of these interviews: to be work-focused.
Through this form of detailed, comparative analysis we showed that
advisers were most effective when they were — to put it in non-technical
terms — collaborative, directive, proactive, positive, and challenging in
their interactions with claimants. For each of these characteristics we
provided concrete examples showing how to enact them in practice and what
the typical consequences were for the unfolding interaction.

The study was innovative in two main ways. Through our focus on actual
practice, we:

Addressed the question of `what works'directly.
By contrast, previous research has relied almost exclusively on indirect
and retrospective evidence, including research interviews with claimants
and quantitative outcome measures (e.g. job entry). Such studies cannot
illuminate what is effective about the precise ways in which advisers
manage the interaction with claimants. Yet this is crucial, as it is the
only form of support over which advisers have direct control.

Produced effective practice recommendations that areconcrete
and detailed, by showing, for example, the difference small
changes in wording can make. By contrast, other recommendations in this
area have tended to be very general, based on theories of communication
(e.g. ask `open questions'), rather than on specific evidence of what
works with claimants in practice.

References to the research

Findings from this research have undergone rigorous peer review for
academic publication, including for the high-ranking international Journal
of Social Policy. They were also circulated to stakeholders through
regular Working Papers and our final report, which all underwent extensive
review within the DWP. The report is publicly available on the DWP's
website.

A study of language and communication between advisers
and claimants in
Work Focused Interviews

Awarded to

Professors Paul Drew & Roy Sainsbury

Period

April 2007-September 2009

Sponsor

Department for Work and Pensions (contract no. PRO 2198)

Value

£366,015

Details of the impact

This study was commissioned by the Department for Work and Pensions (DWP)
to feed directly into policy, training and practice. On the basis of our
research, we have brought about changes within DWP and Jobcentre Plus. The
following impacts can be highlighted:

4.1 Through regular presentations, Working Papers, and our final report,
our research substantially increased policymakers' understanding of a
key area — advisory interviews - over which they have policy control
(see 5.1, below). As a civil servant based in the Jobcentre Plus Strategic
Planning Division commented in June 2009 (see 5.2, below): "Your papers
have proved very useful... as we develop our strategy story. I think
much of what your papers tell us is about how we approach and deal with
our future learning routeway. Our Learning and Development colleagues
will probably be adamant that they encourage open questioning as part of
their training techniques etc., but your research offers so much more
and I think we need to take that on board".

This same civil servant (now a DWP manager for the Fraud National Account
Team) more recently commented that the study's impact has travelled beyond
its initial remit: "The skills and experiences I picked up from [this]
work and the video evidence she [Toerien] shared from the joint
workshops we held, have helped me, four years later, to consider how I
can better equip my current team, who focus on interviews where benefit
fraud is the primary driver, to be more effective and successful in
asking the right questions and then responding to the customer in often
tense and demanding interview situations" (April 2013).

The study's reach has also extended beyond its immediate users (in
Jobcentre Plus and DWP) to the Citizens' Advice Bureau (who cite our
research in their 2010 report, "Fair welfare: supporting claimants into
work") and to the senior committee advising Government on social security
policy - the Social Security Advisory Committee (SSAC). In March 2010, we
were invited to present our findings to SSAC, allowing them to see for the
first time how advisory interviews are actually conducted. Thanked by the
Chair for "an open discussion with Members", we showed clips from our
recordings which led the committee to reflect on how the interviews might
be improved. The meeting minutes can be provided for audit purposes (see
5.3, below).

The relevance of our research has continued to be recognised following
the 2010 change in Government and subsequent changes to the benefits
system. In March 2013, we were invited to present our findings at the Behaviour
Change and Adviser Effectiveness Workshop, organised by the DWP (see
5.1). There were over 200 attendees, including senior policymakers and
advisers. As a result, we have subsequently met with the Principal
Psychologist at the DWP, to consult on how our work might contribute to a
`behavioural framework' (still under development) for DWP at both policy
and delivery levels (see 5.2).

4.2 DWP's approach to evaluating advisory practices in Jobcentre Plus
has changed as a result of our study. This is evident in two
subsequent pieces of research. First, on the basis of our interim results,
DWP commissioned us to assess — using the same methodology — whether there
was evidence of age-based discrimination in advisory interviews (DWP
report no.634, HMSO; see 5.4). Second, citing our final report, an
evaluation by the Department for Business, Innovation and Skills used our
criteria for assessing the effectiveness of adviser techniques and styles.
The evaluation was commissioned specifically to shape Government reforms
of the welfare-to-work programme (see 5.5).

4.3 The evidence base generated by our study has contributed directly
to policy changes concerning the content and conduct of advisory
interviews. This is evidenced by two announcements made in the
official DWP Press Releases (June 2010) for the main study report and the
related report on age-based discrimination (see 4.2, above). The former
states that: "Since the research was conducted in 2007 and 2008,
Jobcentre Plus has introduced a range of materials for Advisers and
their Managerswhich draws on findings from the study"
(see 5.6, below). The latter states that: "As a result of this
researchmore training is being introduced to help
advisers understand the detailed issues faced by some people over the
age of 50" (see 5.7, below). A senior civil servant based in the DWP
(Labour Market Strategy Unit) also noted that: "The findings from this
research have already helped inform decision-making around the
moving/removal of a claimant segmentation tool which was shown to be
impeding the Work Focused Interviews between advisers and some
claimants. Their research results have deepened our understanding of
what makes an effective adviser" (April 2013, see 5.2). Since DWP
policy determines what advisers are expected to cover with claimants on
the frontline, such changes impact advisers' day- to-day practice and,
hence, the quality of support received by claimants.

4.4 Our findings have been translated directly into an evidence-based
training package for advisers. At the invitation of the DWP — and
part-funded by them — we held a two-day workshop at York in June 2013 (see
5.1). The workshop offered intensive, tailored, interactive training for
fifteen advisers with the goal of enabling them to implement our findings
effectively when talking with claimants. Feedback showed the value of
using recordings of real interviews in training: "Brought home how much
we need to simplify communications instead of overloading claimant —
particularly at first interview" (anonymous feedback from
participant). Run as a pilot for a forthcoming DWP-led randomised
controlled trial of different approaches to adviser training, this
workshop may be rolled out across Jobcentre Plus nationally (depending on
the results of the trial). A senior civil servant based in the DWP (Labour
Market Strategy Unit) noted: "This research has had both impacts in the
immediate and potentially long term performance gains. We are now using
the results of this research to develop and test a CA informed adviser
training programme. The results of this research have the potential to
change the whole adviser training approach" (April 2013, see 5.2).

In sum, our research has had a significant impact beyond academia, with
beneficiaries including those on the frontline (advisers), those expected
to obtain personalised support from advisers (claimants), those who manage
Jobcentre Plus and DWP, and those with responsibility for UK social
security policy. The evidence presented in this case study shows not only
impact to date, but the strong potential for ongoing impact, as our
findings continue to be used to improve the conduct of adviser-claimant
interviews in Jobcentre Plus.

Sources to corroborate the impact

The following items are either available in the public domain (see
websites listed below) or can be made available for audit purposes on
request. For each, we have highlighted which key impact it corroborates
(i.e. 4.1 - 4.4 above).

5.1 Senior civil servant, DWP (see 2; 4.3; 4.4) Working papers, and
handouts and slides for presentations conducted throughout and after the
study (see 4.1).

5.5 Department for Business Innovation and Skills, Research Paper No.43
(2011) Identifying Claimants' Needs: Research into the Capability of
Jobcentre Plus Advisors. (A.Bellis, J.Oakley, M.Sigala &
S.Dewson) (direct use of our findings on page 21, which are drawn on in
section 5.1 of the paper) (see 4.2).