Research Subject Area(s)

Download original

Summary of the impact

Sure Start, the flagship New Labour anti-poverty initiative launched in
1999, was an area-based early intervention targeting pockets of social and
economic deprivation in England. The multi- disciplinary National
Evaluation of Sure Start (NESS), to which Professor Angela Anning
was the central educational contributor, was the largest social science
evaluation contract ever awarded in the UK. Findings were continually fed
back to ministers and civil servants to inform policy and practice.
Impacts (discussed below, section 4) include:

(1) systemic changes in integrated services for vulnerable families;

(2) rethinking the resourcing and funding of Children's Centres;

(3) revision of training and qualifications of early childhood staff;

(4) enhancing the role of family support and parenting projects.

Underpinning research

Anning (Professor of Early Childhood Education, University of
Leeds, 1997-2004; Research Professor, University of Leeds, 2004-2006) has
researched and written extensively on early years education and services
for children. From 1999 to 2002, she led DfEE-funded evaluations of three
early years Centres of Excellence, which pioneered integrated services,
and, from 2002-2004, she completed an ESRC-funded project with colleagues
from medicine, health care and social policy into multi-agency teamwork.
Both projects informed the multidisciplinary NESS evaluation research. The
impact reported here refers both to Anning's role as a core member
of the Impact team, its central educational contributor and to those
aspects of the evaluation she led personally.

Context:

From 1997-2010, Labour invested massively in Early Years provision.
Radical policy changes included integrating and expanding the early
childhood education and care sectors, initiating multi-agency teamwork and
targeting the whole family. The policies had a particular focus on
improving child health, education and welfare outcomes for those families
deemed to be at risk or vulnerable. Sure Start Local Programmes (SSLPs)
were seminal in modelling new ways of working. Guidance was offered to
SSLPs but the approach was not prescriptive. Programmes were negotiated
with communities and responded to local needs to deliver services. There
were, therefore, wide variations in treatment.

NESS Evaluation:

The Impact Module of the National Evaluation of Sure Start (NESS), of
which Anning was a team member and the sole educationalist from
2000-2008, conducted large-scale surveys of child and families in SSLP
areas and rated programme variability (PV) to investigate characteristics
of the SSLPs found to be more effective in achieving outcomes (5).
It was central to the evaluation of the programme and has played a key
role in shaping subsequent government policies in this area.The Impact
team used an `intention to treat' model on 7,000 children and their
families for a cross-sectional study (at 9 months and 3-years-old) in 150
representative SSLP areas. Early findings found variation in effects on
subgroups within local communities, with some adverse effects on the most
disadvantaged groups and greater impact on parental measures than child
outcomes (6). The final contract for the NESS evaluation in January
2001 specified that an education expert should be part of the evaluation
team. On the basis of her expertise in multi-agency work and evaluation of
integrated services for young children, Anning was contracted for
this key role and worked as the educationalist responsible for the Impact
Module (cf Source C)

The disappointing early results of the cross-sectional study resulted in
SSLPs being rebranded as Children's Centres and taken under Local
Authority control (3). The Impact Team were subsequently asked to
identify characteristics of SSLPs that showed better than expected
outcomes. Anning led this programme variability (PV) research
(2004- 2006) (2). Its purpose was to analyse what made some
programmes more effective than others and what were the qualities of
services connected to better outcomes for children and families. The team
worked `blind' on measuring the processes of more and less effective
delivery of treatments. Statistical analysis demonstrated that variations
in PV measures of SSLPs' effectiveness were linked to more or less impact
(4). Key dimensions of proficiency identified in this research
were: effective governance and management, informal but professional
ethos, and empowerment of service providers and users. Key strategies to
attain proficiency for universal services were: tuning into local
communities and specialist services, early identification and treatments,
recruiting and training quality staff and managing multi-agency teamwork.

The PV findings formed the basis of guidelines for Children's Centres and
more accountability for outcomes. The Impact of Sure Start services on both
parents and children was found to be more positive over time. 2008 findings
in the longitudinal study (using a matched Millennium Cohort Study sample
receiving mainstream services as controls) were positive effects for 7 of 14
outcomes, particularly parental measures, no significant differences in
subgroups; but overall more `advantaged' families gained most. Evidence from
a themed study on the Quality of Early Learning and Play in Sure Start Local
Programmes, published in 2005 and with Anning as the lead
researcher(1), also emphasised the importance of improving the child
focused activities, and had an impact on wider discussions on the need to
improve the training requirements for childcare workers.

[3] Belsky J, Melhuish, E. and Barnes, J. and the NESS Research
Team(2008) Research and Policy in Developing an Early Years Initiative:
The Case of Sure Start. International Journal of Child Care and
Education Policy. 2, 1-13.

[5] Anning, A., Ball, M., Belsky, J., and Melhuish, E. (2007).
Predicting impact in an early years intervention: the design of a tool
using qualitative and quantitative methods. Journal of Children's
Services, 3, 27-42.

Quality of the research: Sir Michael Rutter in an independent, project
external contribution to [7] wrote of the work produced by NESS `as
rigorous and careful an evaluation as could be undertaken...high quality
research... every reason to trust the research findings' 7,
p.204). [3] and [4] above are included to represent
publications based on the NESS Impact team findings in high quality
international journal, providing an additional quality indicator for the
research.

Details of the impact

The NESS evaluation, and Anning's work as the educationalist and
core member of the impact module of the National Evaluation of Sure Start,
had an immediate impact on the development of Sure Start services. The
Director of the Sure Start Programme (1999-2006) states: "Sure Start had a
profound and direct impact on the way early years services are delivered
in England. In particular, early disappointing Sure Start impact results
published in 2006 led to an in depth study on programme variability for
which Anning was the project leader ... The findings from this
study had a direct impact on the requirements for the great expansion of
Children's Centres that followed. Of particular relevance was the
importance of good local data systems to understand local needs, and what
groups were not using Sure Start services, and the importance of matching
local empowerment strategies with a clear focus on activities that would
enhance child development." (H) The Sure Start Director (1999-2006)
continues: "Evidence from the themed study on the Quality of Early
Learning and Play in Sure Start Local Programmes (Anning et al
2005) was also was used by civil servants to ensure local programmes were
improving the child focused activities, and had an impact on wider
discussions on the need to improve the training requirements for childcare
workers." (H).

Ultimately, the significance of this case lies in Anning's role
in promoting enhanced life chances for children in poverty or at risk (D)
and over 3,000 children's centres across England continue to deliver
provision shaped by the Impact Module of the NESS evaluation. Reports,
accessible on the DfE website, are widely used by professionals and
students. NESS publications from the Impact Team provided the evidence
base for policy changes to combat poverty, educational failure,
worklessness, family and relationships breakdown. The Chief Research
Officer at the Department for Education in 2012 states in a letter to
support this case study: "On behalf of the Department for Education, I am
pleased to confirm that the Impact Module of NESS findings informed
systemic changes in services for families with young children defined as
at risk/vulnerable. In particular, though the concept of multi-agency
teams delivering universal services remained central to successive Labour
and Coalition government policies, NESS Impact Module findings evidenced
the need to target specialist expertise at point of needs (examples being
the enhanced role of family nurse/health visitors and day-care staff for
identifying and responding to needs for two-year-olds deemed to be at risk
of health, welfare or educational related targets.)" (G). The
Minister of State for Children from 2005 to 2009 also stressed the
continuing impact of the NESS evaluation: "During the period 2008-2013,
cumulative findings from the National Evaluation of Sure Start impacted on
policy and practice in early years services for young children defined as
vulnerable/at risk and their families." (J).

During the period 2008-2013 cumulative NESS findings of the Impact Module
informed:

(1) Systemic changes in integrated services for vulnerable families
The concept of multi-agency teamwork remained central to Labour and
Coalition policies. NESS evidence clarified the need to target specialist
expertise at point of need (e.g. the enhanced role of family nurses/health
visitors) whilst multi-agency teams delivered universal, non-stigmatised
services. Evidence of the importance of targeting 2-year-olds at risk
resulted in extending free childcare to them (A).

(2) Rethinking resources and funding of Children's Centres The
average funding allocated to SSLPs/CCs was £500,000 revenue per year.
Early disappointing NESS findings on Impact in 2005 mediated by PV
evidence of `what worked' formed the basis of guidance and increased
accountability for all Children's Centres. Continuing this impact from
2012, the Coalition government instigated payment by results (e.g., for
take up of free nursery places by 2 and 3-year-olds) (B).

(3) Revision of training and qualifications of early childhood staff
Evidence of the difficulty of operationalising integrated services
resulted in upgrading the workforce. As a result of this evidence Early
Years Professional Status (EYP), and masters level Leadership and
Management qualifications (NPQICL) for managers were introduced .
References [1] and [2] are referenced in Families in the
Foundation Years Evidence Pack: [2] on Page 14, Anning, [1]
on Page 29. This Evidence Pack is a key guidance document for EY services
(E).

(4) Enhancing the role of family support and parenting projects By
2008 NESS Impact findings (C) indicated positive effects of the
intervention. Of 14 outcomes 7 showed significant differences between SSLP
and non-Sure Start (MCS sample) areas. These included: improved child
social behaviours, improved child independence and self-regulation, less
harsh discipline and home chaos, improved home learning environments,
parents making more use of services, higher rates of immunisations and
fewer child accidents . In 2012, NESS reported significant effects on 4 of
15 parental outcomes in survey of 7-year-olds (D). Children's
Centres are perceived to work and maintain a central role in supporting
vulnerable families with young children. The Director of the Sure Start
Programme from 1999 to 2006 stated in a letter to support this case study,
written in 2012: "Sure Start has had a profound influence on key
principles in the delivery of children's services across England: the
importance of working with adults as well as children, the importance of
education, social care and health working together for family welfare, and
the difficulty of offering universal services that are successful in
reaching and improving outcomes for the most disadvantaged families have
all been part of the original Sure Start thinking. The National Evaluation
of Sure Start provided crucial evidence on how such services should be
delivered, what we got wrong in the original design, and the changes to
the original design that resulted in the delivery of over three thousand
children's centres across England. The evaluation results will continue to
play a key role in the arguments going forward on childcare and early
education policy."[H] Evidence of continued impact, particularly of
the PV research which Anning led, continued after 2010. The Coalition
Government shifted policy imperatives for Children's Centres (funded
directly by the government) to focus on the neediest families and to
involve charities and private companies with a track record of helping
parents [B].

[C] Melhuish, E and the National Evaluation of Sure Start Research Team
(2008) The Impact of Sure Start Local Programmes on Three Year Olds
and their Families, Sure Start Research Report 027, Nottingham:DfES
http://www.ness.bbk.ac.uk/impact/documents/42.pdf