11. Views On Designing Trials For Scalability

Background The Scottish Government wishes to maximise the value of the
ELC
programme by ensuring that the lessons learned from trials can be
applied to similar
ELC
settings in Scotland. Consideration is required on what needs to be
factored into the design of the trials to enable this.

Question 10: How can we design the trials in such a way as
to ensure scalability?

11.1 44 respondents addressed this question. An overarching
theme across a wide range of responses was that trials should be
set up from the outset as
learning mechanisms intended to produce lessons of
more general applicability.

11.2 A few respondents recommended that
thorough groundwork be undertaken prior to setting
up trials including examining evaluations of trials in other
jurisdictions, consulting with key stakeholders and ensuring common
understandings of concepts such as "playful approaches to arts and
creativity".

11.3 17 respondents from a wide range of sectors emphasised the
need for
trials to test a variety of settings and provider
models; patterns of provision (
e.g. seasonal); scale of
budget; size; rurality; "hard to reach" communities; outdoor space
access. By doing so, they envisaged that aspects of the trials
would be relevant to areas throughout Scotland.

11.4 11 respondents recommended that
robust evaluative methods are used to assess the
trials. Standardised approaches to evaluation with measurable
outcomes were advocated, with appropriate indicators and data
collection mechanisms in place from the start. One respondent
called for "experts" to conduct the evaluations which some
emphasised should be done with rigour. The challenge of time needed
for outcomes to be realised was highlighted once again, with one
respondent identifying the need for:

".....a reasonable timeframe for the trials so that approaches
can be applied effectively without appearing to be a 'quick fix'
and ending up being ineffective in the long term when rolled out"
(Voluntary organisation).

11.5 A few respondents specified the importance of
establishing baseline data; comparator areas where
the same provision is trialled but in different settings;
representative samples; and qualitative data collection during
interviews with parents and other key stakeholders, in addition to
quantitative measurement.

11.6 It was suggested that the Care Inspectorate Registration
Team could usefully provide the Scottish Government with
information on the pattern and location of variation to existing
services and any new applications to register services in order to
provide a database of provision as a backcloth to the trials.

11.7 28 respondents identified broad ways to set up the trials
to make them mechanisms for shared learning. A few recommended
establishing "cluster groups" which involve all providers in one
area, supported by regional funding.

11.8 Two respondents referred to the "test of change" model
established through the Early Years Collaborative as having
potential for application to the trials. A voluntary organisation
suggested action research as an appropriate approach to learning
from the trials, or a "quality improvement model".

11.9 Two respondents emphasised their view that trials should be
kept simple; a local authority recommended focusing trials on key
themes and core concepts. One representative body suggested that
standardisation across trials, for example, in childminder pay,
would aid scalability.

11.10 A common view amongst five respondents from different
sectors was that the trials should be
underpinned by realistic budgets to provide a
genuine picture of potential for expansion. One respondent,
however, cautioned that limited funds should not be wasted:

"The government should ensure that what money is available for
trials is not wasted on never to be employed initiatives - we know
what the problems are with early learning and care provision. Hence
a handful of excellently funded initiatives targeted in specific
local authorities and issues (universality, rural provision,
disability services, poverty mitigation childminding,
etc.), because of their nature,
would be better than a plethora of trials that didn't amount to
anything" (Other body).

11.11 Eight respondents from a range of sectors identified
robust dissemination of lessons, including sharing
events run by the Scottish Government, as necessary to ensure
issues of scalability can be addressed. One local authority
suggested that there are currently effective local models of
provision which could be usefully shared and scaled up and these
could be the focus of early dissemination.

11.12 Several local authorities and others cautioned that it is
unlikely that successful approaches can be transported wholesale to
settings elsewhere in Scotland, but that
aspects of approaches are more likely to be
relevant to other areas. In learning from trials, they emphasised
the need to identify pilots which could be
tailored for other local circumstances, with
aspects extrapolated for scaling up elsewhere.

11.13 Four respondents from different sectors emphasised the
need for the
supporting infrastructure for trials to be
recognised as crucial and not simply the initiatives
themselves. For example, appropriate funding and staffing were
crucial for other areas looking to replicate successful aspects of
trials, demonstrating the importance of transparency over the
structural resources required for the trials.

Implications for proposed trials

11.14 In order for successful aspects of trials to be scaled up,
respondents identified a number of design features to be
incorporated from the start. These included ensuring they are set
up as learning mechanisms; are simple and focused to enable lessons
to be clearly identifiable; that thorough groundwork is undertaken
to make sure they are evidence-based and testing clearly specific
concepts and approaches; that they test a variety of settings which
will produce relevant learning for different locations across
Scotland; that they are underpinned with robust evaluative methods
and have firm baseline data; that budgets for the trials are
realistic and provide a genuine picture of funding and supporting
infrastructure required; and that dissemination of lessons is
robust and information widely shared post-trial.