1 Introduction and background

Introduction

The Scottish Schools Adolescent Lifestyle and Substance Use
Survey (
SALSUS)
is a continuation of a long established series of national surveys
on smoking, drinking and drug use.
SALSUS
measures progress towards Scottish Government targets for smoking
and drug use, and is used to inform the Scottish Government
priority for addressing harmful drinking among young people. The
survey series also provides local prevalence rates for smoking,
drinking and drug use across Alcohol and Drug Partnerships (
ADPs),
local authorities and
NHS Boards. More
information can be found at:
http://www.isdscotland.org/Health-Topics/Public-Health/SALSUS/.

SALSUS
currently uses a paper, self-completion questionnaire, administered
in class in schools under exam conditions. However, as technology
has advanced, a transition from paper to online administration was
trialled during the 2015 wave of
SALSUS.
There are two main reasons to move from paper to online:

cost efficiencies: with paper surveys, printing and postage
alone make up a substantial proportion of the costs and there are
only minor economies of scale with a larger sample size. While
online surveys are more expensive to set up, beyond a certain
sample size, increasing numbers only results in a small cost
increase.

improved data quality: online surveys provide greater control
of the responses given by pupils, particularly in relation to
complex routing.

However, previous research
[2,3,4] and experience
[5] suggests that it can be harder for schools to administer
online surveys, which may have an impact on response rates. There
may also be a mode effect i.e. pupils give different responses
depending on whether they are completing the survey online or on
paper.

Ipsos
MORI
Scotland was commissioned to undertake the 2015 wave of
SALSUS
and, as part of that contract, to conduct research to explore the
administration of the survey online. This included a feasibility
study, an online pilot
[6] and a mode effect study. The methodology for each phase is
summarised in Figure 1.1.

Figure 1.1 Summary of methodology for the three online
feasibility phases

The feasibility study

The aim of the feasibility study was to explore teachers, head
teachers and local authority
IT officers'
views on the viability of changing the mode of data collection for
SALSUS
from paper to online.

It was found that the majority of local authorities (24 out of
32) would be able to complete
SALSUS
2015 online. While eight local authorities were classified as
possibly finding it problematic, to some extent this was due to a
lack of information as there was a low response rate from both head
teachers and liaison teachers in these areas.

The results suggested that an online survey could result in a
reduced response rate. Participants in the study highlighted a
number of potential risks when completing the survey online
including timetabling issues, a lack of computers and software
compatibility. Nonetheless, liaison teachers thought that moving
the survey online would be more enjoyable for pupils. Only a small
proportion thought it would have a negative impact on pupils'
honesty or concentration.

The online pilot

Subsequent to the feasibility study, an online pilot was
conducted. The aim of the pilot was to test concerns raised in the
feasibility study and to highlight any technical problems in
advance of rolling out online administration to a much larger group
of pupils.

Taking into consideration the issues highlighted in the
feasibility study, the electronic pilot focussed on the following
areas:

Overall, pilot schools found that administering the survey
online was relatively straightforward. Pupil reactions were
positive and they preferred to complete the survey online rather
than on paper.

Both liaison and class teachers were positive about
administering the survey online. The need to book Information and
Communication Technology (
ICT)
suites (or to book laptops or tablets for the classroom) meant that
aspect was more burdensome than administering a paper survey and
required more advance planning - but it was possible. However, they
also reported that once that aspect was arranged, the actual
administration of the survey with the class was much easier.

On the basis that there were no unresolvable problems
identified, the decision was taken to undertake the mode effect
study.