3. Performance in Collaborative Problem Solving

Average scores

19. In collaborative problem solving, Scotland scored
513 in
PISA
2015.
Chart 3.1 illustrates Scotland’s
score with 95-per-cent confidence intervals
[2] next to the scores for the
OECD
average. Note that Scotland, with a smaller sample, has larger
confidence intervals than does the
OECD
average, where the combined sample makes for more certainty.

20. Scotland’s score was above the
OECD
average in 2015. The
OECD
average is set at 500, reflecting that collaborative solving is a
new domain.

Comparisons with other countries

21. Compared to the 31
OECD
countries
[3] who participated, plus the three other
UK administrations,
Scotland performed similarly to six countries, including Northern
Ireland and the United States, and above 19 countries including
Wales, France, Norway and Italy. Nine countries performed above
Scotland, including Canada, Germany, Korea and England.
Table 3.1 below shows which countries were
statistically significantly above, similar to and below Scotland in
2015.
Table A.1a, located in the annex, shows
each country’s score.

Table 3.1:
OECD
countries and
UK administrations,
higher than, similar to and lower than Scotland in collaborative
problem solving

22. Among the participating non-
OECD
countries and economies, Singapore (the highest scoring
participant), Hong Kong-China, Chinese Tapei and Macao-China
performed significantly higher than Scotland. No country was
similar to Scotland. Sixteen countries and economies performed
significantly below Scotland, including Brazil, the Russian
Federation and the four provinces participating from the
Peoples’ Republic of China (“B-S-J-G”
[5]).

23. Table A.1b (annex) records the mean scores for the non-
OECD
member states. The
OECD
reports published at the same time as this report have full details
on all countries’ scores, and also where data has been
collected on a “regional” basis (including
Scotland).

Distribution of scores

24. As well as comparison between countries’ mean scores,
it is important to look at how these are distributed within a
country. It is likely that there is much more variation within than
between countries. Scotland’s spread of performance in
collaborative problem solving, as measured by standard deviation of
the scores (99 points), was larger than the
OECD
average (95 points). Seventeen countries had a narrower
distribution, eleven similar and six greater.

High and low achievers

Low performance (below Level 2)

25. As set out in
Chapter 2, the
OECD
categorise students into levels according to their ability to
undertake certain tasks. However, the group below Level 2 merits
particular attention, as the
OECD
consider that Level 2 is the baseline of ability to participate
effectively in society. 23.8 per cent of 15 year-olds in Scotland
performed below Level 2 in 2015 – lower than the
OECD
average of 28.1 per cent.

High performance (Level 4)

26. At the other end of the distribution, the proportion of
students who were “higher” achievers (Level 4) was 9.8
per cent, greater than the
OECD
average of 7.9 per cent.

27.
Chart 3.2 below shows the distribution of
scores in Scotland compared to the
OECD
average.
Table A.2 (annex) shows each
OECD
country and
UK administration’s
distribution of scores by proficiency level.

28. Girls in Scotland performed better than boys. The average
score in collaborative problem solving was 530 for female students,
and 497 for male students. The gap (33 points) was similar to the
OECD
average (29 points).

High and low achievers

29. In terms of the gender share of higher and lower achievers,
girls were more likely to record high performance (Level 4) than
boys (12.4 per cent of girls and 7.3 per cent of boys). 29.0 per
cent of boys and 18.5 per cent of girls were below Level 2 (a
significant difference between boys and girls at Level 1, but not
Below Level 1).

30. Girls recorded a higher share of Level 4 performers than the
OECD
average. Boys’ share was similar to the
OECD
average. Boys recorded a lower share of low performers than the
OECD
average for Below Level 1 only. Girls recorded a lower share of low
performers for Level 1 only.

Social background

31. The
OECD
measure social background using their own Index of Economic, Social
& Cultural Status (
ESCS)
calculated from data given by students in the background
questionnaire. A number of indicators are used to explain how much
performance is related to student background. Further information
is provided in the Scottish Government’s previous
PISA
report, and the
OECD
volumes.

32. The
share of variation[6] in test scores for collaborative problem solving that was
explained by students’ background was 5.6 per cent. This was
lower than the
OECD
average (7.9 per cent) This means that Scotland’s pupils were
more likely to break away from the pattern of background affecting
performance. Although there was still a clear link between
background and performance, there are other things that affect
performance, and many pupils do not follow the pattern.

33. The
gradient in Scotland, i.e. how much the mean
performance score in collaborative problem solving changes with
social background, was 28 points. This was statistically similar to
the
OECD
average of 30 points and was greater than in five countries,
similar to 21 and less than eight others.

34. The difference between the 5
th and 95
th percentiles by
ESCS
was 2.63 points. Combined with a 28-point gradient, this implies
that the average scores in collaborative problem solving between
the most disadvantaged and least disadvantaged students are apart
by nearly 74 points. If, as the
OECD
surmises with maths, reading and science, a year’s schooling
is equivalent to about 30 points, that would imply a difference of
nearly two and half years’ education.

35.
Table A.3 (annex) has each
OECD
country and
UK administration’s
scores on the gradient and strength of relationship.

Students and immigration background

36. The survey also asks about students’ background in
terms of whether they or their parents were born outside the
country of the test (for these purposes, the
UK). In 2015, students
without an immigrant-background performed similarly to those with
an immigrant-background (defined as both parents being born outside
the
UK). This contrasted with
the
OECD
average, where Scotland’s (statistically non-significant) gap
of 4 points in favour of immigrant students, compares with a
statistically significant gap of 36 points in favour of
non-immigrant-background students. This gap in turn was
significantly different to Scotland’s. This indicator is also
reported in
Table A.3 (annex) for each
OECD
country and
UK administration.