Wednesday, 24 May 2017

An interesting and problematic study from Northern Ireland about iPads
in early-years settings hit the interweb today. Interesting because it makes
some extraordinary claims about their efficacy that, if true and replicable,
could revolutionise the way we teach in those settings; and problematic because that ‘if’ has
a lot of heavy lifting to do.

The study ‘Mobile Devices in Early Learning’ was carried out
for two years and involved 650 pupils in five Belfast primary schools and five
nursery schools.

‘Schools which took part were in some of the most deprived
areas of the city.They were each supplied with sets of iPads for nursery,
primary one, primary two and primary three classes.’ (1)

What did they find? Fans of chalk boards and cuneiform look
away now:

The
introduction of digital technology has had a positive impact on the
development of children's literacy and numeracy skills

Contrary
to initial expectations, principals and teachers report that the use of iPads
in the classroom has enhanced children's communication skills

Children
view learning using handheld devices as play and are more highly
motivated, enthused and engaged

Boys
appear to be more enthused when using digital technology, particularly
when producing pieces of written work (2)

Impressive stuff, and these findings represent prizes we all
value: improved gateway skills, engagement, enjoyment, motivation. Game over
for sceptics surely? Alas, one Boss-level obstacle remains. Is it true?

The quotes above are taken from a news website, which
only describes the authors' findings. But in order to understand if
the research findings are robust, and that they flow from the iPad
intervention, we need to be able to access methodology, study design,
attainment measures and so on. We need to hear a critical voice to contract with the claims. Otherwise we could just say anything.

Thrilling sub-heading supported by weak evidence in paragraph 14

What’s wrong with reporting like this? In my opinion, it's unhelpful. In fact I think taken as a whole it makes the business
of knowing how to educate children harder. Because if we want to make sure that what we do
with children in classrooms is useful rather than frivolous, it’s important
that claims of efficacy are matched by evidence, and extraordinary claims
matched by extraordinary evidence. This project set the Belfast Regeneration
Project back £300K, with change back for a Solero- or a teacher’s salary for a
decade if you prefer. School budgets are finite systems and getting more finite
by the year.

'Sir! This intervention appears to based on weak findings.'

When we report unconfirmed results like this without
challenge, the intellectual landscape of education discourse is changed subtly.
This news report will be cited somewhere, by someone who wants to bring a cache
of iPads into a school, and someone somewhere will say ‘OK’. That’s great if
they have the effect they claim, but what if they don’t? At best a waste of
money and time. In fact, that’s also the ‘at worst’ scenario, because children-
especially children in deprived areas, don’t have second chances, or time for
expensive substitutes for teaching time. When we report research without
question, it enter the collective psyche as factual: ‘iPads make kids smarter
and happier.’ But what if they don’t? And I don’t have skin in this game. I
love iPads. But I also loved Tom Hardy’s performance in Taboo, and I’m not
using that in any lessons soon because there is no obvious reason for me to do
so.

Show me the money

Ok, so go beyond the slightly breathless news report. Where is the
research itself?

The article doesn’t link to anything we can look at, so a
quick search reveals that this study is:

‘Gray, C., Dunn, J., Moffett, P., & Mitchell, D. (2017).
Mobile devices in early learning. Developing the use of portable devices to
support young children's learning. Stranmillis University College: A College of
The Queen's University of Belfast, 24.05.2017’

To the website, Robin. Over at Stranmillis University
College, we find a link to a press release, where one of the report’s author’s
makes these claims:

“The study’s findings showed that, in the five participating
schools, all of which were located in catchment areas of high social
deprivation and academic under-achievement, the introduction of digital
technology has had a positive impact on the development of pupil literacy and
numeracy skills. And, contrary to initial expectations, principals and teachers
also reported that their use had enhanced children’s communication skills,
acting as a stimulus for peer to peer and pupil to teacher discussion.” (3)

There’s a link at the bottom of this breathless
review, but it doesn’t work- happily the study is elsewhere on the website (4).

1. Completely subjective self-reporting: If you were hoping to find some evidence that children's literacy or numeracy had been demonstrably improved in an objective way, you will go home with empty pockets. All the evidence collected in this areas was in the form of semi-structured interviews with teachers, school principals, student focus groups and parental questionnaires. So the teachers (small focus groups from each of the 5 schools and pre-schools) said things like 'I think they've improved their literacy.' How do we know this? How can we separate any gains from normal progress, or progress attributable to other interventions or processes?

2. Questionnaire response rate: 27% (after a second push- the first response was 8%), which seems to my mind to be a poor response. We have no way of knowing how representative this is (although I'll suggest 'not very')

3. Possible design biases: schools were selected to participate in this project based on their commitment to the project, their pre-existing use of ICT and iPads in the school, and their commitment to use iPads in the future, as well as a troubling commitment to 'The benefits of developing literacy and numeracy skills to be gained from the use of iPads.' So, to summarise: schools that were enthusiastic about iPads, already used them and believed they had big educational benefits, participated. 'Person who likes x, thinks x is good' isn't so much a research finding as a disappointing maxim in a fortune cookie.

4. Variable usage: schools used them at different times, with different apps, in different ways, with different children. In some schools they were used more than others. It seems very hard to discern if like is being compared with like.

5. Funding. This whole program came about because the Belfast Educational & Library Board was awarded a grant from the Belfast Regeneration Office to 'develop an ICT program.' Was there sufficient critical examination of the need to do so in the first place? Every study needs to suspend disbelief in its own utility, and question its own existence.

6. No control group. What is this intervention better than?

Duvet days: no longer a get-out from teaching.

This study' findings may well be found to be correct, and I’m
sure that the authors and everyone involved has the best of intentions and
conducted themselves with scruples and integrity. That’s not in question. But questions
are all we have at this stage. All we are holding in our hands is a fog of
grand claims and optimism. Do iPads turn frowns upside down? Do they turn light
bulbs on above confused heads? Are they just a novelty or a distraction? We can’t
tell, not from this. A day of terrific press is great for
the University, but doesn’t help the debate.

Never mind the quality, feel the tech

I’ve looked at a lot of research that often gets used to
support positive claims for the utility of tech in the classroom, and often
they don’t stand up in court. Some of the most duplicitous research I have read
in this area uses proxies of success that are entirely subjective or impossible
to substantiate. ‘tech has the potential to do x’ is the same as ‘tech has not
done x yet.’ And ‘boys appear to be more enthused when using digital technology’
could be uncharitably responded to with a ‘so what?’ and a ‘oh really?’ and a 'did it take a £500 iPad to do that?'

And that’s important, because schools are poor and kids don’t
often get second chances when they come from deprived areas. Universal, free education
is one of humanity’s greatest inventions. Wasting that is a sin, and a theft
from people with nearly nothing. Who would rob a child, from a family with nothing
but debt?

Other people's children

Public money needs to be spent as carefully as if it were
our own. Other people’s children need to be taught as carefully as if they belonged
to us. No child should endure the loss of their right to an education, no
matter how digitally it is dressed. If iPads and their ilk can bring benefit
to the table, then let them demonstrate it in public. Let everyone see how well
they work, and if they do, the truth will be unmistakeable. But when claims are
made without data that substantiates it appropriately then we have a right to ask if our money is being spent wisely. This matters. Ominously, the report suggest that:

'These findings should inform the future rollout of similar initiatives and will be of interest to practitioners, policy-makers and parents.'

Ireland, I love you. My family migrated from Ireland. I wish you and your beautiful island nothing but fortune and love. For the good of your children, and the wealth of your nation, and the prospect of better things to come, I suggest that you use these findings wisely. Keep your hands away from the cheque books for now and wait until better data supports swapping out precious resources for digital magic beans.

I'll end with a lovely quote from Piaget, which starts the report:

'The principal goal of education is to create men and women who are capable of doing few things, not simply of repeating what other generations have done—men and women who are creative, inventive, and discoverers, who have minds which can be critical, can verify, and not accept everything they are offered (Piaget,1973).'