The ICT Impact Report

EUN has published a major report on ICT impact on schools in Europe. The report, written in the framework of the ICT Cluster
of the European Commission, draws on the evidence from 17 recent impact studies across Europe. It identifies 12 major findings
concerning the impact of ICT on teaching and learning, identifies three types of barriers and makes 13 recommendations to
policy-makers, educators and researchers.

The report is written by Anja Balanskat, Roger Blamire, Stella Kefalla

The “impact studies’ in focus include a variety of studies, large and small scale, national and European, and cover evaluations
of ICT programmes, inspection reports, specific technology interventions, research reviews, European comparisons and case
studies. The major focus of the report was to look at impact of ICT in schools on two main areas learning and learners and
teachers and teaching. Further to this, it shows the barriers for effective ICT use as identified in the studies. The report
likewise tackles the research methods used to reveal impact in the two areas of learning and teaching.

Based on the body of evidence the report then raises a couple of key questions to be addressed in the near future, such as:
Is it sound policy to concentrate resources on ICT for those subjects and sectors where results are proven? or How can we
tackle the growing divide between e-mature and e-immature schools?In the final section recommendations are given to policy makers, schools and research and development. These recommendations
emerged from extensive discussions with the ICT cluster members and other ICT in education experts and point to possible solutions
and actions to be taken on Member State and European level. In an overview table, the report also lists the core review studies
according to their scope, research methods and the specific areas of impact in the two areas of learning and teaching, such
as students’ motivation and engagement, the personalization of learning or the emergence of new pedagogical practices. This
can help policy makers to identify studies most relevant to their interest.

Impact of ICT on learning and learners

ICT and pupil attainment

Six studies show statistical evidence that ICT can enhance attainment in subjects. UK’s largest impact study shows a raise
in subject performance through ICT use in English, science and design, and technology. Also specific ICT uses, such as interactive
whiteboards in the UK, had a positive effect on pupil’s performance in literacy, mathematics and science tests compared to
students in other schools. They especially improved the performance of low achieving pupils in English and impact was greatest
on writing. Another large impact study in the UK, which looked at ICT impact from an economic angle, confirms ICT investment
impacts positively on educational performance in primary schools, particularly in English and less so on science but not in
mathematics. On an international level, the analysis of the OECD PISA results indicates that longer use of computers by students
is related to better results in mathematics in PISA results. As regards better results in national test, two other UK studies
show that ICT can make a difference. Broadband access in classrooms is one necessary condition to benefit from new technologies
for learning. It results in significant improvements in pupil’s performance in national tests taken at age 16. Overall, evidence
from the studies reviewed shows that attainment improves as a result of embedding ICT into teaching and learning. Schools
with higher levels of ematurity demonstrate a more rapid increase in performance scores than those with lower levels.

Most opinion based studies investigating ICT impact on student performance, such as the elearning Nordic study, published
in 2006, give a positive picture with teachers being convinced that pupil’s subject related performance and basic skills (calculation,
reading and writing) as well as educational achievements improve.

Wider benefits for learners

An overwhelming majority of studies reviewed in this report confirm wider positive benefits of ICT on learning and learners,
such as motivation and skills, concentration, cognitive processing, independent learning, critical thinking and teamwork.
Increased motivation goes together with a positive learning attitude and leads for example to more attention during lessons
with students being more involved in the learning activities. The fact that ICT enhances a more student-centred learning approach
is often cited as among its most important benefits. ICT can benefit likewise academically strong and weak students as well
as students with special needs. Studies also reveal that the benefits can not only remain technology driven but should be
more intentionally exploited following a pedagogical approach. Case studies show, for example, that teamwork does not automatically
means increased collaboration. Many tasks which teachers called collaborative merely involved pupils working alongside one
another rather than jointly addressing a problem.

Impact of ICT on teaching and teachers

Despite the growing body of evidence on the impact of ICT use on learners, whether it will deliver its potential depends to
a large extent on how teachers use ICT within the teaching and learning process. As the evidence shows impacting on teachers’
practice have been proven to be a difficult endeavour. Whereas teachers estimate a high impact of ICT on learning and learning
outcomes, the perceived impact on teaching methodologies is seen much more moderate.

Most progress has been made in recent years in raising teachers’ positive attitude towards ICT by realising its value for
learning through more experience and embedded use. Teachers increasingly use ICT to prepare their work more efficiently and
achieve time gains. As the latest Eurobarometer benchmarking survey (published in September 2006) 90% of teachers in Europe
already use ICT to prepare their lesson.

There is also evidence of changes in roles of teachers either forced by the technology itself or more actively steered by
teachers. In changing the teacher–student relationship, as part of the new educational paradigm, the most difficult process
for teachers is to give up control and have more trust in students planning their work independently.

Literature stresses the importance that each use of ICT needs a pedagogical approach to improve learning. On the other hand
the overwhelming body of evidence shows that the majority of teachers have not yet embraced new pedagogical practices. Teachers
do not feel confident yet in exploiting ICT to support new approaches in teaching. Most of the teachers are still in stage
of using ICT to enhance existing pedagogical practice. Current pedagogy is subject centred, and uses ICT for differentiation
and project based teaching in more advanced cases. Collaboration between students is not yet sufficiently exploited.An important research finding is that ICT impacts most in e-mature schools and with e-confident teachers, suggesting that
once the foundations are laid the benefits will be considerable. The challenge is therefore to enable all teachers and schools
to reach e-maturity.

Research methods to reveal impact

The Impact report shows that the evidence available from impact studies or the evaluation of government initiatives is unevenly
spread across Europe. There is a predominance of UK research in the field, studies from the Nordic countries, The Netherlands
and Estonia, whereas little is known from some other European countries such as France, Spain, Germany, Greece and most Eastern
European Countries.

Currently, there are two main approaches to identify impact on a larger scale. In the UK approach focus is on proving the
causal relationship between ICT and better learning outcomes in national tests (measurable systemic indicators), whereas the
Nordic impact approach is on the perception of teachers and learners.Both approaches have their pros and cons, the first tries to establish a statistically relevant association between two variables
ICT and higher achievements to show the actual impact. On the other hand, inferring a causal relationship between ICT and
pupil achievement from simple correlations can be misleading. Many unobserved factors may influence better learning results
in national tests. Isolating the variable that caused the impact from other factors is problematic in education.

Studies that base their findings on perceptions of teachers’ students and parents have its limitations with regards to the
actual impact on teachers which can be in sharp contrast to what is perceived. However, they are an important part of the
evidence base in the field as they document the views teachers and students hold and express about the consequences using
ICT in schools.

With results gained from specific technical interventions limitations remain in the generalisation and transferability of
these findings in a wider context where the project specific conditions can not be met. Larger European comparative studies
such as Eurydice or the Eurobarometer Benchmarking survey provide important baseline data, but lack to give more qualitative
insights into how ICT is exploited for learning in the different European education systems.The review also shows that current education systems hinder ICT impact. Correspondingly impact studies and evaluations often
measure against traditional systems, where the potential of ICT can not be fully exploited. This poses a challenging question,
Are researchers looking at the wrong outcomes? Are policy makers clear or realistic about what they expect the results of
ICT investment to be?

The impact report therefore argues that a comprehensive body of evidence coming from various sources is needed to inform policy
makers as well as finding ways to research “unexpected” outcomes. A combination of different methods e.g. enhance quantitative
data with qualitative data increases the validity and reliability of the body of evidence.

Emerging questions based on the evidence

Based on the findings the impact report raises a number of key questions to be addressed. As regards to the positive findings
in certain subjects (English as a home language, science) or levels of education (in primary schools); would it be therefore
justified to concentrate resources for ICT on those subjects? Or should secondary schools be more remodeled like primary schools
to take account of the greater impact in primary schools?On the other hand to what extent are results transferable or are they contextually dependent? Can we deduce for example that
investment in ICT in French schools will yield similar gains in test scores in French primary schools?Moreover, the picture of evidence is only representative for the countries in focus. These are quite e-mature countries on
a wider European scale; there are still large differences between countries. UK and Denmark are for example countries where
almost all teachers use ICT as a teaching aid as opposed to countries such as Greece or Latvia where only 36% and 35% use
it. What about evidence in those countries? And how can we gather it?

Recommendations

Finally, the report outlines specific recommendations for actions at national and European level in the areas of policy, schools
and research and development. The recommendations emerged from the main results of the review and follow up discussion with
ICT cluster members and the European Commission to have a real impact on the education systems in Europe.

Policy

Future key aspects in the field of policy making are to support the transformation process and management of change, of which
ICT is and enabler and amplifier. New competencies, such as team work, independent learning or higher order thinking skills
are part of a new education context with ICT and should be formally taken up in national curricula and assessment schemes.
Concerning the continuous professional development of teachers, it will be important in the future to implement new forms
of training; training that place in workplace environment, is part of peer learning, is much more personalized and adapted
to the individual needs and wishes of teachers and in the long term helps to support lifelong learning of teachers. Initial
teacher training, not tackled in this review, is also seen as an important area for improvement in the future. The evidence
shows that ICT impacts most with e-mature schools. Policy makers should therefore work towards ensuring that the majority
of schools (80 per cent by 2010 for example) reach this point. The value of high quality digital content, support services
and highly motivated teachers in schools are essential in this respect.

Schools

As the latest evidence confirms teachers that assess to experience a more positive impact of ICT are most likely to be found
in schools where headmasters have used ICT to support the development of the schools values and goals. If the ICT strategy
is integrated into the overal’s strategy ICT has the greatest potential to act as a catalyst for change. Furthermore schools
should capitalize on positive attitudes and support teachers to underpin ICT use with a pedagogical approach. This can be
achieved by hand on practical training, providing easy to use ICT based materials, peer learning, securing a reliable infrastructure
and easy access to research findings.

Research and Development

Quite a number of research related recommendations were drawn with the goal to support policy making more effectively. In
the future, there is a need to evaluate more concretely school contexts, learning processes and teaching practices to show
under which circumstances ICT based activities can enhance learning. To create closer links between research and practice
it is recommended to combine more fundamental, small scale research or research focused on specific ICT tools with research
that is much more linked to practice. One important finding is the limited access to national research. National research
into ICT impact needs to be commissioned in some countries first and where already available made accessible, not only English
speaking. Therefore it is recommended to set up mechanisms at national and European level that allow us to know better the
results of such research and overcome language barriers in the field. This should include various types of research small
and large scale, qualitative and quantitative research. Policy makers also need to broaden the current approach to identify
impact, from measuring against well established policy goals towards more independent research to reveal the unexpected outcomes
of ICT use in schools.