TEL-ling tales – where is the evidence of impact?

Research is complex. It can be a messy business, but it matters. Higher education revolves around research and student degrees yet when it comes to the REF, pedagogical research in HE has a poor showing. A recent HEA funded investigation found critiques of submission quality* while back in 2002, Jenkins described it as having Cinderella status. A paper by the HEA researchers (Cotton, Miller and Kneale, 2017) suggests pedagogical research in HE remains the Cinderella of academia.

If pedagogical research in HE is struggling for recognition where does this leave the field of education technology or Technology Enhanced Learning (TEL)? The critiques are plentiful** so where is the evidence of impact?

I have great respect for the expertise of TEL colleagues so wearing my curiosity hat, I headed off to a closed learning technology mail list. Citing Surowiecki’s ‘wisdom of crowds’, I invited members to point me to evidence of enhancement via technology.

The response was not quite what I expected. Maybe I asked the wrong question. Maybe my view is different and maybe this is a Hull issue – in the nicest possible way! As Philip Larkin said, we ‘re on the edge of things rather than the centre and being on the edge can give you a different perspective. Whatever the reason, there were lots of ensuing discussions, some tweets and a couple of blogs – all showing a variety of reactions – Show me the Evidence by James Clay and In Defence of Technology by Kerry Pinny – but no links. There was also an #LTHEchat invitation to host ‘Establishing an evidence base for TEL’ which will take place on Twitter, 3rd May, 8.00-9.00 (diary date!) If the questions were wrong at least they generated some positive consequences.

I think Kerry was closest to my position when she described asking questions as scholarly practice. If we’re not research-informed and engaged how do we know if we’re having impact? Familiarity with the literature and taking time for critical reflection is about thinking academically and we work within academic environments where TEL is promoted as an enabler and enhancer of student-learning. Pedagogical research might not be scoring 10 out of 10 with the REF but it’s our daily bread and no reason to ignore what’s out there or not adopt a scholarly approach to evidencing our own practice – in particular with TEL matters. Institutions are investing huge amounts of money into digital platforms supporting learning and teaching but less into supporting staff to develop the digital capabilities and confidence to use them.

It’s now twenty years since the Dearing Report into the future of higher education which preceded the arrival of the VLE. Since those early days we’ve shifted from a read-only environment to user generated content, file sharing, mobile devices, social media, apps, virtual reality etc etc yet there’s still disparity of adoption and a widening divide between the innovators and those yet to climb aboard the TEL train.

What came out of the discussions (and what I see every working day) was how resistance to TEL remains high. Also it’s clear what’s missing includes the time, space, reward and recognition for staff engagement. We’re grappling with this at Hull and to make our case to SMTs requires evidence of impact on student learning and staff well-being. To find the evidence we need the research.

So where is it?

What do other TEL people use as their rationale for TEL matters?

footnotes

* critiques of pedagogical research in higher education include small sample sizes, localised research not capable of wider dissemination and limited contribution to theory. This is similar to the examples of critiques of TEL shown below.

** examples of TEL critique

‘Our analysis of articles published in two leading journals [these were the Australasian Journal of Educational Technology) and ALT-J (since renamed Research in Learning Technology)] found…poorly conceived or poorly applied methodologies, limited reference to theory, weak results, incomplete descriptions, uneven presentation of data and overblown and unsupported claims of impact and importance.’ (Gunn and Steel, 2012:11)

‘….where the potential of technology to transform teaching and learning practices does not appear to have achieved substantial uptake, this is because ‘the majority of studies focused on reproducing or reinforcing existing practices.’ (Kirkwood and Price 2012: 24)

‘The majority of papers published in BJET and the other educational technology journals are in the form of small-scale, unconnected trials and applications which can have little influence on policy making.’ (Latchem, 2014: 2)