Learning analytics – Final interim results ;-)

June 13, 2012

‘We are leaving the comfort zone’

Jochen Robes couldn’t be more right with his statement given in the online session. No other topic so far is so controversy discussed as learning analytics (LA). About 100 participants attended the online session this time. The somehow lower numbers can be explained with the earlier time (2 pm). Martin Ebner from the university in Graz was this time speaker. So far 19 blogs posts were posted in the course blog regarding LA.

The main purpose of LA is to measure success in learning, development or adjustments of (new) learning concepts (Luka), coaching, and quality management (F. Hausmann), to monitor the learning process and success, for comparison, to provide feedback to students (Jasmin Hamadeh) and/or to monitor the own teaching (Göldi). LA can be used not only to judge learner, but also educator or the institution (from Simon Buckingham Shum quoted in J. Robes post).

Some wondered whether LA is really new, because everybody somehow collects data from e.g. tests and compares them with another or a previous class (Göldi) – or just a recycled issue – Riesenbeck calls that Retro. Karlaspendrin thinks that LA is just a better sounding name for evaluation and quality development and that LA is more an illusion than reality. The only thing that is new is that with the introduction of LMS more ‘Big Data from heterogeneous sources is now available. However, Riesenbeck thinks that not the amount of data is determining and suggests that e.g. a weekly questionnaire would provide better qualitative data, instead of collecting large amounts of quantitative data like login times or what material is viewed for how long. This data is less meaningful. Jasmin Hamadeh e.g. argues that students may print out learning material, thus login times can be misleading. It is necessary to clarify beforehand how realistic LA is, financing, support and training for educators and how to convince critical voices, be it students, parents or educators (F. Hausmann).

J. Robes provided in his post a SWOT (Strength, Weaknesses, Opportunities and Threats) – Analysis in form of a mind map about the topic which is interesting from Mohamed Amine Chatti. Quite a lot of the issues identified in the mind map were named by the OPCO12 participants, yet going through all the blog posted about LA, it is evident that more threats and weaknesses of LA are identified contrary to strength and opportunities.

Luka states that LA can help to draw a more detailed picture from each student which helps to support the learner more effectively as before. It can be used as a quality management tool to assess performance, as well as for self-assessment, in case those involved are willing to reflect on their own work (F. Hausmann). LA can be used also for continuous learning process improvement, as guidance for learning, early invention monitoring, reflection and awareness tool. LA can support personalized learning.

The Bildungscafe Blog examined PLE (personal learning environment) in relation to LA. PLE is becoming increasingly important and many learners make use of informal learning tools, like Skype, YouTube, etc. next to their formal learning environment, like the LMS. The Bildungscafe questions whether and how this data from informal data sources should be used and if students are willing to disclose these sources, but on the other side how will the data from formal sources like the LMS provide a complete picture from the student performance if he uses other learning sources/tools as well.

Data privacy issues are probably the most discussed topic. The transparent student (Sonja Gerber), Digital Big Brother à la Orwell (Hausmann) are phrases used to describe concerns. Which data will be collected, how will it be used, will that be communicated to all involved are recurring questions. Göldi explains that students from students in Switzerland have the right to information of all collected data. Göldi suggests communicating beforehand what data will be collected to increase acceptance.

Luka thinks that the close-to-real time adjustments are utopia and that LA is very time intensive for educators and might result in a ‘permanent burnout situation’.

Technical concerns were also raised. How easy are the tools to handle and how effective are they? Is the analysis and interpretation possible only by a skilled user? Can they really substitute an educator (Sonja Gerber)? Automatized analysis is according to Karlaspendrin ethically not acceptable, it is not a self-determined process. Learning4me scrutinize which criteria are applied for the analysis. Educators might fear to be assessed and are therefore reluctant to apply LA.

Conclusions

As the list of pro and contra LA demonstrates there are chances and risks attached to LA. Learning4me suggested two possible solutions either to demonize or to face the challenge, viewing LA as chance not as threat. I like the comparison with Snake in grandmothers clothing coming out of the wardrobe. Educators need to be willing to reflect on their own work (F. Hausmann) and be willing to improve their teaching, which Göldi sees as the first priority for LA.

Overall there is a guarded optimism recognizable. With appropriate data protection in place, as well as a supportive framework (support for educators, e.g. training how to use the tools and how to analyze and interpret data (Luka) LA could thoughtfully applied improve learning and teaching, hence be equally useful for educators and learners. Yet, that requires brushing away old dust from the shelves (Luka) and to leave the comfort zone (J. Robes)

Thanks a lot 🙂
Enjoy watching the EURO 2012, you deserve leaning back since you wrote already (and will write) so many fantastic reports, keeping us greatly up-to-date with all #OPCO12 topics. Thanks for that, you are a great moderator 🙂