Author: sjnamwp

AIM Analytics: U-M Community Lightning Talks

December 10 @ 12:00 pm – 1:30 pm

Join us on Monday, December 10 from 12:00 p.m. to 1:30 p.m. in North Quad (105 S State St) Space 2435 for AIM Analytics as we invite members of the U-M community to share interesting projects they are working on in relation to learning analytics.

AIM Analytics is a bi-weekly seminar series for researchers across U-M who are interested in learning analytics. The field of learning analytics is a multi and interdisciplinary field that brings together researchers from education, learning sciences, computational sciences and statistics, and all discipline-specific forms of educational inquiry.

Social Comparison in MOOCs: Perceived SES, Opinion, and Message Formality by Heeryung Choi

Abstract: There has been limited research on how perceptions of socioeco- nomic status (SES) and opinion difference could influence peer feedback in Massive Open Online Courses (MOOCs). Using social comparison theory [11], we investigated the influence of ability and opinion-related factors on peer feedback text in a data science MOOC. Perceived SES of peers and the formality of written re- sponses were used as the ability-related factor, while agreement between learners represented the opinion-related factor. We focused on understanding the behaviors of those learners who are most prevalent in MOOCs; those from high socioeconomic countries. Through two studies, we found a strong and repeated influence of agreement on affect and formality in feedback to peers. While a mediation effect of perceived SES was found, a significant effect of formality was not. This work contributes to an understanding of how social comparison theory can be operationalized in online peer writing environments.

Abstract: Evidence from past research has suggested that gender differences in collaborative learning often map onto stereotypical gender expectations. For instance, men use more aggressive language while women appear to be more agreeing and emotional. To explore gender differences in collaborative communication, we employed the methodology of Group Communication Analysis (GCA), which allows us to examine multiple sociocognitive aspects of learner interactions. Counter to some previous findings, we did not find significant differences between men and women in the degree of participation. However, our results suggest that women have significantly higher social impact, responsivity and internal cohesion in small group collaborative environment. Comparing the proportion of learner interaction profiles between men and women further strengthen the evidence that women are more likely to engage in effective discourse. Our findings provide implications for pedagogical practice to increases equity and inclusivity in online collaborative learning.

Abstract: Randomized experiments ensure robust causal inference that are critical to effective learning analytics research and practice. How- ever, traditional randomized experiments, like A/B tests, are limiting in large scale digital learning environments. While traditional ex- periments can accurately compare two treatment options, they are less able to inform how to adapt interventions to continually meet learners’ diverse needs. In this work, we introduce a trial design for developing adaptive interventions in scaled digital learning environments – the sequential randomized trial (SRT). With the goal of improving learner experience and developing interventions that benefit all learners at all times, SRTs inform how to sequence, time, and personalize interventions. In this paper, we provide an overview of SRTs, and we illustrate the advantages they hold com- pared to traditional experiments. We describe a novel SRT run in a large scale data science MOOC. The trial results contextualize how learner engagement can be addressed through inclusive culturally targeted reminder emails. We also provide practical advice for researchers who aim to run their own SRTs to develop adaptive interventions in scaled digital learning environments.

What Can We Learn About Learner Interaction When One Course is Hosted on Two MOOC Platforms? By Yuanru Tan

Abstract: Since the inception and adoption of MOOCs, pedagogues have criticized the quality of social learning within centralized platforms. Learning analytics researchers have investigated patterns of forum use and their relationship to learner performance. Yet, there are currently no cross-platform comparisons that explain how technical features of MOOC platforms may impact social interaction and the formation of learner networks. To address this issue, we analyzed MOOC discussion forum data from a single data science ethics course that ran concurrently on two different MOOC platforms (edX and Coursera). Using Social Network Analysis methods, the study compares networks of active forum posters using “Direct Reply” and “Star” tie definitions. Results show that the platforms afforded formation of different networks, with higher connectedness and higher network centralization seen on edX. The study presents preliminary results, discusses limitations inherent within the current analysis, and sets further directions of research investigating design features of centralized discussion platforms.

The Impact of Student Opt-Out on Educational Predictive Models by Warren Li

Abstract: Privacy concerns may lead people to opt-in or opt-out of having their educational data collected. These decisions may impact the performance of educational predictive models. To understand this, we conducted a survey to determine the propensity of students to withhold or grant access to their data for the purposes of training predictive models. We simulated the effects of opt-out on the accuracy of educational predictive models by dropping a random sample of data over a range of increments, and then contextualize our findings using the survey results. We find that grade predictive models are fairly robust and that kappa scores do not decrease unless there is significant opt-out, but when there is, the deteriorating performance disproportionately affects certain subsamples of the population.

Join us on Monday, December 4 from 12:00 to 1:30 p.m. in the Hatcher Gallery of the Hatcher Graduate Library (913 S. University Ave.) for AIM Analytics.

AIM Analytics was created to bridge the gaps in the support of UM learning analytics researchers with respect to the building of technical skills, sharing knowledge of educational datasets, and facilitating collaborative investigations.

For this event, we welcome members of the U-M community to share “late breaking work” within their departments.

Presentations will include:

Measuring the Pros and Cons of a Blended Course by Perry Samson, Arthur F Thurnau Professor, Professor of Climate and Space Sciences and Engineering, College of Engineering and Professor of Information, School of Information

Sentiment Analysis of Student Evaluations, and (separately) the Impact of Peer Feedback/Grades on TA Feedback/Grades by Heather Newman, Director of Marketing and Communications, School of Information

The U-M Learning Analytics Architecture (LARC) Dataset: What is it, How to Access it, and How it Enables LA Research by Steve Lonn, Director of Enrollment Research and Data Management

Predicting Short- and Long-Term Vocabulary Learning via Semantic Features of Partial Word Knowledge by SungJin Nam, Graduate Student Research Assistant, School of Information

Social Comparison Theory as Applied to MOOC Student Writing: Constructs for Opinion and Ability by Heeryung Choi, Graduate Student Research Assistant, School of Information

Scale MOOC Discourse Analysis with In Situ Coding by Phoebe Liang, Graduate Student Research Assistant, School of Information

AIM Analytics was created to bridge the gaps in the support of UM learning analytics researchers with respect to the building of technical skills, sharing knowledge of educational datasets, and facilitating collaborative investigations.

For this discussion, we welcome a panel of experts from the University of Michigan to share their knowledge and experience in understanding how to access and responsibly use educational data at U-M. Suitable for all faculty, postdocs, researchers and students who are looking to use educational data, this panel will provide insight into the “how,” “who” and “why” of educational data at U-M. The panel discussion will be followed by a Q&A session.

Learning analytics as an academic research space has been growing in influence for nearly a decade. Campuses globally are deploying learning analytics to address a range of challenges including student dropout, poor engagement and targeted marketing as well as predict teaching and resource needs. As a field, learning analytics has advanced rapidly both as a research domain and as a practical on-campus activity to increase organizational use of data. In this presentation, Dr. George Siemens will explore both the research and the practice of analytics in education, focusing on the development of the Society for Learning Analytics, models for research and organizational data use and growing sophistication of data collection through psychophysiological approaches.

Dr. George Siemens researches networks, analytics, wellbeing and openness in education. Dr. Siemens is Professor and Executive Director of the Learning Innovation and Networked Knowledge Research Lab at University of Texas, Arlington and cross-appointed with the Centre for Distance Education at Athabasca University. He has delivered keynote addresses in more than 35 countries on the influence of technology and media on education, organizations and society. His work has been profiled in provincial, national and international newspapers (including The New York Times), radio and television. He has served as Principal Investigator or Co-Principal Investigator on grants totaling more than $15 million, with funding from the National Science Foundation, Social Sciences and Humanities Research Council (Canada), Intel, Bill & Melinda Gates Foundation, Boeing, and the Soros Foundation. He has served as a collaborator on international grants in European Union, Australia, Senegal, Ghana, and United Kingdon. He has received numerous awards, including honorary doctorates from Universidad de San Martín de Porres and Fraser Valley University for his pioneering work in learning, technology and networks. He holds an honorary professorship with University of Edinburgh and adjunct status with University of South Australia.

Dr. Siemens is a founding President of the Society for Learning Analytics Research. He has advised government agencies in Australia, European Union, Canada and United States, as well as numerous international universities, on digital learning and utilizing learning analytics for assessing and evaluating productivity gains in the education sector and improving learner results. In 2008, he pioneered massive open online courses (MOOCs). He blogs at http://www.elearnspace.org/blog/ and on Twitter (@gsiemens).

Join us on November 7th at 12 p.m. in the Hatcher Gallery Lab for a panel and Q&A with Maya Kobersy (U-M Associate General Counsel), Sol Bermann (University Privacy Officer), Cindy Shindledecker (IRB Director) and Mike Daniel (Director of Policy for Academic Innovation) to understand how to access and responsibly use educational data at the University of Michigan. Suitable for all faculty, postdocs, researchers and students who are looking to use educational data, this panel will provide insight into the “how,” “who” and “why” of educational data at U-M and plenty of time will be left to ask questions of these experts. A light lunch is provided.

Questions covered include:

1) How does an exemption determination from the Internal Review Board (IRB) for research involving “normal educational practices” differ from a standard IRB approval?

2) What does Family Educational Rights and Privacy Act (FERPA) mean to the researcher, and how does the research ensure their work complies with U-M FERPA requirements?

3) Who are the data stewards, and how do you find the right person to ask for educational data?

4) What are best practices for de-identifying data? What is the difference between de-identifying data and anonymizing data?

5) What privacy and ethical considerations and best practices should I be thinking about?

6) What data security practices do I need to follow and/or should I consider?

This talk will present results from recent work that uses language to assess social dynamics during collaborative interactions. I will introduce group communication analysis (GCA), a novel approach for detecting emergent learner roles from the participants’ contributions and patterns of interaction. This method makes use of automated computational linguistic analysis of the sequential interactions of participants in online group communication to create distinct interaction profiles. We have applied the GCA to several collaborative learning datasets. Cluster analysis, predictive, and hierarchical linear mixed-effects modeling were used to assess the validity of the GCA approach, and practical influence of learner roles on student and overall group performance. The results indicate that learners’ patterns in linguistic coordination and cohesion are representative of the roles that individuals play in collaborative discussions. More broadly, GCA provides a framework for researchers to explore the micro intra- and inter-personal patterns associated with the participants’ roles and the sociocognitive processes related to successful collaboration.

Bio: Nia Dowell is a cognitive psychology doctoral candidate at the Institute for Intelligent Systems in the University of Memphis. Nia is currently pursuing her PhD under the mentorship of Professor Arthur Graesser. Her primary interests are in cognitive psychology, discourse processing and learning sciences. In general, her research focuses on using language and discourse to uncover the dynamics of socially significant, cognitive, and affective processes. She is currently applying computational techniques to model discourse and social dynamics in a variety of learning environments including teacher education programs, intelligent tutoring systems (ITSs), small group computer-mediated collaborative learning environments, and massive open online courses (MOOCs). Her research has also extended beyond the educational and learning sciences spaces and highlighted the practical applications of computational discourse science in the clinical, political and social sciences areas.