4
Educational Data Sciences and the Need For Interpretive Skills An Interdisciplinary Perspective

5
Some Driving Questions What counts as good learning analytics? What kind of profession will data sciences be? What are its ancestor, sister, and adjoining disciplines? Which kinds of skills and dispositions are important for preparing future practitioners and scholars?

6
Our Sociotechnical Thesis Data exist inside a social context; shaped by and shaping that context.

7
Our Sociotechnical Thesis Data exist inside a social context; shaped by and shaping that context. Interpretation is not technical. It is itself socially situated with goals, predispositions/ biases, and norms.

8
Our Sociotechnical Thesis Data exist inside a social context; shaped by and shaping that context. Interpretation is not technical. It is itself socially situated with goals, predispositions/ biases, and norms. Professional communities have developed valuable ways to reason from imperfect evidence. We can leverage/translate them to this new sociotechnical terrain.

21
Educational Data Sciences 1.A new field with growing interest from leading universities, foundations, USED 2.Journals, conferences, and programs now emerging 3.What is the disciplinary focus, what counts as rigor and success? Education Data Sciences

22
Statistical Data Analysis Education Data Sciences Much of the digital ocean is compatible with statistical analysis. Exploratory data analysis (ex: Tukey with satellite data in 70s asked many questions that are being asked today about “big data” Already established (entrenched) in educational power structures Can produce strong claims

23
Learning Technology Statistical Data Analysis Education Data Sciences Classroom/ Learning Technology This is where the data we want most often come from… This area is seeing an explosion in media/learning resources and classroom management tools

33
Approaching Digital Age Data Analysis What counts as rigor and success? Which parts of what disciplines are needed? What methods are best? What kinds of processes will make good, great, and poor educational data analysts? How much of the requirements are technical versus attitudinal ?

34
Core Principles 1.All analytic processes are socially situated and iterative 2.Data is a mediational tool in an iterative process of discovery 3.Data is an imperfect lens for context and for interactions within that context 4.Organizational/systems thinking helps expand the reach of educational data science 5.Ethical as well as legal considerations are important.

36
Some Insights from ECD Arguments and Layers Domain Analysis: What is important in the domain? Domain Modeling: The Structure of Assessment Arguments Student, Evidence, and Task Models

37
Some Insights from CHAT/LASF Engeström Cultural Historical Activity Theory (CHAT) Piety and Behrens Linked Activity Systems Framework Data analysis generally involves more than one activity system. The technology plays a linking/mediating role across contexts as well as within Framework allows for conceptualizing privacy/authorized access space

38
Some Insights from EDA Underlying Heuristics (4Rs) Revelation (graphics) Residuals (models) Resistance /robustness Re-expression: scale Broader Meaning What do we “see” in the data? What in our data fits/does not fit with our emergent model? Do we have a summary that is not easily fooled by unusual distributions/examples How do the explanations we see apply more broadly.

39
What Kinds of Skills/Aptitudes? Broad fluency with a range of qualitative/quantitative methods Ethics, privacy, and confidentiality (FERPA+) Technology accumen

41
Though a famous and successful statistician, Tukey wanted to create a field that dealt with all data, even when it came in such poor shape that it was not amenable to statistical analysis. He called it “data analysis” and created the field called “Exploratory Data Analysis”. My undergraduate degree was in Psychology and Philosophy. I thought if I knew the logic of how we know things (epistemology) and understood the human lens through which all perception and thought occurs (psychology) I would have the fundamental layers of knowing from which to acquire more knowledge. After serving as a social worker and studying special education, I sought my Ph.D in Educational Psychology with a cognate called “Measurement, Statistics & Methodological Studies”. I would approach it as applied epistemology: How do we learn from data? When I discovered Tukey’s writings I knew I had found the right place. I conducted psychological studies on perception of statistical graphics and wrote about the logical foundations of data analysis. When I wrote such a chapter called “Data and Data Analysis” [6] people told me it was a silly title – data wasn’t a subject, it’s only a piece of the background to other sciences. Philosophy is concerned with understanding meaning and the application of logic. The philosopher asks What do we mean by ‘data’? What do we mean by ‘analysis’? If data are symbols that point to elements in the world, what kind of logic do we need to understand that linkage? Like very good scientists, philosophers question the obvious. Such questioning may not be essential for what you do today, but it may open the door to do new ways of thinking you never imagined. The successful learning analyst will avoid two common errors: Failure to understand the context and failure to become intimately familiar with the data. 1.The first error is caused by lack of contextual knowledge. Studying the learning sciences, education, and related disciplines will help. 2.The second is error is caused by a substitution of complex statistical or computational models for detailed mental models. We only build computational models or display to help our mental models. Question the assumptions of your work deeply. It is important that analysts understand their work is about “revelation” or “unveiling” the reality of the world. It is a special (at times prophetic) role in society and should be taken very seriously. Do not think of data science as a set of techniques but as a collection of viewpoints (epistemic positions) and habits of mind. To undertake good visualization we need to know the techniques of data display, but also the psychology of perception, the anthropology of semiotics, the mathematics of fluctuation and the philosophy and art of aesthetic engagement. We will always need good technical analysts, but we need them to be

43
Big Question What is the new role for the teacher? Privacy Getting designation as school vendors Funder issues (12-24 months 2,3,5 yrs) Speak Gate-ish Issue of interoperability… Continuous Improvement from ECD

44
Activity theory tensions Activity systems in coherence/contrast Innovations that help resolve Activity System tensions (Engestrom, end of life care). More data about the context