DataShop Pipeline

This page provides information on datasets in various stages of progress. If you see an error in any of this information please feel free to correct it by editing this page. If you have a comment or concern on where a dataset sits in the DataShop pipeline, please contact the DataShop staff.

2/27: Started running data munger for wilkinsburg-algebra i. 3/5: Munger done, have 1,378,891 txs instead of 1,187,841. 5/4: Munger fixed, Jon Steinhart running the conversion, Tristan to QA. Supposed to have data week of 5/7 Load of Wilkinsburg student data delayed due to problems copying the student files. 6/25: Hampton numbers on production not right. CWCTC numbers on the-cooker not right. Wilks data doesn't load. Requested a new complete CD from CL. 7/9: Modifying analysis database to support improved pruning, waiting for new CD with latest version of munger. 7/23: Loaded onto production with all 3 schools, one student from Wilkinsburg failed to load.

8/07/07: Files loaded to QA. Some still need fixed (contain invalid XML) 8/16/07: All CTAT logs free of errors. All have been loaded to QA - waiting for researcher verification. 8/24/07: Need to get logs from Discussion Board from Erin Walker. Let's ping them again late Sept. 9/25/07: Files are on QA - waiting for go-ahead to move to prod from the researchers.

6/22/07: Tim to work on better skill model with Brett van de Sande & Anders. Waiting on result of this. 6/27/07: Anders regenerating raw logs 6/28/07: Files distilled ~100 have invalid xml. Anders to fix. 7/3/07: Reloading to the-cooker, some tutor semantic names are off. Also asked Anders to include units. 7/24/07: Received new data from Anders. Fixing bugs in distiller. 7/29/07: Data loaded to production - will be reloaded for changes to distiller.

Physics Course

Physics - USNA - Spring 2007

157

Andes

Physics

Kurt VanLehn

US Naval Academy

Spring 2006

Waiting to receive files from Anders.

8/14/07: files on Cooker, waiting for verification. 09/28/07: Loaded on the cooker under v2.3, waiting for verification 10/26/07: Will reload to production once new release is live. 11/5/07: Loaded to production

39

Public

Pre_Summer_School_01_Jun_05

10

CTAT-Flash

Chemistry

unknown

unknown

June 2005

Used to show DataShop features, example data only.

Robust Learning of Vocabulary

REAP ELI Reading 4 Summer 2006

118

REAP

English

Maxine Eskenazi

Pitt-ELI

Summer 2006

3/29/2007 - Sent message to Michael, Maxine to determine status

4/4/07 - Waiting for Michael's studies to finish, he will then convert to DS format. 6/12/07 - Waiting for Michael to add <conditions> to the datasets 6/25/07 - Loading datasets to the-cooker 6/26/07 - Ready to load to production 6/27/07 - Loaded to production

Robust Learning of Vocabulary

REAP ELI Reading 4 Spring 2006

117

REAP

English

Maxine Eskenazi

Pitt-ELI

Spring 2006

3/29/2007 - Sent message to Michael, Maxine to determine status

4/4/07 - Waiting for Michael's studies to finish, he will then convert to DS format. 6/12/07 - Waiting for Michael to add <conditions> to the datasets 6/25/07 - Loading datasets to the-cooker 6/26/07 - Reloading to the-cooker 6/27/07 - Ready to load to production 6/27/07 - Loaded to production

Paper written. Problem name labels include the condition and probably should not, makes error report difficult. Also missing problem descriptions.

44

Stoichometry Study

SummerSchool2005

11

Chemistry

Bruce McLaren

45

Stoichometry Study

Winter_Workshop01

59

Chemistry

Bruce McLaren

46

Stoichometry Study

PSLC Stoichiometry Study Demo

3

Chemistry

Bruce McLaren

The Effect of Generation and Interaction on Robust Learning

Self Explanation - Electric Fields - Spring 2006

104

Andes

Physics

Kurt VanLehn

US Naval Academy

Spring 2006

Waiting to receive files from Anders.

8/14/07: files on Cooker, waiting for verification. 09/28/07: Loaded on the cooker under v2.3, waiting for verification 10/26/07: Will reload to production once new release is live. 11/5/07: Loaded to production

READY TO GO LIVE

(datasets that are ready to be loaded to learnlab.web, highest priority on top)

Project

Dataset Name

Tool

LearnLab

P.I.

School(s)

Date

Notes/Status Remarks

Physics Course

Physics - USNA - Fall 2006

Andes

Physics

Kurt VanLehn

US Naval Academy

Fall 2006

Waiting to receive files from Anders. 8/14/07: files on Cooker, waiting for verification.

10/26/07: Will reload to production once new release is live.

IN TESTING

(datasets that have been loaded and are to be reviewed for correctness, highest priority on top)

Project

Dataset Name

Tool

LearnLab

P.I.

School(s)

Date

Notes/Status Remarks

Algebra I Course

Algebra I 2006-2007

CL Algebra - Mungable

Algebra

Albert Corbett

CWCTC, Hampton, Wilkinsburg, 3 LA schools

2006-2007 school year

Waiting for harvest from CL. Also need a new version of munger to load. Loaded to the-cooker, had multiple errors on load.

A Multimodal Interface for Solving Equations

Handwriting - Spring 2007

CL-2006 (Munger)

Algebra

Lisa Anthony

Spring 2007

07/06/07: Being QA'd by Tristan

09/24/07: Data loaded to the-cooker. 09/28/07: Updated on cooker to v2.3, waiting for new version of munger to load on production with v2.3 analysis_db 10/26/07: Bug fixes completed in Munger, waiting to receive new version. 11/5/07 New munger version had errors on run, awaiting word from tristan on causes.

Contiguity - CWCTC

Contiguity Spring 2007

CL-Lisp tutor (no CTAT)

Geometry

Vincent Aleven, Kirsten Butcher

CWCTC

07/06/07: Will receive from CL week of 7/10/07

07/09/07: Octav received data 07/18/07: Loaded on cooker 09/28/07: Loaded on the cooker under v2.3, waiting for verification.

09/25/07: DataShop has the dataset file. Waiting to upgrade the-cooker before loading for vetting. 09/28/07: Loaded on the cooker under v2.3, waiting for verification 10/9/07: Problem with the number of observations in Learning Curves. DS team looking into it. 11/12/07: Data reloaded to bunny.

Knowledge Tracking

Chinese Vocabulary Winter 2007

Phil's own

Other

Phil Pavlik

Spring 2006

7/06/07: Phil working on conversion. ETA 4-6 weeks.

9/26/07: Loaded to the-cooker 10/9/07: Problem with the number of observations in Learning Curves. DS team looking into it. 11/12/2007: Data reloaded to bunny.

Physics

Joint Explanation - Electric Fields - Pitt - Spring 2007

Andes

Physics

Bob Hausmann

US Naval Academy

Spring 2007

10/26/07: Bob to send files to Anders for conversion.

10/30/07: Anders sent files to DataShop team. 12/2/07: Loaded to the cooker, waiting for verify. renamed from previous "The Effects of Interaction on Robust Learning - Spring 2007"

IN PROGRESS

(datasets that require additional conversion work or tweaking, highest priority on top)

Project

Dataset Name

Tool

LearnLab

P.I.

School(s)

Date

Notes/Status Remarks

Geometry Course

Geometry 2005-2006

CL-2005 (lisp)

Geometry

Vincent Aleven

CWCTC, Hampton, Wilkinsburg HSs

2005-2006

Octav needs to convert this data into XML.

UPCOMING

(datasets we expect to receive soon - may or may not require additional processing on our part)

Project

Dataset Name

Tool

LearnLab

P.I.

School(s)

Date

Notes/Status Remarks

Algebra

Bridge to Algebra 2005/2006

CL-Munger

Algebra

6/6/07: Data is good to munge.

Request sent to CL to break the dataset into smaller pieces 07/09/2007: Confirmed that the munger can load sections only. Steve Ritter wishes to speak with ken/kurt to cover legal issues with this dataset.

7

Stoichiometry

VLAB data from Stoich studies

Chemistry

Bruce McLaren

Oliver Scheuer to oversee conversion in Germany.

6/20/07: Mike Karabinos to work up examples for each type of VLAB Action, DataShop to fill in dummy and real <tutor_message>. Will then send this info to Oliver. 7/27/07: Kyle completed transformation to XML. Sent out for review. 8/14/07: Working on storage of replay data - waiting for response from CTAT and Germany 9/25/07: Jonathan Sewall wants to see what logs look like in DataShop.10/8/07: Alida loaded sample Vlab dataset to QA (Dataset: Chemistry VLAB Example Data Fall 2007).

8

ESL

ESL data

7/27/07: Moving the Online Search System to PSLC servers in the near future. Will tie in to DataShop with single-sign on. No current plan to move any ESL data into DataShop itself.

Improving Algebra Learning and Collaboration

PTS Algebra I 2005

CL Algebra Tutor

Algebra

Erin Walker

?

2005-2006 (not sure)

Researcher has raw logs and has written a converter for analysis. Need to find out if it's in DataShop format.

Improving Algebra Learning and Collaboration

PTS Algebra I Spring 2007

CL Algebra Tutor

Algebra

Erin Walker

CWCTC

Spring 2007

Researcher has raw logs is responsible for submitting to DataShop for import.

7/06/07: Frank sent anonymized raw logs to Erin.

Geometry Course

Geometry 2006-2007

CL Geometry

Geometry

Vincent Aleven

CWCTC, Hampton, Wilkinsburg

2006-2007 school year

Waiting for harvest from CL. Will need a conversion from Octav.

Robust Learning of Vocabulary

Fall 2006 Personalization Study

REAP Tutor

English

Juffs

English Language Institute

Fall 2006

No ETA from Michael yet.

9/25/07: Sent follow-up email to Michael 10/03/07: Waiting for files from REAP Team.

Robust Learning of Vocabulary

Spring 2007 Reading 4

REAP Tutor

English

Juffs

English Language Institute

Spring 2007

No ETA from Michael yet.

9/25/07: Sent follow-up email to Michael 10/03/07: Waiting for files from REAP Team.

Robust Learning of Vocabulary

Summer 2007 Word Sense and Pronunciation Audio Study

REAP Tutor

English

Juffs

English Language Institute

Summer 2007

No ETA from Michael yet.

9/25/07: Sent follow-up email to Michael 10/03/07: Waiting for files from REAP Team.

Verb aspect

Verb aspect

CTAT

French

Amy Ogan

2007

8/24/07: Ping in late Sept to get these logs. 09/28/07: E-mail sent to amy asking for data status update.