Transcription

1 ANALYZING THE BENEFITS OF USING TABLET PC-BASED FLASH CARDS APPLICATION IN A COLLABORATIVE LEARNING ENVIRONMENT A Preliminary Study YoungJoo Jeong Human Computer Interaction Institute, Carnegie Mellon University Computer Science Department, Carnegie Mellon University Ananda Gunawardena Computer Science Department, Carnegie Mellon University Ken Koedinger Human Computer Interaction Institute, Carnegie Mellon University 1. ABSTRACT This paper examines the potential use of the Tablet PC-based Flash Cards Application ( application ) in a collaborative learning environment. The application provides the flexibility of handwritten input and the ability to share flash card decks. The application was deployed in an eighth grade geometry classroom at a girls' school in Pittsburgh. Nine students participated in the study and each student was asked to individually create decks of flash cards reviewing the material that they had already learned ( review case ) as well as create separate decks previewing upcoming material ( preview case ). We randomly selected groups of students of different sizes, pooled their flash card decks, and measured the percentages of topics represented on the cards. The teacher's decks were used as the control. We found, in the preview case, that the decks of fewer students covered a higher percentage of material than in the review case. We also surveyed eight students to determine their criteria for selecting material when they created the cards. Half of the students created flash cards for material with which they were confident that would answer correctly, half created cards for material that they thought they should review more, and 3 out of 8 did both. The findings of this study gave us a better insight into the use of Tablet Flash Cards in a collaborative K-12 educational setting and also for higher education. 2. PROBLEM STATEMENT AND CONTEXT 2.1. Development of the Application In recent years, several web based flash card programs and different intelligent testing algorithms were studied in the context of language training [2]. Microsoft also created a basic ink flash card program in its experience pack. However, it does not appear that these programs are developed using sound pedagogical principles to maximize the utility. So, we developed a new flash card application at Carnegie Mellon University (CMU) to increase the usability, provide more efficient and enjoyable ways to increase memory retention rate, and create more opportunities for ink flash card research. We iteratively created the application by following user-centric design methodologies and conducting usability studies in Human Computer Interaction (HCI) in order to make this application intuitive and useful in an educational context.

2 These methods include contextual inquiry, heuristic evaluation, and think-aloud testing on the prototypes and initial versions of the application. We also observed that students usually have two different types of sessions when studying with flash cards: card creation session and reviewing session. To suit their needs in both sessions, we created two modes in this application: the authoring mode [Figure 1] and the game mode [Figure 2]. In the authoring mode, they can easily draw on the cards which are especially useful in math educational settings. They can also easily draw figures or insert images on the cards. In the game mode, an algorithm reorders the card decks according to the individual student s performance on each card by keeping track of the scores in order to present the next best card for review (this feature is called smart testing ). The current version of the application also enables the users to review the cards sequentially as well. These features were added to the original application after studying the use cases in a classroom environment. During pilot studies, we found that the learning materials are often built upon previously learned materials and hence sequential presentation of material was important for new material or for the material that a student has not yet mastered. Application also allows images to be imported to individual cards and to entire card deck. This feature is especially useful in adding diagrams and grids for drawing graphs. These card decks can be saved as electronic files for exchange or further enhancement Application in Context of a Larger Study In an effort to extend the application as an online collaborative educational tool, we wanted to first explore the benefits of the use of this application in a collaborative environment. More specifically, we wanted to first verify whether sharing decks with others would be more beneficial than using them alone. While observing the students use of this application, it was apparent that each student s perception of her study material played a major role in forming the questions and the answers on the cards. Therefore, each student emphasized on different points as they selected the portion of the material. Hence we wanted to understand students criteria in selecting the material, the amount of the material each student covers individually, and the optimal number of group size for effective collaboration to increase the full coverage of the material Relevant Research Our study was first motivated by Bonnie John s study of The evaluator effect in usability studies: problem detection and severity judgments [3]. In this study, she focused on finding usability problems in software applications. She measured how the usability problem detection rate increased as the number of evaluators increased. Following a similar idea, we framed our study to measure the number of students needed to cover all or most of the study material in a Tablet Flash Card setting Research Question Our research question in this study was to understand how the amount of the coverage of the study material varies as the number of students increase. Since the students create cards individually, it is apparent that the criteria that each student uses to select the topics and to form questions about chosen topics can vary widely. We asked individual students to create card decks for the materials that they had already learned in the class and for the upcoming material that they had not yet learned. Then we analyzed the student decks compared to controlled deck files from their teacher. We also conducted online surveys with the participants to see how their

3 selection criteria varied depending on whether they have already studied the materials or not. We also attempted to understand how sharing the decks would help in both cases as well as finding out the optimal number of students needed to cover the most amount of the material Deployment and Participants The participants of this study were students from the Ellis School, an urban private girl s school in Pittsburgh. Since 2004, Carnegie Mellon University has been helping eighth grade geometry students in Ellis school to use Tablet PC application programs for note taking, reading electronic books and classroom student-teacher interactions. The students and the teachers use electronic ink to write mathematical equations, draw diagrams, and highlight materials in the electronic books. The school has a one to one laptop program and especially the students in eighth grade Geometry class uses Tablet PC s for all school related activities. For this particular study, all of the participants were from the eighth grade Geometry class. There were total of nine students in this class and these students were among some of the best students in eighth grade class. 3. SOLUTION EMPLOYED As classroom activities and as homework, the students in eighth grade Geometry class have been using the application to create flash cards since In spring 2009, each student was asked to individually create a deck of flash cards covering the material that they had learned in the previous two weeks. Later, they created another deck previewing upcoming material. The decks were used to review and preview material for their geometry course. No specifics were given to the students as to which topics within the assigned materials shall be included in the card decks. We wanted to observe how each student decides which topics to include in the decks. The students freely created the questions and answers from the material. An example of a student created card is shown in Figure 1. To make it easy for the research team to access the decks, the students uploaded their deck files on a Google group a useful online collaboration tool. Figure 1: Card Authoring Mode Figure 2: Card Game Mode The teacher created control cards for each of the topics within the assigned materials. The topics range from finding angle measures to finding arc length of a circle. Among the nine students, only five decks per each material were analyzable due to problems from the participant: some did not create the card decks, and some faced technical difficulties in uploading or ing the deck files. A total of twelve decks were collected and analyzed including two decks from the teacher. On average, each deck had about seven cards for the review case and five cards for the preview case. For both preview and review cases, we randomly chose groupings of 1, 2,

4 3, 4, and 5 decks in a group to measure whether more students collectively cover a wider range of materials, and if so what would be the optimal number of students needed to cover most of the material. Figures 3 and 4 display the relationship between the group sizes (x-axis) versus percentage of topic coverage (y-axis). The averages were taken in order to measure the percentages of the material covered for different grouping sizes. A follow-up survey was used to determine the inclusion and exclusion criteria that they used when creating the decks as well as to find out how the participants felt about collaboration in learning. The survey questions and topic list references are available in additional resource section. 4. EVALUATION 4.1. Results on Coverage of Material There were ten topics for the learned material and only three topics for the new material. A topic is defined as an important conceptual fact that student should master. The list of topics and their difficulty levels were generated by analyzing the teacher s decks that were created based on teachers perception of what students should know. Then each of the students cards was analyzed to see which of the topics the card covered. The results from the deck analysis are given below showing the group size (x-axis) versus percentage of topics covered (y-axis) by the group. Figure 3: Review Material Case Figure 4: Preview Material Case For the review material case, even when the decks from all five analyzable decks were combined, the combined deck only covered 80% of the materials from the control deck. For the preview material case, 100% of the material was covered by groups of four students or more. Thus, for the review case, we would need more than five students to cover most of the materials, whereas for the preview case four students are sufficient to cover all of the material Survey Results on Selection Criteria Out of eight respondents, half preferred to study alone while the other half preferred to study in a small group of size two or three. All of the respondents felt comfortable sharing their deck files with their peers. Seven students used the application for science classes, three students used it for language courses, three students used it for social science classes, one student used it for music and art related courses, and one used it for other math courses. It was interesting to observe that three students noted that they like collaborative learning, because they feel safer or more comfortable that everyone s would be on the same page with them. Furthermore, five students thought that in a collaborative learning environment they feel that others can correct them in case they were wrong. Half of the students also liked the fact that they get to learn more about different ideas and opinions from others. Some negative perceptions about collaborations were also noted by students. Seven students indicated that the biggest problem in collaboration in learning was that there are free riders who do not do their part in a group. The next biggest

5 complaints (five students each) were that when collaborating with others, the organization of the work is inefficient and that there are conflicts of opinions and ideas. Furthermore, it was interesting that for both the review of the learned material and the preview of the new material, half of the students created some of the cards for the topics that they did not know well enough so that they can review it more in the game mode; and half of the students created some of the cards for the materials they knew they would answer them correctly. Two students indicated that they created the cards for the materials they thought the teacher will ask them on the exams later, and two students responded that they created the cards without much thought Limitations of this Study Creating flash cards were not part of the student grade and hence the incentive to participate was low. If grades were to be an influencing factor, that may have changed students behavior in creating the cards and interacting with the application. A larger study is needed to be able to more accurately understand the different behaviors in creating the cards for the learned material and for the new material. The experiment must be repeated for a large group size in order to verify the optimal group size to measure the benefits in collaborative learning. Since this study was conducted at a girls school, it will be interesting to repeat a similar study in boys schools and coeducational schools to learn if gender plays a role in collaborative educational settings Impact on the Students and the Teacher The students were eager to use new technologies in their classroom. They were excited to use an application that clearly demonstrates the benefits of Tablet PC s. They started using the application in other classes and at home on their own. Many indicated that the application is useful for language, science, music, and art courses as well. The teacher also appreciated that this application was an effective way to encourage students not only to study on their own but also to share their knowledge with each other. Students were excited to use tools such as Google groups for collaboration. 5. FUTURE WORK Our study focused on finding the ideal deck size in authoring mode. A study to measure the optimal number of repetitions of the cards in the game mode would also be a good follow up study. We can also study how we should choose the cards to merge with other decks. As shown from this study, sharing all the cards from everyone may not increase topic coverage drastically after four decks are merged. An interesting experiment would be to select cards based on a few criteria such as the students grades and students topic coverage in previous cards to see if we can have less number of users deck to create better topic coverage. As vast majority of the students were concerned about the free riders problem that in traditional group settings some members of the group do not contribute. Hence it would be interesting to test different methods of collaboration in order to minimize the free riders problem. We were able to avoid this problem in the current study by encouraging students to share the cards only after they have created the cards individually first, instead of encouraging the collaboration in the creation of the cards. We can effectively share the cards by merging the decks of three to four students randomly and perhaps be able to cover most of the topics. This way, we can also satisfy both types of people: people who prefer to study alone and people who

6 prefer to study in small groups. Further research can be done to find the optimal number of cards, not just the number of students to achieve the best topic coverage. By making the participation of this study voluntary and performance not counted towards their grades, we were able to introduce the application to the students as a fun way to help them study more effectively. The game mode also seems to have largely contributed in making their interaction with the application and learning experience more enjoyable. To this end, it would be interesting to study how grades and teachers recommendation (or their enforcement) affect students interaction with educational software and the likelihood of using it voluntarily outside of classroom environment. Though it may take years of research, measuring the memory retention rate or the learning outcome of the material studied using the application would also be interesting. Especially because using electronic pen input as the new input modality decreases the cognitive overload in entering math equations and figures using keyboards or mouse [1], it may help students to focus more on study material itself rather than entering math on a computer. In addition, we should also investigate whether students learn better or remember for a longer period of time for the material in which that they had created the cards compared to the material that their peers have created and they merely used in the game mode. Even if the students remember better when they actually create the cards than when they just study their peers cards, it may still benefit them to study the cards that their peers created since their peers may include some material that they had missed. Further studies with larger sample sizes must be conducted to verify these hypotheses. 6. ADDITIONAL RESOURCES The survey questions, topics, images, videos and software downloads are available from 7. ACKNOWLEDGEMENTS The authors of this paper would like to acknowledge the unconditional support provided by Mr. Russ Schopper, the math teacher and students in the Geometry class at Ellis School and their parents who consented to this study. We also want to thank CMU IRB office for providing prompt approval of the study of human subjects. 8. REFERENCES [1] Anthony, Lisa. Developing Handwriting based intelligent tutors to enhance Mathematics learning, Ph.D. Thesis, Human Computer Interaction Institute, Carnegie Mellon University, CMU-HCII , 2008 [2] Frishkoff, G., Levin, L., Pavlik, P., Idemaru, K., & de Jong, N. A Model-based Approach to Second-Language Learning of Grammatical Constructions. In V. Sloutsky, B. Love & K. McRae (Eds.), Proceedings of the 30th Conference of the Cognitive Science Society (pp ). Washington, D.C., 2008 [3] Jacobsen, N.E., Hertzum, M., and John, B.E. The evaluator effect in usability studies: problem detection and severity judgments. Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting: Santa Monica, CA: Human Factors and Ergonomics Society, 1998

Research Basis for Catchup Math Robert S. Ryan, Ph. D. Associate Professor of Cognitive Psychology Kutztown University Preface Kutztown University is a 4 year undergraduate university that is one of 14

This document contains Chapter 2:, Data Analysis, and strand from the 28 California High School Exit Examination (CAHSEE): Mathematics Study Guide published by the California Department of Education. The

Using A Learning Management System to Facilitate Program Accreditation Harry N. Keeling, Ph.D. Howard University Department of Systems and Computer Science College of Engineering, Architecture and Computer

Abstract Student Preferences for Learning College Algebra in a Web Enhanced Environment Laura Pyzdrowski West Virginia University Anthony Pyzdrowski California University of Pennsylvania It is important

Game Information 1 Introduction Math is an activity that is perfect for reviewing key mathematics vocabulary in a unit of study. It can also be used to review any type of mathematics problem. Math provides

Game Based Learning for Spanish 101 Action Research By Naomi Wahls Page 1 Table of Contents Introduction and Problem Statement Purpose and Intended Audience Research Questions Context of the Study Procedures

Education at the Crossroads: Online Teaching and Students' Perspectives on Distance Learning Jacqueline Leonard and Smita Guha Temple University Abstract The Internet offers colleges and universities new

Engaging Middle School Students in the Analysis and Interpretation of Real-World Data by Cheryl A. McLaughlin Copyright 2013, National Science Teachers Association (NSTA). Reprinted with permission from

Research and Digital Game- based Play: A Review of Martha Madison White Paper Compiled by Anne Snyder, Ph.D. Second Avenue Learning is a certified women- owned business, specializing in the creation of

Student Perceptions On Computer Coding Ashfield Public School Term 3, 2015 The Initiative 21 st Century education is about innovation. It is about a being open to new ideas and new educational trends that

School district promotes learning innovation with advanced technology for every student www.kent.k12.wa.us 4,500 employees United States Public education When decided to expand its One Student One Device

Strategies for Winning at Math Student Success Workshop Just the Facts Poor performance in math is NOT due to a lack of intelligence. The key to success in math is having the right approach to studying

TEFP1011 Susanne Hashmi-Lewis - Kings College Guildford Improving the Progression, attainment and enjoyment of KS3 students in Mathematics using single sex classes in a co-educational, comprehensive school

Bridging the Gap for First Year Students with the Use of Peer Tutoring* Georgina Carmody University of Technology, Sydney Georgina.Carmody-2@uts.edu.au Leigh Wood University of Technology, Sydney Leigh.Wood@uts.edu.au

The Development of an Online Engineering Alphabet Ismail Fidan 1, Marketta Laurila 2, Robert J. Clougherty, Jr. 3 Session S3D Abstract - Engineering is an unknown profession to many children, as current

multiply Your students interest will with Microsoft Mathematics 4.0. While teachers are under pressure to raise math and science test scores on standardized tests, many students find math their most frustrating

STo appear in Proceedings of Workshop on the Impact of Pen-Based Technology on Education (WIPTE) 2008. Assessing the Impact of a Tablet-PC-based Classroom Interaction System Kimberle Koile David Singer

A STATISTICS COURSE FOR ELEMENTARY AND MIDDLE SCHOOL TEACHERS Gary Kader and Mike Perry Appalachian State University USA This paper will describe a content-pedagogy course designed to prepare elementary

Using ipads in Undergraduate Mathematics Eric J. Thornburg This paper was completed and submitted in partial fulfillment of the Master Teacher Program, a 2-year faculty professional development program

Introduction The SAT provides a valid, nationally consistent measure of what students have learned and how well they apply that knowledge.* The SAT is the nation s most widely used college entrance examination.

On-Line Courses: A Comparison of Two Vastly Different Experiences by Sharon Testone, Ph.D. Introduction The busy lives that people lead today have caused a demand for a more convenient method to gain a

Research on Graphic Organizers Graphic Organizers are visual representations of a text or a topic. Organizers provide templates or frames for students or teachers to identify pertinent facts, to organize

What is independent learning and what are the benefits for students? Author(s): Bill Meyer, Naomi Haywood, Darshan Sachdev and Sally Faraday Publisher: London: Department for Children, Schools and Families

A Sustained Professional Development Partnership in an Urban Middle School Cathy Liebars, The College of New Jersey liebars@tcnj.edu Abstract This paper describes a sustained professional development project,

GOALS: The goal for this session is: Learn how to connect and use the SMART Board for effective instruction Ability to annotate and save work using Microsoft Office and SMART Notebook Software OBJECTIVES:

TM parent ROADMAP MATHEMATICS SUPPORTING YOUR CHILD IN GRADE TWO 2 America s schools are working to provide higher quality instruction than ever before. The way we taught students in the past simply does

What is the impact of multisource learning on History at Key Stage 3? Researchers: Lloyd Brown and Rolf Purvis Date of research: September 2000 August 2001 Contact details: Chesterton Community College,

Running Head: EFFECTIVE USE OF TECHNOLOGY IN EDUCATION 1 Effective Use of Technology in Education Inez Escandón The University of Texas at San Antonio EFFECTIVE USE OF TECHNOLOGY IN EDUCATION 2 Effective

Introduce diagnostic teaching Alan Bell and the Toolkit team Summary Many students who appear to understand a topic at the end of the teaching unit, do not retain that even for a few months. Better long-term

ing Achievement Why are so many parents choosing to home school? Because it works. A 7 study by Dr. Brian Ray of the National Home Education Research Institute (NHERI) found that home educated students

Methodology Teaching in a Digital Age Phase 1 Participants Forty-four educators from seven schools participated in the study, representing a diverse range of technology initiatives, years of implementation,

Students develop aptitude for taking advanced math courses through the Project K-Nect Mobile Learning Initiative in Onslow County School System Prepared by Project Tomorrow for Digital Millennial Consulting

Impact / Performance Matrix A Strategic Planning Tool Larry J. Seibert, Ph.D. When Board members and staff convene for strategic planning sessions, there are a number of questions that typically need to

Students Perceptions of Learning College Algebra Online using Adaptive Learning Technology Lori Ogden West Virginia University Adaptive learning technology was used in the teaching of an online college

Delivering Intervention in Mathematics and English (DIME) Jane Moreton 8/9/2008 Introduction What were your reasons for doing this type of development work? IDS data identified several schools across the

William Floyd Union Free School District A Parent s Math Resource Guide to 2nd Grade Mathematics Dear Parents, The Parent s Math Resource Guide is a resource to assist you in becoming familiar with Common

American Educational Research Journal Spring 1979, Vol. 16, No. 2, pp. 181-187 The Influence of Categories of Cumulative Folder Information on Teacher Referrals of Low-achieving Children for Special Educational

LEA Information A. LEA Information 1. What is the total student enrollment based on the most recent BEDS Day submission? 2. 249 What is the student enrollment by grade band based on the latest BEDS Day

Objective To review the partial-quotients division algorithm with whole numbers. 1 Teaching the Lesson materials Key Activities Students review and practice the use of a friendly number paper-and-pencil

High School Psychology and its Impact on University Psychology Performance: Some Early Data John Reece Discipline of Psychology School of Health Sciences Impetus for This Research Oh, can you study psychology

DKeane MSMC ED 5700 ICP - Lesson Plan Using Technology in the K-8 Science Curriculum Creating a Jeopardy Review Game using PowerPoint software. (Sample Performance objectives and questions are based on

Spring 0 Michigan Statewide Revised March, 0 Assessment Transition OVERVIEW This document is the next in a series of communications designed to support Michigan districts and schools with the transition

Classroom Identification of Visual-Spatial Learners Steven C. Haas The visual-spatial and auditory-sequential learning model is providing parents and teachers with a whole new way of thinking about how

Motivation: Using the Self-Determination Theory in Foreign Language Classrooms Quite possibly the source of the motivation is very important in a practical sense to teachers who want to stimulate students

Outcomes of Preservice Teacher s Technology Use William F. Morrison, Assistant Professor, Bowling Green State University Tara L. Jeffs, Assistant Professor, East Carolina University Abstract: At a time