2 DISTANCE LEARNING OUTCOMES 2 Abstract This study examined the rates of unsuccessful outcomes for Hilbert College students enrolled in three course formats, distance learning, hybrid and traditional for the period from Spring 2003 through Summer Unsuccessful outcomes were defined as grades of F, I, or W. Courses offered in two or more of these formats were compared. Overall, students enrolled in distance learning courses were twice as likely to receive an F, I or W than were students in the same traditional, on-campus courses. Hybrid courses, combining elements of both distance learning and traditional formats, are relatively new at Hilbert College only being offered since Spring Overall, outcomes for hybrid courses tended to fall in between the other two formats and were more similar to traditional course outcomes than to distance learning outcomes.

3 DISTANCE LEARNING OUTCOMES 3 A Comparison of Unsuccessful Student Outcomes for Distance Learning, Hybrid and Traditional Course Formats At a December 5, 2011 Center for Learning session entitled, Distance Learning Planning Session, several participants informally observed that students in distance learning courses were more likely to get lost over a semester than was the case for traditional, oncampus courses. Although the reports were anecdotal, several faculty spoke of students who were irregular in logging in to access course materials, apparently not completing the readings, lectures and assignments until the last minute. Other examples were given of students who simply seemed to disappear from the course at some point during the semester. Since these reports pointed to possible significant problems that some students may have in taking courses in a distance learning format, a suggestion was made to look at data on grades as a measure of outcomes for courses taught in both distance learning and traditional formats. An additional suggestion was made to also include courses taught as hybrids. Hybrid courses are relatively new at Hilbert College, only having been offered since the spring 2010 semester. These are courses which combine both the traditional live classroom as well as the online environment. A 2010 U.S. Department of Education report from the Office of Planning, Evaluation, and Policy Development was based on a meta-analysis of effect sizes from 99 empirical studies contrasting online and face-to-face instruction. The key findings of the report were that students in online conditions performed modestly better, on average, than those learning the same material through traditional face-to-face instruction (p. xiv), and that instruction combining

4 DISTANCE LEARNING OUTCOMES 4 elements of both formats (hybrids) had a larger advantage relative to traditional instruction than did purely online instruction. Methodology The data for this study were obtained from a PowerCampus database with records of students academic transcripts covering the period of Spring 2003 through the Summer of The current semester, Fall 2011, was not included as final grades were not yet reported. All students with reported final grades for any undergraduate, credit-bearing course were included. Students auditing a course were excluded as was any course that reported grades on a pass-fail. During this period a total of 73,401 students were enrolled in traditional, face-to-face courses, 3,001 were enrolled in online or distance learning courses and 294 students were enrolled in some form of a hybrid course. Outcomes of interest were final grades of F indicating academic failure, I or Incomplete indicating a student s inability to complete coursework due to extenuating circumstances, and W for Withdrawal. Withdrawals from a course may be granted after the drop/add date (end of second week of a semester) and by the ninth week. It recognized that students may receive Incompletes or apply for Withdrawal for a number of varying reasons not all having to do with academic difficulty. It was not possible in this study to separate out those various reasons. Incompletes and withdrawals were included as measures of negative course outcomes because along with the F grade these outcomes result in zero quality points toward a diploma after a student has enrolled and started a course. An incomplete may, of course, be converted to a regular grade if a student satisfactorily completes the coursework with a specified period of time. In those cases the regular grade is substituted. Incompletes represent a potential negative course

5 DISTANCE LEARNING OUTCOMES 5 outcome of interest in that the I grade may be a result of extenuating circumstances related to the course format as well as other circumstances in the student s life. Quality point averages (QPA) over the period of this study were also computed for each course to provide an additional measure of outcome. QPAs are based on a system which awards 4 points per credit hour for an A, 3 points for a B, 2 points for a C, 1 point for a D, and 0 points for an F. Incompletes and withdrawals also result in 0 points for that course. To provide some comparison between formats, the primary data analyses of this report looked at just those courses which, during the period cited above, were offered in at least two of the formats, for example, a course which was conducted as both a traditional course and as an online course at least one time each. Furthermore, online courses were included in the analyses only if the total historical enrollment for the course since spring 2003 was at least ten students. This restriction was an effort to avoid outliers in the data on distance learning course outcomes. A total of 53 courses were conducted in both online and traditional formats. No attempt was made to equate instructors or semesters and years. As mentioned before, hybrid courses are relatively new at Hilbert College. Since the Spring semester of 2010, 22 separate courses have been offered as hybrids at least once. Of these 22, only two courses have been also offered as both online and traditional courses. It should be noted that this study did not look at outcomes other than those described above. Thus there was no attempt made to compare the number of A, B, C or D grades received across formats.

6 DISTANCE LEARNING OUTCOMES 6 Results For all 73,401 students enrolled in traditional courses between spring of 2003 and summer of 2011, 5.52% received an F grade, 0.63% had an Incomplete grade (I), and 5.36% withdrew (W) from the course. Combining these percentages provides a rough measure of absence of success or the percentage of students who received zero Quality Points for the course. Among all students in traditional courses this was 11.51%. In general, students in distance learning or online courses during this same period were twice as likely to have a negative outcome in their courses compared to students in traditional courses. Almost one in ten online students (9.56%) received an F final grade, 1.6% received an I, and 10.7% withdrew from the course (see Figure 1.). The combined negative outcome rate for distance learners was 21.86%. S 03 A 11 Student Count Percent F s Percent I s Percent W s Percent Unsuccessful (F+Inc+W) Distance Learning % 1.60% 10.70% 21.86% Courses Traditional Courses % 0.63% 5.36% 11.51% Figure 1. Outcomes for all distance learning versus traditional courses When only those courses which were provided in both online and traditional formats were examined the results looked very similar as shown in Figure 2. These percentages exclude traditional courses that were not also offered at least once in an online format. They also omit online courses with fewer than 10 total enrollments over the 8 year period (n=9). Typically these small enrolled online courses were internships or independent studies. A total of 53 courses were

7 DISTANCE LEARNING OUTCOMES 7 included, many offered multiple times both online and in traditional formats. The overall outcome results were very similar to those seen in the overall course results with distance learning courses having an approximately two to one greater unsuccessful rate. S 03-A 11 Student Count Percent F s Percent I s Percent W s Unsuccessful Rate Distance % 1.48% 10.19% 20.95% Traditional 12, % 0.77% 5.77% 11.86% Figure 2. Overall outcomes for courses matched on distance learning and traditional formats. An examination of the courses matched for distance learning and traditional formats showed that for 44 of the 53 courses (83.02%) the Unsuccessful Rate was higher for the distance learning format than for the traditional face-to-face format. In 11 of 53 courses (79.25%) distance students had higher percentages of F final grades than did traditional students. In 83.02% of courses (44 of 53) distance students had higher rates of Incompletes and in 75.5% (40 of 53) distance students had higher rates of Withdrawals than did students in the same course taught in a traditional format. Looking at average QPAs for the matched courses, distance learning students earned an overall average QPA of 7.62 compared to an average of 8.43 for students in matched traditional courses. In only 10 of the 53 courses was the QPA of distance learners larger than for their matched traditional counterparts and then by an average of 0.69 QPA points (note: one letter grade difference is equal to 3 QPA points). On the other hand in the 43 courses in which the QPA was higher for traditional students than for distance learners the average difference was

8 DISTANCE LEARNING OUTCOMES QPA points. In 8 of those 43 courses the QPA difference was -2 points or greater and as much as -3.5 points (more than a letter grade) in one course. Examination of the standard deviations for QPAs also revealed that the distribution of letter grades tended to have greater variability or spread in distance learning courses than was the case for the matched traditional, face-to-face courses. Hybrid Courses Between the spring semesters of 2010 to 2011, 22 courses have been offered in a hybrid format with a total of 294 students enrolled. Of these only two have also previously been offered as distance learning courses. Therefore direct comparisons of hybrid to online to traditional formatted courses was not considered possible although in time this should be an area of focus. Overall, in those 22 courses, 5.1% of students received an F grade and 6.1% withdrew. No students had received an Incomplete. The overall unsuccessful rate for hybrids, then, was 11.2% which is comparable to the overall rate for traditional courses and much lower than was seen for distance learning courses. Conclusions Undoubtedly many, if not most, students taking online courses at Hilbert are quite successful. The steady growth of enrollments in these offerings as well as the successful addition of hybrid courses continues to meet very real needs for many of our students. The fact that this study chose to focus on negative rather than positive outcomes is in no way meant as an indictment of online instruction. However, in order to continue to provide the highest quality

9 DISTANCE LEARNING OUTCOMES 9 instruction possible, regardless of format, it is important to consider the potentially negative outcomes as well as the positives. This study provides strong evidence that the risk of a negative outcome, defined as receiving an F, an Incomplete, or withdrawing from a course, is much higher for students who enroll in distance learning courses at Hilbert College when compared to students who take the same courses in a traditional, face-to-face format on-campus. These negative or unsuccessful outcomes can have a very real impact on extending or disrupting a student s college career, especially failing grades and withdrawals. Given that a growing body of research is showing that students in online instruction perform at least as well and possibly modestly better than those learning the same material through traditional face-to-face instruction (U.S. Department of Education, 2010), it is important to look more closely at Hilbert College s online offerings to ask why this does not seem to be the case for our students. Unfortunately there is nothing in the data for this study which can directly address that question. Several possibilities present themselves as factors which may contribute to greater negative outcomes for distance learning students. It is possible, for example, that Hilbert students who self-select to enroll in an online course over a traditional course are not well prepared for that format of instruction. While students are expected and encouraged to attend special orientation sessions for online and hybrid courses, it is not clear how many actually do or how such orientations prepare students beyond providing technical practice with the course management system. In recent semesters at least one faculty member has begun trying out a Readiness survey with prospective students in an online course. It would be useful to look at whether students scores on such an instrument are related to outcomes. If such a survey proved

10 DISTANCE LEARNING OUTCOMES 10 to be even modestly predictive of success/failure a broader use across online and hybrid courses could be of tremendous value. It is also possible that the history of a rather organic growth process for distance learning at Hilbert College has created greater variability in the ways that such courses are conducted than is true for traditional offerings. The very flexibility and individual initiative that has led to the development of online courses might also be a factor in more variable student outcomes. Consideration of these findings will hopefully evoke discussion among the faculty about developing and instituting standards for online and hybrid course offerings at Hilbert. In fact, the discussion group which initially prompted this study was considering this very same issue and recommending work toward a set of best practices in online instruction. The preliminary data from hybrid courses suggest that this format may be more conducive to our students than strictly online offerings. However, as noted, there is as yet insufficient data to draw any firm conclusions from comparisons of this format with that of online or traditional courses. This is a question that should be revisited in another year or two.

Degree and Certificate Requirements Statement of General Education A general education is applicable to all students, no matter what their career and personal ambitions may be. It is the process by which

3359-60-06.2 Graduate student standards. (A) International students. (1) An international student is normally admitted only in the fall, and all credentials should be received by the graduate school by

A STUDY OF STUDENT PERFORMANCE IN AN ONLINE INTRODUCTORY FINANCIAL ACCOUNTING COURSE COMPARED TO A TRADITIONAL CLASSROOM SETTING Mike Simon, MBA, CPA Louisiana State University in Shreveport A STUDY OF

Assessment Findings and Curricular Improvements Department of Psychology Undergraduate Program Assessment Measures The Department of Psychology uses the following measures to assess departmental learning

The University of Alabama in Huntsville 1 Registration Dates, times, procedures and eligibility conditions for registration are published on the UAH website. Registrations for Fall and Spring Semesters

B.S. Psychology 1 Annual Assessment Summary B.S. in Psychology Part One: Summary and Analysis of Assessment Results In Fall 2010, we implemented two changes to our curriculum based on earlier years results

Routing Slip Keep attached to front of proposal Complete your section Sign and date Send to next step Approval Process for New PST Course Name of Department/Faculty Initiating: Name of Course: Step 1 Proposal

62 63 COURSE LOAD The minimum course load for full-time students is 15 semester hours, and the maximum course load is 19 semester hours. Students wishing to enroll in more than 19 semester hours must receive

in Online Database Courses George Garman The Metropolitan State College of Denver ABSTRACT Using data collected from 2002 to 2011, this paper analyzes the determinants of scores earned by students in a

UNIVERSITY LEADERSHIP COUNCIL Faculty Workload & Compensation in Online and Blended Courses Custom Research Brief Research Associate David Godow Research Manager Katie Sue Zellner May 2012 2 of 11 3 of

1 Bryn Mawr College Graduate School of Social Work and Social Research SATISFACTORY ACADEMIC PROGRESS POLICY I. Purpose This policy defines satisfactory academic progress for all Graduate School of Social

Satisfactory Academic Progress (SAP) Policy for Financial Aid GENERAL INFORMATION The federal financial aid programs at UNCP are authorized under Title IV of the Higher Education Act of 1965, as amended.

To: From: Interested Students Abroad Kristi Larson Bedolla Associate Director Program for the Comparative Study of Societies Department of Sociology University of California, Berkeley Regards: International

Internship Information for the Department of Journalism and Public Relations 1. Think ahead. Plan ahead to find a satisfactory internship placement that fits your curriculum and plan of study. It is important

Admissions and Policies Policy Number 02-Mar-2014 Pages of this Policy 1 of 1 4. Overview Covers policies and procedures relating to grading including the course grading system, Grade Point Averages, academic

Title of the submission: A Modest Experiment Comparing Graduate Social Work Classes, on-campus and at A Distance Topic area of the submission: Distance Education Presentation format: Paper Names of the

Academic Policies and Procedures Academic policies can be changed at any time by the University of North Texas Health Science Center School of Public Health (UNTHSC-SPH). Students should review the Student

Grading Systems at the niversity of Notre Dame Law School grading system: http://registrar.nd.edu/lawgradingsystems.pdf rior to 1931 The passing grade in all courses is 70 percent. Any student whose work

Should Graduate Mathematics Courses Be Taught Fully Online? Raid Amin and Kuiyuan Li ramin@uwf.edu kli@uwf.edu Department of Mathematics and Statistics The University of West Florida Pensacola, FL 32514

Wichita State University COUNSELING, EDUCATIONAL LEADERSHIP, EDUCATIONAL AND SCHOOL PSYCHOLOGY SCHOOL PSYCHOLOGY PROGRAM DESCRIPTION DESCRIPTION The Department of Counseling, Educational Leadership, Educational

TEACHER PROGRAM FREQUENTLY ASKED QUESTIONS Admission & Program Requirements What is the first step? Review the program details and complete the online application. What are the entrance requirements? Review

Florida A&M University Graduate Policies and Procedures Each graduate program has a different mission, and some programs may have requirements additional to or different from those in the Graduate School.

AFFIRMATION OF COMMUNITY RESPONSIBILITY Illinois College is committed to the development and welfare of every member of our community: students, faculty, staff, and administration. To achieve the working

Michigan State University, College of Nursing Certificate in College Teaching Program The College of Nursing Certificate in College Teaching Program (CCTP) transforms the way graduate students are prepared

ACADEMIC REGULATIONS COLLEGE-WIDE ACADEMIC STANDARDS Academic regulations for all programs at ACPHS are developed and adopted by the faculty and are administered by the College administration. Oversight

credit for the course. Change from credit to audit status must be done before the 12th class day in the long session and before a proportional period of time in the summer and short sessions. Grading System

FERRIS STATE UNIVERSITY COLLEGE OF HEALTH PROFESSIONS SCHOOL OF NURSING HANDBOOK FOR GRADUATE NURSING STUDENTS Supplement to the Ferris State University Code of Student Community Standards JULY 2015 TABLE

FERRIS STATE UNIVERSITY COLLEGE OF HEALTH PROFESSIONS SCHOOL OF NURSING HANDBOOK FOR GRADUATE NURSING STUDENTS Supplement to the Ferris State University Code of Student Community Standards JULY 2012 TABLE

Congratulations on your admission into our Program! First, the faculty of Tarleton s College of Education and the Department of Curriculum and Instruction are happy to welcome you to our graduate program!

Guidelines for the Development of Certificate Programs University of Nebraska at Kearney I. Purposes of Certificate Programs Certificate programs provide a means for the University of Nebraska at Kearney

Student Handbook Master of Science Program in Educational Psychology Georgia State University Department of Educational Psychology and Special Education This document has been prepared as an overview for

Graduate Council Document 11-4j Approved by the Graduate Council on 2/16/12 GRADUATE CERTIFICATE PROGRAM DISTANCE EDUCATION SUBMITTED BY DEPARTMENT OF CURRICULUM AND INSTRUCTION COLLEGE OF EDUCATION PURDUE

College Vocabulary Handbook academic concentration See major. academic year Usually this refers to the September-June school year. In some cases it refers to the entire year. accreditation Certification

International Journal for the Scholarship of Teaching and Learning Volume 6 Number 2 Article 25 7-2012 Analysis of the Effectiveness of Traditional Versus Hybrid Student Performance for an Elementary Statistics

Department of Computer Science Academic Policies Version 4 August 2008 This manual describes the academic policies for the Master of Science Degree in Computer Science, Master of Science Degree in Computer

Rockhurst University Financial Aid Programs & Policies Rockhurst University is pleased to be able to offer a financial assistance program for students who would be unable to attend without such aid. The

Program Overview What is LIFTS? Department of Administrative and Policy Studies Leadership Initiative For Transforming Schools (LIFTS) K-12 Principal Certificate Program Leadership Initiative For Transforming

Online Class* Development Guidelines Middlesex Community College March 11, 2015 I. Online Class Proposal: Submission and Review** The proposal to develop a new online course should start six months before

ADMISSION REQUIREMENTS AND PROCEDURES Changes to some admission requirements have occurred, edits are indicated below in red. Santa Fe University of Art and Design is an academically challenging institution

Department of Computer Science Academic Policies January 2012 This manual describes the academic policies for the Master of Science Degree in Computer Science, Master of Science Degree in Computer Information

Transfer Advising for the Human Development Major Undergraduate Advising Services, 2010 Why am I watching this? You probably fall into one of these categories: You came to an orientation where no Human

STATE UNIVERSITY OF NEW YORK AT ALBANY Department of Economics Doctor of Philosophy Program Guidelines: May 2013 I. Admission Each application is evaluated on its individual merits by the Economics Department.

ENGLISH DEPARTMENT GRADUATE HANDBOOK UNIVERSITY OF TEXAS AT ARLINGTON JANUARY 2015 2 TABLE OF CONTENTS General Information...3 Academic Calendar...4 Forms...4 Registering for Courses...4 General Courses...4

ADMISSION REQUIREMENTS BY PROGRAM MASTER OF ARTS IN COUNSELING Candidates for admission to the Master of Arts in Counseling program must meet the following criteria: 1. Have earned an undergraduate degree

Dual Enrollment Dual enrollment is an acceleration mechanism that allows students to pursue an advanced curriculum relevant to their individual postsecondary interests. Each year, more than 50,000 students

ACCOUNTING Ph.D. PROGRAM REQUIREMENTS KRANNERT GRADUATE SCHOOL OF MANAGEMENT (revised July 2014) Introduction The doctoral program in Accounting at the Krannert Graduate School of Management is designed

GUIDE TO GRADUATE STUDIES DEPARTMENT OF SOCIOLOGY Terminal (one-year) MA Program UNIVERSITY OF CINCINNATI 2014-2015 GRADUATE STUDY IN THE DEPARTMENT OF SOCIOLOGY In addition to the PhD program, the department