Educators have long sought to define quality in education. With the
proliferation of distance education and online learning powered by the
Internet, the tasks required to assess the quality of online programs
become even more challenging. To assist educators and institutions in
search of quality assurance methods to continuously improve their
distance education programs, the Sloan Consortium (Sloan-C) published
Elements of Quality: The Sloan-C Framework (Moore, 2002), outlining five
pillars of quality--learning effectiveness, access, student
satisfaction, faculty satisfaction, and cost effectiveness for online
programs. Based on a relevant literature review, this article explores
the reasons behind the push for online program quality assurance, key
benchmarks recommended by major accreditation agencies and some best
practices currently utilized to ensure online program quality standard.
It serves as a starting point for distance education administrators and
educators to formulate program goals and assessment policies regarding
their online programs.

Introduction

The Internet has created unprecedented opportunities for widespread
electronic delivery of information and services. Education is becoming a
ubiquitous service delivered over global networks with the promise of
being accessible anytime and anywhere. With the growing popularity of
distance education and online programs, compelling questions demand
attention--How do we assess the quality of such a program under the new
paradigm? What makes a cyber-college or online university worth
attending (Witherspoon & Johnstone, 2001)?

To assist educators and institutions in search of quality assurance
to continuously improve their online programs, the Sloan Consortium
(Sloan-C) published Elements of Quality: The Sloan-C Framework (Moore,
2002), outlining a scaffold with five pillars of quality--learning
effectiveness, access, student satisfaction, faculty satisfaction, and
cost effectiveness. Based on a review of relevant literature, this
article explores the reasons behind the push for online program quality
assurance, key benchmarks recommended by major accreditation agencies,
and some best practices currently implemented to ensure online program
quality standards. It serves as a starting point for distance education
administrators and educators to devise program goals and assessment
policies regarding their online programs.

Need for Quality Assurance

The globalized information economy and its marketplace are
propelling educators to reshape higher education around the world. Never
has education been so important to so many. Governments, companies, and
individuals all recognize that while an assembly-line worker is
valuable, the real competitive advantage comes from a well-educated
mind, producing breakthrough ideas that advance technologies and lead to
new products, new initiatives, and ultimately a stronger society. As
universities and businesses alike implement updated strategies they are
redefining venue and pedagogy. Consequently, they must also redefine
measures of quality (Witherspoon & Johnstone, 2001).

McLoughlin and Visser (2003) assert that educational quality
assurance is a matter of accountability and national interest.
Governments at the federal, state, and local levels mandate it.
Accreditation agencies require it. The general public expects it, and
faculty need it to support their actions (McKenzie, Mims & Bennett,
2003). In the U.S., the benchmarks for measuring educational quality
have traditionally been provided by accreditation agencies at regional
and national levels. Responding to the challenges that distance
education presents under the Internet Age, major accreditors have made
significant changes to their standards, policies, and procedures that
serve as benchmarks, also known as best practices for quality online
education. The most widely followed guidelines are provided by (1) the
Council of Regional Accrediting Commissions (C-RAC, 2000); (2) the
American Federation of Teachers (AFT, 2000); and (3) the Council for
Higher Education Accreditation (CHEA, 2002).

Being consumers of educational services, individual students, their
parents or employers also demand quality assurance as higher education
is becoming a more consumer-driven market (Carnevale, 2000). The world
is well along in its transition from an industrial age to an information
society where knowledge is accumulating at an exponential rate,
information technology has permeated into nearly every aspect of
one's life, and information--its acquisition, management, and
deployment is the key competitive advantage (Alva, 1999). The
knowledge-based economy mandates workers to continuously improve and
upgrade skills in order to process information and create new knowledge.
Thus, lifelong learning and just-in-time learning is becoming
progressively critical to employees seeking to upgrade their skills and
to employers pursuing individuals with the necessary experience and
education to help their organizations succeed. To remain viable in the
competitive marketplace, these learners and their employers are able and
willing to pay the price necessary to obtain the required qualification
at the time and the place they need it. This potentially huge lifelong
learning market has given rise to the commercialization of education as
it is increasingly seen as another application of e-commerce. Indeed,
competition is on the rise with the private sector (e.g., University of
Phoenix and DeVry University) entering the education arena. Competition
brings choices and choices empower consumers, who can now shop around
for the best products and services at the most reasonable prices.
McLoughlin and Visser (2003) point out that as today's students pay
a greater share of their own educational costs, they expect universities
to provide services they demand in the market at large: better service,
lower price, higher quality, and a mix of products that satisfies their
own sense of good education.

Echoing assertions by Fernandez and Lampikoski that client
satisfaction is a good indicator of quality, Blfer (2000) claims that
student satisfaction is a key criterion for institutions to determine
quality in distance education. This idea is also affirmed in the
statement that student satisfaction is the most important factor for
continuing education as it reflects learners' evaluation of the
quality of all aspects of the educational program (Moore, 2002).

Best Practices Supporting Quality Assurance

A number of organizations and accreditation agencies have developed
sets of principles, guidelines and benchmarks in an effort to ensure
quality online programs. An overview of several such best practices is
presented in this section.

The five pillars constituting the Sloan-C Quality Framework (Moore,
2002) define the following elements:

1. Learning effectiveness (LE) -- demonstrates that learners who
complete an online program receive educations that represent the
distinctive quality of the institution. The goal is for online learning
to be equivalent to or better than learning through the
institution's other delivery modes (traditional face-to-face,
classroom-based instruction). There are nine specific principles
applicable to LE.

2. Access -- provides the means for all qualified and motivated
students to complete courses, degrees or programs in their disciplines
of choice. Access includes three areas of support: academic (e.g.,
tutoring, advising and library); administrative (e.g., financial aid and
disability support); and technical (e.g., hardware reliability, uptime
and help desk). There are twenty distinct principles associated with
Access.

3. Student satisfaction (SS) -- reflects the effectiveness of all
aspects of the educational experience. The goal is that all students who
complete a course express satisfaction with course rigor and fairness,
with professor and peer interaction, and with support services. There
are five principles pertinent to SS.

4. Faculty satisfaction (FS) -- indicates that instructors find the
online teaching experience personally rewarding and professionally
beneficial. Personal factors contributing to FS include opportunities to
extend interactive learning communities to new populations of students,
to conduct and publish research related to online program modality, and
to achieve recognition and collegiality. Institutional factors
associated to FS include support, rewards, and institutional
study/research. There are eight principles relevant to FS.

5. Cost effectiveness (CE) -- enables institutions to offer their
best educational value to learners. Online programs are regionally
accredited in the same manner as on-campus ones. The goal is to control
costs so that tuition is affordable yet sufficient to meet development
and maintenance costs, and to provide a return on investment in startup
and infrastructure. There are four principles related to CE.

The Best Practices developed by C-RAC (2000) have outlined five
components to address a particular area of institutional activity
relevant to distance education.

1. Institutional context and commitment -- include consistency of
the online program with the institution's role and mission; its
budgetary and policy commitment; the adequacy of technical and physical
plant facilities; reasonable technical support for faculty and students;
appropriate internal organizational structure which enables the
development, coordination, support, and oversight of the online program;
and meeting the legal and regulatory requirements.

2. Curriculum and instruction -- constitute assurance that each
program of study results in collegiate level learning outcomes
appropriate to the rigor and breadth of the degree or certificate
awarded by the institute; full participation of academically qualified
persons in the decisions concerning program curricula and oversight;
implementation of a coherent plan for students to access all courses
necessary to complete the program; appropriate interaction between
instructor and students and among students; and consortia or outsourcing
standards.

3. Faculty support -- addresses issues of faculty workload;
compensation; evaluation; ownership of intellectual property resulting
from the program; on-going technical support and training; and support
for course design and management.

4. Student support -- comprises institutional commitment
(administrative, financial and technical) to a time period sufficient
for all admitted students to complete the program; proper communication
with students regarding enrollment standard, technical competency,
curriculum design, time frame, cost, payment and refund policies;
adequate services provided in the areas of library, bookstore, academic
advising, financial aid, tutoring, career counseling and placement, and
ongoing technical support and training.

5. Evaluation and assessment -- consist of documenting assessment
of student achievement conducted in each course and at the completion of
the program by comparing student performance to intended learning
outcomes; assuring the integrity of student work; securing personal
information in the conduct of assessment and dissemination of results;
carrying out ongoing self-evaluations pertinent to program improvement,
more effective use of technology to improve pedagogy, student
achievement of intended outcomes, improved retention rates, effective
use of resources, and enhanced services to its internal and external
constituencies.

The Guideline for Good Practice advocated by AFT (2000) has
recommended fourteen standards to gauge quality of distance education.
Incorporated in the standards are: (1) faculty must maintain academic
control; (2) faculty must be equipped to meet the special requirements
of teaching at a distance; (3) course design should be shaped to the
potentials of the medium; (4) students must be fully aware of course
requirements and be prepared to succeed; (5) close personal interaction
must be maintained; (6) class size should be set through normal faculty
channels; (7) courses should cover all materials; (8) experimentation
with a wide range of subjects should be encouraged; (9) equivalent
research opportunities must be provided; (10) student assessments should
be comparable; (11) equivalent advisement opportunities must be offered;
(12) faculty should retain creative control over use and re-use of
materials; (13) full undergraduate degree program should include
same-time same-place coursework; and (14) evaluation of distance
coursework should be undertaken at all levels.

CHEA requires that institutions must document, as a process of
program evaluation, that they are in fact meeting their educational
missions and goals and their students' outcomes are at an
acceptable level. This is applicable to both distance and campus-based
learning.

Although these accreditation agencies vary in their benchmarks
governing quality standards for online programs, they have uniformly
emphasized the following elements in their guidelines: (1) strong
institutional commitment; (2) adequate curriculum and instruction that
fit the new delivery medium and match the rigor and breadth of
equivalent on-campus programs; (3) sufficient faculty support; (4) ample
student support; and (5) consistent learning outcome assessment. The
following section presents some best practices that are
research-supported and empirically tested as successful implementations
of the essential quality elements discussed thus far.

Exemplary Quality Assurance in Practice

1. Administrative leadership and support

Lee and Dxiuban (2002) assert that the fundamental quality
assurance strategy to support online programs for any institution is a
functional administrative, technological, and organizational
infrastructure. The University of Central Florida (UCF) offers web-based
courses to its diverse, growing student population. UCF uses a
centralized approach which deploys a technical infrastructure design
that provides administrative leadership, structures faculty development,
and assesses course delivery services.

The Office of Academic Affairs, through the Vice-Provost for
Academic Programs, initiates service through several units. The Center
for Distributed Learning (CDL) administers interactive television, video
and online programs. CDL acts as an information clearinghouse to
initiate administrative support and resolve accreditation issues created
by transforming face-to-face programs to the virtual environment.
Cavanaugh (2002) notes that UCF's budget reflects the importance of
distance learning programs in meeting institutional goals. The
university funds the technical infrastructure, faculty development,
learner support, research and development in distance learning, and
impact evaluation. The technical infrastructure is made up of wired and
wireless network connections to every building, access to Internet,
dedicated servers for online courses, and always-on access to
information and services.

2. Faculty support

Corderoy, Stace, and Pennell (2002) cite best practices for faculty
support at the University of Wollongong, Australia. As a support to
instructors involved in the design, development, implementation,
evaluation, and enhancement of subjects for flexible delivery, the
university has provided, through its Centre for Educational Development
and Interactive Resources (CEDIR), academic staff development,
educational design support as well as production support for the
development of teaching resources required. Resources to assist these
processes have also been developed and made available at the Flexible
Delivery website: http://cedir.uow.edu.au/CEDIR/services/serv_flexdel_01.html.

Cavanaugh (2002) finds that faculty development services at UCF are
especially noteworthy. All distance teaching instructors take a faculty
development course, for which they receive a stipend or a laptop
computer. They are given release time and extra pay for course
development and are assisted in all phases of course design and delivery
by trained Tech Rangers. According to Corderoy, et al. (2002), the
Faculty Center for Teaching and Learning (FCTL) at UCF offers a wide
range of support services for faculty entering the web-based teaching
environment. FCTL's centerpiece is a summer institute where
faculties from across campus integrate to create innovative teaching
approaches including online instruction. The Research Initiative for
Teaching Effectiveness (RITE) is an internally funded unit that assists
faculty research in effective teaching practices.

3. Student support

To support online students, UCF has built a full range of services
delivered on the Web ranging from application for admission, course
registration, course schedules, technical guides, online orientation,
financial support information, to library access and textbook
purchasing. 24/7 technical support is provided by phone or Web and a
group of CyberKnights act as student computing consultants. A CD-ROM containing Internet software, a browser test, tutorials, and automatic
configuration for network dial-up access is distributed at the time of a
student registration. Impact evaluation at UCF focuses on student issues
such as success rates, withdrawal rates, learning styles, attitudes, and
demographics (Cavanaugh, 2002).

4. Curriculum design and instruction

Based on Corderoy, et al. (2002), to maintain curriculum
continuity, ongoing evaluation of individual courses related to the
overall program is conducted at UCF. Evaluation strategies utilize
comments from outside reviewers, student inputs, faculty evaluations,
current research relevant to online programs, and professional
literature. Web course development at the university reflects a team
approach involving subject matter experts, instructional designers, web
programmers, and graphics artists. As a result, the end product
encompasses the instructor's content choice, interaction,
assessment, and other functional aspects. After a decision is made
regarding either developing a new online course or converting an
existing face-to-face course, the instructor usually conducts a
comprehensive course analysis relative to the program focusing on course
prerequisites, program sequencing, assessment instruments, instructional
methodologies, and communication strategies.

5. Assessing student achievement

Indeed, Carnevale (2001) states that assessment is taking center
stage as online educators experiment with new ways of teaching and
proving that they are teaching effectively. Higher education is moving
toward outcomes-based assessments with online education leading the way.
Although assessment models vary in the literature, the current practices
used in assessing student achievement in an online program are broadly
divided into two paradigms: (1) associationist- and behaviorist-based
assessment with scientific measurement, and (2) cognitive- and
constructivist-based assessment with qualitative measurement (Shepard,
2000). The former emphasizes the learning outcome while the latter
measures the interaction during the learning process.

Leading in the practice of utilizing standardized tests in the U.S.
are the Western Governors University, the University of Phoenix Online,
Excelsior College in Albany, NY, Pennsylvania State University's
World Campus, Thomas Edison State College in Trenton, NJ, the State
University of New York's Empire State College, and University of
Maryland University College. These schools require students seeking a
degree to show competency in a number of domains including general
education (e.g., writing and mathematics) and subject specific knowledge
(e.g., business management). Outlined competencies are met through
passing of a series of assessment exams (Carnevale, 2001).

O'Reilly (2000) advocates alternative assessment or authentic
assessment methodologies which are formative in nature. The best
practices extracted from his research include: (a) group support for
individual product ("Discrimination and the Law" project used
at Macquarie University, Australia); (b) team collaboration for common
product ("Science and Communication" website at University of
Melbourne, Australia and "Environmental Psychology," a
co-authored paper for publication at University of Pretoria, South
Africa); (c) peer review plus self-review (assess students' own
progress and the contribution of their peers); and (d) debate
("Communication and the Media" online debate internationally
collaborated among University of South Australia, Governors State
University, Chicago and University of Technology, Sydney).

CONCLUSION

With the enabling technology readily available and a growing demand
for lifelong and just-in-time learning, online programs which free
learners from the constraints of time and space and offer flexible
learning opportunities are flourishing. At the same time, competition is
also intensifying as traditional publicly funded universities, the new
for-profit, post-secondary institutions, corporate universities, and
training companies contend for "knowledge workers" who are
ready and willing to pay for skill upgrade (Bates, 2000). Hence,
insuring the success and validity of online programs lies in an
institution's ability to deliver a high quality education service
at a reasonable price.

The best practices cited in this article can serve as a baseline
for institutions seeking quality assurance in the five key areas of (1)
institutional commitment (e.g., administrative leadership and support,
technical infrastructure, and budget priority); (2) curriculum and
instructional development (e.g., team approach in course design, ongoing
course evaluation, and applying online learning pedagogy); (3) faculty
support (e.g., faculty development, ongoing technical support, and
institutional rewards); (4) student support (e.g., full range of
academic and administrative services, interaction with faculty and
peers, and technical support); and (5) learning outcome assessment
(e.g., learning outcome assessment (summative and aptitude) and learning
process assessment (formative and authentic)).

Online education is incredibly dynamic and constantly driven by
changes in demand and technology. Therefore, the author concurs with
C-RAC's (2000) position, "Given the rapid pace of change in
distance education, these Best Practices are necessarily a work in
progress" (p. 1). As practitioners and theorists continue their
efforts to explore new venues to assess quality of online programs, no
doubt more examples of best practices will continue to emerge. Thus,
this article serves only as a starting point for online program
administrators and educators as they engage in further study and
practice of quality assurance.

References

Alva, J. K. (1999). High education and its millennial predicament.
The League for Innovation in the Community College Conference on
Information Technology.

O'Reilly, M. (2000). Assessment of online interaction: Helping
or hindering the goals of educators and learners? World Conference on
Educational Multimedia, Hypermedia and Telecommunications 2000(1),
868-873.

Shepard, L. A. (2000). The role of assessment in a learning
culture. Educational Researcher, 29(7), 4-14.

The American Federation of Teachers (2000). Distance education,
guidelines for good practice. Retrieved May 20, 2004, from
http://www.aft.org/higher_ed/downloadable/distance.pdf

The Council of Regional Accrediting Commissions (2000). Best
practices for electronically offered degrees and certificate programs.
Retrieved May 20, 2004, from
http://www.ncacihe.org/resources/electronic_degrees/Best_Pract_DEd.pdf

The Council for Higher Education Accreditation (2002).
Accreditation and Assuring Quality in Distance Learning. CHEA Monograph
(1, Series 2002). Retrieved May 20, 2004, from
http://www.chea.org/pdf/mono_1_accred_distance_02.pdf?pubID=246