Allison M. Leppke, MPH, CHES

Kara B. Robinson, MS

Erik P. Mettler, MPA, MPH

Kathleen R. Miner, PhD, MPH, CHES

Iris Smith, PhD, MPH
Rollins School of Public Health at Emory University

Abstract

Training of public health professionals through web-based technology is rapidly increasing. This article describes one school of public health's effort to establish an online Master's program that serves students nationally and internationally. It examines the critical components in the design and implementation of distance education, including curriculum and instruction, management, facilities and finance, school support, technology services and support, student services, learning resources, and evaluation of the program. Focal components for the development of successful online programs recommended from the literature are administrative support, allocation of time for planning, provision of faculty and learner support, and matching technology to the needs of the learner. This article provides some guidance in the development of the organizational structure and the supporting components for online educational programs.

Distance education is defined as “planned learning that normally occurs in a different place from teaching and as a result it requires special techniques of course design, special instructional techniques, special methods of communication by electronic and other technology, as well as special techniques of course design, and other technology, as well as special organizational and administrative arrangements” (Moore & Kearsley, 1996, p. 2).

Distance education provides several advantages to students. “The Internet is enabling us to address these educational challenges, bringing learning to students instead of bringing students to learning. It is allowing for the creation of learning communities that defy the constraints of time and distance as it provides access to knowledge that was once difficult to obtain” (Web-based Education Commission, 2000, p. i). The Distance Education Training Council (2003) reported decreasing lost work days, travel and living expenses associated with a traditional program, and the flexibility of distance study as important benefits of distance learning. In addition, the students are able to set their own pace and schedule for learning, and experience learning in a one-on-one setting. Additionally, distance education can help address the growing need of training the public health workforce. In the Institute of Medicine report Who will keep the Public Healthy? , it advocated for distance education as a method “…Enabling workers to continue in their work responsibilities by completing self-paced coursework, this approach reduces the burden overworked and understaffed agencies feel as their staff members participate in educational programs” (Institute of Medicine, 2003, p. 56).

The Distance Education at Degree-Granting Postsecondary Institutions: 2000-2001 report from National Center for Education Statistics (NCES) found that more than half (56 %) of the nation's two- and four-year degree-granting institutions offered distance education courses in the 2000-2001 academic year . Furthermore, the Association of Schools of Public Health (2005) reported that 12 of the 36 (33%) accredited schools of public health provide distance education in some format. According to the U.S. Distance Learning Association (USDLA) (2003), there will be over 2.3 million U.S. students taking distance learning courses in the next few years.

Due to the growth of distance education, there is a need for systematic program planning. Previous research has suggested criteria to consider in planning distance education programs. The Commission on Institutions of Higher Education North Central Association of Colleges and Schools (NCA) (2001) provides guidelines under the following components: curriculum and instruction, evaluation and assessment, library and learning resources, student services, facilities, and finances. Eddy, Donahue, and Chaney (2001) described six important factors in the design of a distance education program in health education, including learner, technology, faculty, profession, purchaser, and university.

In designing distance learning programs, research has found factors that are common barriers and elements that lead to success. Berge and Muilenburg (2001) identified faculty compensation and time, organizational change, and lack of technical expertise and support as prominent obstacles faced by distance learning program at all stages of development. On the other hand, key factors to successful distance learning relate to issues of faculty, student orientation to the school and technology, student services for advisement, and administrative support (Garrett & Weiner, 1999). Within faculty considerations, a program should hire faculty specifically for distance education to allow for preparation time, train its faculty well, and develop a policy for faculty compensation. Moore (1996) recommended five tips for setting up a successful distance learning program: (1) obtain management support, (2) develop the program in small steps, (3) allot time and resources for planning and designing, (4) select media tools according to program needs, and (5) provide learner support. After the development of a distance program, Berge (2001) identified support services for the learners, support services for instructors, administration support, and program evaluation as critical factors in sustaining distance training. Thus, the design of distance learning requires a great investment in program infrastructure.

The purpose of our article is to describe the development and components of the Career Masters of Public Health (CMPH) program, a distance education program for a Masters in Public Health degree, at the Rollins School of Public Health at Emory University . Within our description, we will discuss its development applying the Distance Education Program Development Model (Figure 1), that incorporates the recommended components for building a distance education program found in the literature.

Figure 1. The Distance Education Program Development Model

Program Application

History of CMPH

The Career Masters of Public Health program was funded by the Centers for Disease Control and Prevention (CDC). The Graduate Certificate Program (GCP) grant's function was to support and supply federal government employees with continuing education, thereby enabling them to become professionally proficient in the field of public health. Four schools of public health were awarded with the grant: the Rollins School of Public Health at Emory University , Johns Hopkins University , Tulane University , and the University of Washington .

The Rollins School of Public Health's (RSPH) proposal was to provide a certificate program in public health to government employees, specifically CDC public health advisors. This certificate program provided the participants with 18-24 hours of public health training, including the majority of the core classes required in a masters level curriculum. The program had three concentrations: Epidemiology, Health Management, and Prevention. One stipulation added to the proposal was that RSPH would offer an additional program for the individuals to earn all necessary credits to fulfill the requirements of a Masters of Public Health degree.

After the grant was awarded, RSPH utilized the first year to develop and implement the program. During this year of development, the school encountered several challenges. The first was to select a user friendly system and technology infrastructure that students would be able to access and use in course interactions. RSPH employed various course management systems in the program's evolution: a in house e-learning system, WebCT, and finally Blackboard. Additional challenges included building the comprehensive program staff and infrastructure. Finally, the program had to be promoted among administration, faculty, and information services to institutionalize it within the school and the greater university.

As the grant advanced to the second year, the program began offering courses adapted from the traditional RSPH curriculum. The implementation of the program was met with issues of student recruitment, intensive faculty training, and general acceptance of distance learning. The low enrollment during the first year of course offerings led RSPH to offer the program to state and local public health officials in addition to CDC staff. This decision increased enrollment and interest in the program dramatically. The GCP allowed for the development of the current RSPH Career Masters of Public Health (CMPH) Program.

The following section will provide a description of the current CMPH program which is in its fifth year. The program was built around the planning model presented in this article, and CMPH has continued refine and improve the components over the past five years. The program components addressed will include the students, curriculum and instruction, program management, facilities and finances, school and university support, student services, technology services and support, library and learning resources, and evaluation and assessment.

The CMPH program is designed for mid-career professionals who have 5 to 10 years experience in public health or related fields. The students come from a variety of settings including federal, state, and local public health agencies, non-profit organizations, hospitals and other health care settings, research centers, and insurance and pharmaceutical companies. Approximately 45% of students entering in fall 2003 already had a master's level degree or higher. The average age of a CMPH student is thirty-eight years old. Approximately half of the students reside in Georgia with the other half representing 28 states, 1 U.S. territory, and 3 countries. Approximately 40% of CMPH students represent minority groups. The demographic of the typical CMPH student has shifted as the program has developed. Initially, a majority of the students worked for public health agencies. With the addition of new options to the curriculum, a growing number of CMPH students are drawn from the private sector.

The Distance Education Program Development Model

This section will describe the components in the Distance Learning Model and how they have been applied at the Rollins School of Public Health.

Curriculum and Instruction

The CMPH program provides effective, engaging, and easily accessible graduate courses to working professionals through a combination of web-based technologies and face-to-face interaction. Students come on-campus for course orientation at the beginning of the semester and course conclusion at the end of the semester. Between those times, students interact with professors, graduate assistants, and other students using the World Wide Web and the Blackboard learning management system. The program has the same 42-hour credits as the traditional Masters of Public Health program. The program offers three options: prevention, health care outcomes, and maternal and child health (MCH) epidemiology. Students are required to take six core classes which include: Social Behavior in Public Health, Introduction to Biostatistics, Perspectives in Environmental Health, Fundamentals of Epidemiology, Health Policy and Resource Allocation, and Introduction to Health Care Management. Students typically take three courses or six credit hours per semester. All students are required to complete a Special Studies Project (SSP), or thesis project. The SSP is a capstone experience that integrates the knowledge and skills from coursework with the professional practice of public health.

Faculty for the CMPH program are comprised of traditional faculty from the Rollins School of Public Health and adjunct faculty from the Centers for Disease Control and Prevention, American Cancer Society, and other public health organizations. The program offers training to the faculty through workshops and faculty retreats. All faculty receive a faculty orientation manual that provides an overview of distance education, instructional design principles and strategies, a guide to online course development, and worksheets. The manual worksheets include developing learning units for students, outlining learning objectives, readings, and assignments, and planning for different teaching strategies to engage the students.

Faculty members also receive training through technology learning series courses and learning opportunities developed by the coordinator of faculty professional development. They learn about web editing, basics of learning management systems, gaming, use of new software, and other technology-related courses through 2 to 4 hour workshops offered online and face-to-face. An innovations lecture series highlights the use of technology in public health and teaching. Many faculty present at and attend these lectures. A listserv for faculty facilitates communications between the CMPH administration and the faculty and allows for sharing of ideas or discussions among faculty.

Consult with appropriate staff and faculty to develop multimedia materials

Implement and support course applications

Create, maintain, and archive all courses

Research emerging technological applications

A comprehensive team approach has been designed for CMPH course development. The team includes faculty, instructional designer, graduate assistant, and multimedia staff. See Table 1 for the roles of each team member. An instructional designer (ID) plans the course content, assignments, and technological considerations with the faculty. They share knowledge about events for instruction and types of learning (Gagne, Briggs, & Wagner, 1992) and building community online to facilitate course interactions (Palloff & Pratt, 1999). The faculty and ID discuss the ideas of teaching strategies for web-based learning such as use of different media, building a learning community, cooperative learning, interactive teaching strategies, and methods for learning assessment. They work through the worksheets for planning a course in the faculty orientation manual. A graduate student is assigned to help the faculty monitor the course site and student participation, help with construction of the course site, and facilitate course communication. In addition to instructional design, faculty may receive multimedia support to develop interactive course materials. For example, faculty from an international health course and a prevention effectiveness course have worked with the multimedia team to develop course CD-ROMs of Powerpoint or video presentations of guest speakers to supplement course materials and readings.

Program Management

The managerial organizational matrix for the CMPH program is divided into four main roles: program director, coordinator of faculty professional development in distance leaning, assistant director of academic programs (ADAP), and general administration. The duties and responsibilities included in each of these roles are an essential part of the CMPH program's internal mechanism. See Table 2 for the roles of each staff member.

Deliver professional development activities for students, faculty and staff

Ensure that the program is appraised of the latest and best technology

Assist with faculty and staff training in distance learning and software packages

Implement new program initiatives

Program promotion

Assistant Director of Academic Programs (ADAP)

Primary contact for students in the program

Recruit prospective students

Handle admissions process

Assist students in coordinating their academic course of study

Maintain and facilitates the student listserv

Promote the program

Provide career counseling and guidance

Serve as a liaison between the program, faculty, and students

General Administration

 General contact person for the program
 Manage day-to-day operations of the program
 Coordinate faculty contracts
 Schedule logistics for meeting and on-campus events

The program director provides universal management of the program. Specific duties of the director include communication within the school, maintaining relationships with stakeholders outside the school, marketing, and faculty recruitment. The coordinator of faculty professional development in distance learning delivers professional development activities for students, faculty and staff of the program. The coordinator ensures that the program is appraised of the latest and best technology available to deliver innovative distance education. Duties of the coordinator include assisting with faculty and staff training in distance learning and software packages, implementation of new program initiatives, and program promotion. The ADAP serves as the primary contact for students in the CMPH program. The ADAP's duties include assisting students in coordinating their academic course of study, maintaining and facilitating the student listserv, recruiting prospective students, promoting the program, handling the admissions process, and providing career counseling and guidance. As a result of direct contact with the students the ADAP serves as a liaison between the program, faculty, and students. The administrative associate (AA) manages the day-to-day operations of the program. Additional staff includes information services, instructors, instructional designers, graduate assistants, and multimedia staff.

The infrastructure of the CMPH program is complex and diverse. In order to maintain a consistent and streamlined process, it is essential that a specific communication structure be established through monthly staff and ongoing small group meetings. Additionally, current listservs for staff, faculty, and students are available and active. These actions help ensure that everyone in the department is kept up-to-date on both internal and external activities of staff, faculty, and students.

Facilities and Finances

The facilities for the CMPH program are housed within the school of public health and consist mainly of staff office space. During the face-to-face class sessions, the classrooms and computer labs for the traditional MPH program are used. This is accomplished by scheduling the face-to-face sessions for Fridays, Saturdays, and Sundays, which are the days the traditional students do not typically have classes. During the rest of the semester, the students, faculty, and staff communicate through the web-based setting typical of distance learning.

While seed money for development from Graduate Certificate Program (GCP) helped to start the program, the program is tuition driven and financially self-supporting through student tuition, which is assessed at the same hourly tuition fee as the face-to-face MPH program at Rollins. Some school specific scholarships are available for students, including the Merit, Hearst, Seretean, and the public health preparedness (GAPHPS) scholarships. Many avenues exist for the CMPH program to remain financially self-supporting and profitable in the future. Tuition income and grants brought in by the CMPH program are sufficient to cover the costs of the administrative staff and support structure described.

School and University Support

The administration has been supportive of the program, and the CMPH program has been increasingly integrated into the Rollins School of Public Health over the years. The Director of the program attends the departmental chair meetings and CMPH staff sits on the school-wide curriculum committee. CMPH has also expanded the technological capacity of the school by providing a course management system for both traditional and CMPH courses and offering training for faculty and staff in technological applications and distance education.

Student Services

The CMPH program provides comprehensive student services for distance learning students. Admissions/recruitment, international student affairs, professional development, enrollment, and financial aid are services provided to all RSPH students. The CMPH program works in conjunction with the RSPH Office of Student Services to provide targeted services for distance learners in these areas. This section will address student services tailored to meet the needs of distance students.

Recruitment takes places via several methods: conferences, presentations, referrals, and print and online media. In addition, CMPH participates in the school wide Open House and Visit Day events. The Open House is an event for mostly prospective students to learn more about the academic programs at the Rollins School of Public Health . Visit Emory Day provides accepted and prospective students with in-depth information about our academic programs, student services, and the Atlanta public health community . CMPH staff attends approximately three conferences per year for the purpose of recruitment. Two are national meetings and one is a local chapter of a national association. The CMPH program utilizes recruitment tools at these events including: a school display, a marketing CD-ROM, and other free marketing giveaways provided by the Office of Student Services. The Assistant Director of Academic Programs (ADAP) usually staffs the booth at these functions. Presentation requests are made on behalf of various groups within and outside of the school of public health and the greater Emory University community. Additionally, a large portion of recruitment takes place via referrals by current students and alumni and distribution of other print materials. Prospective students may learn about the program through two campus visit days.

Once students are in the program they are required to participate in an online and on-campus orientation. The online orientation takes place several weeks before the on-campus orientation. It provides distance students with an opportunity to become familiar with the Rollins School of Public Health and the course management system; interact with other students prior to coming to campus; complete tutorials on various software packages; and gain an introduction to the world of public health via an online scavenger hunt. The on-campus orientation, a one-day event scheduled the day before the first face-to-face courses of the fall semester, offers an overview of campus and school resources, distance education and leadership development. Students are introduced to technology through a series of interactive one-hour sessions covering Blackboard (online course delivery software), OPUS (online registration system), e-mail and library resources (OVID).

Advisement occurs at many levels within the CMPH program. The ADAP advises students on their academic plan of study. Academic advisement occurs face-to-face at the on-campus session, electronically via email and the web, and via the telephone during the course of the semester. Additionally, students are assigned a faculty advisor for the special study project, which is the capstone experience of the program. Advisement also takes place prior to and during the students' course of study through the admissions, enrollment services, and career services branches within the office of student services. In particular, professional development advisement is of great interest to CMPH students.

Distance learning students are provided career services through traditional and non-traditional methods. The Office of Career Development offers the following services: career development workshops, resume review and career advisement, jobs listserv, Public Health Employment Connection (online database of job postings), Public Health Candidate Connection (database of student resumes for employers), Annual Career Day and other networking opportunities. The CMPH program schedules face-to-face workshops during the on-campus session for students who cannot easily attend on-campus functions. Additionally, CMPH has worked in conjunction with the Office of Career Development to develop an interactive Blackboard site that offers the opportunity for online career and continuing education services and informational lunch-n-learn sessions are scheduled during breaks of the on-campus session. Speakers from the public health community are invited to speak on topical issues such as AIDS, healthcare outcomes, and health services research.

Technology Services and Support

Technical support is an essential component of any distance education program. The CMPH program employs the Blackboard course management system. The system provides for an integrated school site for all online courses for students, which is password protected. It offers the technology to support web pages, PowerPoint and other files, assessments, discussion boards, file exchange areas, and communications. It offers a user-friendly control panel by which faculty and others with administrative access communicate with students and manage the course. It also offers a uniform format for all courses that helps students readily find course materials and access course tools such as email functions, discussion boards, and course materials. To facilitate an operational learning environment, the program offers online technical support, allowing students and faculty to email the information technology department's support staff about technical issues and receive a timely response.

Members of a multimedia team and instructional design team offer technical support for all classes. They assist in electronically publishing course documents on the web, converting documents into Acrobat format, and converting PowerPoint presentations to a suitable web-based format. They help faculty create multimedia web documents such as online guest presentations with PowerPoint slides and corresponding audio or CD-ROMs. They also suggest technological considerations for computer specifications and all teaching strategies for the program. In addition, other technologies are available to support the infrastructure of the CMPH program. Students may register for classes via an online registration system. Students, staff, and faculty are also members of listservs that help to build an online community and communicate program announcements.

Library and Learning Resources

The program provides library and learning resources for students to assist in their academic work. Students receive an overview of library resources during their first year orientation. They can then employ a password to access the library databases such as OVID to access Medline, ejournals, and other databases. They may also request that journal articles be copied for them for a minimal fee. Finally, they may have library consultations face-to-face or via email with a dedicated reference librarian to assist in their academic research.

Evaluation

As distance learning programs are new and have very different needs from a traditional program, the program should be well monitored and evaluated at all levels to improve and establish its credibility. Horton (2001) proposes four types of evaluation that can be applied: 1) response – liking of the course; 2) learning – skills and knowledge acquisition; 3) performance – applying knowledge and skills to the job; and 4) results – meeting of business goals. The CMPH program dedicates one staff member to coordinate and lead the evaluation procedures of the entire program. The program is currently working to revise the program logic model and establish additional evaluation methods to more completely address all aspects of the program. A general evaluation committee has been established and includes different program stakeholders, including students/alumni, instructional designers, faculty, and CMPH program staff. Each group of stakeholders has developed evaluation questions for their areas. These evaluation questions will be incorporated into current evaluation methods. Generally, the program evaluation include these surveys: 1) basic student course evaluations required for all courses in the school of public health, 2) student course evaluations designed to address issues specific to the distance learning environment, 3) instructors' evaluations of the program and support that they received, 4) design teams' (including instructional designers and graduate assistants) evaluations of the program and the instructor, 5) student evaluations of the entire program completed once a year, 6) student exit interviews about all aspects of the program around his/her graduation, and 7) program director evaluation of the faculty's performance. All of the evaluation data are presented and discussed within the CMPH committee to help enhance program operations.

The seven surveys listed above along with other program monitoring including, regular staff meetings, observing course interactions, and examinations of listservs by the staff, serve to evaluate the effectiveness of all components of the program. All of these evaluation activities help to address the four main evaluation questions of the program: to what extent 1) are the students able to apply the skills and knowledge from this program to their work setting?; 2) are the instructional design activities in the courses effective?; 3) are students satisfied with the program?; and 4) are sufficient student support services provided?

The web based exit survey is sent to all graduates of the program at the conclusion of the semester that they graduate the program. For the past year, nine of the twenty-one graduates responded (response rate of 43%). The survey results demonstrated a very high level of student satisfaction. One-hundred percent of respondents reported that they would rate the program very or somewhat worthwhile, that they were satisfied or very satisfied with the program, and that they would recommend or highly recommend the program to their colleagues. Eighty-nine percent of respondents agreed or strongly agreed that the CMPH program is relevant to the field of public health. Sample responses to the question “What were the strengths of the program?” included:

 “The strengths were the use of the technology and diversity and expertise of the faculty. The courses offered were also quite challenging and brought new perspectives to thinking about public health .”

 “Great advisors, good organization to the program, very career related.”

In addition, the results indicated that the program is very relevant to the students work in public health. One hundred percent o f respondents agreed or strongly agreed that they learned skills that were useful to their job and that they would be able to use what they learned as a tool to improve job performance. Sample responses to the question “How did what you learned in the CMPH Program help you on your job?” were:

 “ I was better able to envision health in its entirety and deal with all aspects of public health. I was able to apply many concepts to my current position and incorporate work projects into my school projects.”

 “In my opinion the CMPH Program focused strongly on team work and working with others, which is what is needed daily in our careers. I felt that the subjects we were charged to work with challenged each individual in the group to provide what they could as a public health expert in completing the assigned project.”

As mentioned, all components of the Program Development Model are addressed in our evaluation structure. Here is a description of how some of the components are evaluated and how the results are utilized. For the Technology Service and Support component, the following statements are included in each course evaluation; “sufficient technology support was available for this course,” “instructions to assist me in developing the necessary computer skills required for the course were sufficient,” and “in general, accessing the course website was easy.” Students are asked to respond to each statement on a scale from 1-5, strongly agree to strongly disagree. Over the course of the five years of the program, student and instructor responses to questions like these along with informal feedback has guided the CMPH decision to change course support systems twice and make the final decision to use the Blackboard course support program.

Similarly, the Curriculum and Instruction component is evaluated through a combination of the surveys mentioned above and more informal monitoring by the staff. For each course, students are asked to list what teaching strategies they found useful in terms of increasing knowledge, instructional designers are asked to report new teaching strategies they worked on with faculty members and how successful they were, and faculty are asked what innovative strategies they used in their courses. Some of the techniques that were collected include: group projects to promote collaborative learning, postings on reflections of course content on discussion boards, case study based discussion boards, capstone evaluation assignments like projects or posters, and journaling. When asked about CMPH course strategies on the annual fall 2004 student experience survey, 44.8% of respondents agreed or strongly agreed that group activities contributed to learning of the course material. In addition, 67.9% of respondents reported that Discussion Boards were the most successful instructional activities over readings, group work and chat rooms. When asked to rank various web based lecture styles, students mentioned PowerPoint (39.3%) followed by Text documents (25.0%), CD-ROMS (20.7%) and Learning units (13.8%) as the most effective. All of this feedback is used to inform the program about what pedagogical techniques have been successful and the program then shares this information with the faculty through listserv messages, one-on-one meetings and all faculty trainings.

Finally, the evaluation component itself is evaluated and the information collected is used to inform decisions about future evaluation activities. As new issues or questions arise with the changes in technology or student needs, the evaluation surveys are modified to collect the pertinent information. For example, with the advent of more enhanced e-journal and general library resources, additional questions have been posed to students in regards to the accessibility of these tools. As the evaluation results are reviewed for utility and as the evaluation committee meets, there is an ongoing discussion of how best to monitor all aspects of the program. This discussion has included the development of new tools, other methods of program monitoring including tracking of listservs and conducting focus groups on issues as they arise, and the best method to share data from the evaluation tools.

Conclusion

This article presents an overview of the Rollins School of Public Health's development of a web-based masters program. It presents a model framework for the development of a distance education program that has a central emphasis on supporting the students and faculty of the program. Table 3 presents a checklist for planning a distance education program that may be useful for administrators as they build an online program. Based on the five years of experience of the RSPH program and the literature review, there are three overarching themes that have emerged. First, a distance education program requires an infrastructure with multiple components to support the students, curriculum, and faculty. Second, keeping up with technological and educational trends is vital. This task involves modifying the marketing of the program to attract students, updating technologies and software to support new methods of learning, and revising the curriculum to meet the dynamic needs and competencies required of the public health workforce. Finally, well-established lines of communication are critical to success of a program. This includes linkages with the student services and instructional technology departments of the school, communication with other academic programs within the school to bolster support of the program, and exchange between the program, faculty, and students. Distance education provides a unique opportunity for the training of public health workers. To achieve that end, a distance program requires a commitment to systematic program planning including critical program components for supporting quality instruction.