Transcription

1 Session No. A407 Evaluating a Safety Degree Curriculum Using Job Analysis for Professional Safety Practice Background Roger L. Brauer, Ph.D., CSP, PE, CPE Executive Director Board of Certified Safety Professionals One of the accreditation requirements for certification boards is to ensure content validity for the examination(s) leading to the certification awarded. The certifying organization must demonstrate through recognized methodologies that the examination contents cover what people actually do in practice and the knowledge and skills required to be able to perform their work. In achieving accreditation through the standards of ANSI/ISO/IEC relating to certification of persons and the standards of the National Commission for Certifying Agencies, the Board of Certified Safety Professionals demonstrated that the contents of the examinations leading to the Certified Safety Professional (CSP ) designation cover what safety professionals do in practice. The approach used to demonstrate content validity for certification examinations is virtually identical to studies used to establish the contents of training and education programs for trades and professions. The approach is called job analysis or role delineation. The methodology outlines what people do (functions or domains and tasks or responsibilities) and the knowledge and skills required to perform each task. Completion of such studies is expensive, in part driven by the number of participants in the study. As a result, unless practice for a trade or profession changes rapidly, organizations perform such studies every few years only. The purpose of this article is to summarize the most recent job analysis study conducted for the Certified Safety Professional and to demonstrate how to use the results to evaluate an academic curriculum that prepares people for professional safety practice. Job Analysis Methodology In 1999, the Board of Certified Safety Professionals completed its most recent study of professional safety practice. The study 1 was conducted in three stages, following the currently recognized procedures used by certification and licensure boards. 1 Job Analysis Study for Certified Safety Professional Examinations, BCSP Technical Report , Board of Certified Safety Professionals, Savoy, IL, February 2001.

2 The first stage involved a panel of 16 people, each from a different job setting in professional safety. The study report provides details on panel members. A facilitator led the three-day procedure. The process began by defining the major functions (also called domains) of professional safety practice. The group then defined the tasks (also called responsibilities) that make up each job function. In each step, the group reached consensus on the wording of each function and task. In the final task, the panel defined the knowledge and skills essential for each task. This resulted in 192 knowledge and 147 skill statements that define professional safety practice. Among the knowledge and skill statements, there are approximately 130 unique knowledge statements and 123 unique skill statements. The second stage involved drafting a mail-out survey and pilot testing it. The purpose of the survey was to validate the work of the panel and to acquire ratings of each function and task on three rating scales. The purpose of the pilot study was to gain feedback on the survey content and format in order to refine the instrument and to identify any gaps in the definitions of practice. The pilot survey went to approximately 300 safety professionals in practice. In the third stage of the job analysis study, about 3000 safety professionals received the survey instrument. About 1500 completed and returned the survey. All survey recipients held the Certified Safety Professional designation. The survey asked respondents to rate the safety professional job functions and tasks on three scales. One rating was the importance of each function and task in the respondent s own practice. The second rating was the time spent in individual practice on each function and task. The third rating involved criticality in terms of protecting the public should someone fail to perform the function or task effectively. Job Analysis Results The survey results were compiled and analyzed. One result was the verification of the description of functions and tasks of practice. Another result was estimating the portion of the examination that should be devoted to each function and task. The estimates are derived from the combination of the three ratings provided by each respondent. The study report details the computational procedure. The procedure establishes the recommended weighting for each function. The complete outline of functions, tasks, knowledge and skills with the final weighting (in percent of the examination contents) for each function is called the examination blueprint (see Appendix A). Converting Job Analysis Results to Knowledge and Skill Outlines When considering what knowledge one needs for practice, it is useful to organize the information into subject groups, rather than organizing the information around the functions and tasks of practice. After reviewing knowledge and skill statements, the author created a subject classification scheme to complete this stage of analysis. There is not likely to be full agreement on subject group titles or on which knowledge and skill statements should be assigned to a group. The overall goal was to create a reasonable scheme for the purposes of this paper. There are two major groups of subjects for classifying knowledge statements. The first involves general subjects and the second involves more specific knowledge of particular safety subjects. Within each Page 2 of 35

3 major group, knowledge statements are assigned to specific subjects. Appendix B is a complete listing of knowledge statements organized by group and subject. The skills from the job analysis study of professional safety practice were analyzed in a similar manner and classified into major skill groups. Appendix C lists the skills by skill group. Results of the Conversion Activity One can estimate the importance and relevance of a subject in professional safety practice from the number of knowledge statements falling under a specific subject. The original study identified 192 knowledge statements. Some knowledge statements occur more than once. The weighting for each subject assumes that each knowledge statement represents approximately ½ percent (100%/192 statements= 0.52) of all of the knowledge covered by practice in general. The number of knowledge statements occurring within a subject is then an estimate of the importance and relevance of the subject across all of practice. For example, if a subject contained 20 knowledge statements, including repeated knowledge statements, then 10 percent of the training for professional safety practice might be devoted to the subject. This provides a general indication of the importance of the subject for professional safety practice. Some subjects may also be important in specific areas of practice and be under-estimated or over-estimated by this approach. Table 1 is a list of the group names, subjects and percent of practice weightings. Table 1. Safety Knowledge Statement Groups and Subjects Number of Percent Number of Different of Group/Subject Statements Statements Practice GENERAL A. Laws, Regulations, Standards, & Government B. Mathematics, Statistics, Physical & Chemical Sciences C. Technology (Engineering, Facilities, Equipment, Processes, Computers & Systems D. Behavioral and Organizational Science E. Business and Management F. Training and Education G. Communication H. Security SAFETY, HEALTH, ENVIRONMENT AND ERGONOMICS A. General B. SHE in Design, Controls, Technology C. Risk Management and Insurance D. Safety Management E. Inspections, Investigations, Audits F. Fire Protection G. Industrial Hygiene H. Ergonomics I. Environmental J. Emergencies & Emergency Response K. Professional Development and Ethics L. Applied SHE - Construction M. Applied SHE - Transportation N. Applied SHE - Product Safety O. Applied SHE - Process/Petro-Chemical P. Applied SHE - Systems Page 3 of 35

4 Figure 1 shows the distribution of knowledge statements based on weighting among subjects within the first group, General Subjects. Figure 2 shows the distribution among the specific Safety, Health and Environmental Subjects. Communication Train/Educ Busi/Mgmt Behav/Org Sci Technology Math/Science Legal/Govt Knowledge - General Security Percent Figure 1. Distributuion of General Knowledge Statements Knowledge - SHE & Ergo Systems Process/Petro-Chemical Product Safety Transportation Construction Prof Devel/Ethics Emergency/Response Environmental Ergonomics Industrial Hygiene Fire Protection Inspect/Invest/Audit Safety Management Risk Mgmt/Insurance SHE in Design/Tech General Percent Figure 2. Distribution of Specific SH&E Knowledge Statements The importance of each skill group was estimated in a manner similar to that used for knowledge statements. Since the job study identified 147 skill statements, including repeated skills across functions and tasks, one can estimate the value of each skill statement at approximately 0.68 percent (100%/147 = 0.68). The total value for a skills grouping is the number of skills times 0.68 percent. Table 2 identifies the skill groups and the weighting for each group. Figure 3 shows the distribution of skill statements among the skill groups based on group weighting. Table 2. Safety Number of Percent Number of Different of Group/Subject Statements Statements Practice A. General and Applied Interpreting B. Ethics Application C. General and Applied Mathematical, Analytical and Scientific D. Creative E. General and Applied Verbal Communication F. General and Applied Written Communication G. General and Applied Training H. General and Applied Computer and System I. Applied Safety and Health J. Inspection and Investigative K. General and Applied Human Behavior L. General and Applied Business, Organization and Leadership Page 4 of 35

5 Business/Leader Behavior Inspect/Invest Applied S&H Computer Training Written Verbal Creative Math/Anal/Sci Ethics Appl Interpreting Safety Evaluating a Curriculum Most certification and licensure examinations test knowledge and not skills. However, academic programs help students acquire both knowledge and skills. One can use the job analysis data to identify whether particular courses cover various knowledge and skills. The evaluation may cover only the courses in a safety degree major or all courses included in a degree program. Percent One approach for applying the job analysis data to Figure 3. Distribution of Skill Statements evaluate a curriculum is to determine whether the curriculum covers the breadth of knowledge identified in the study. The procedure begins by creating one or more matrices. The vertical axis of a matrix is a list of courses by title and number. The list may include only the courses in the major or all courses in the degree program. The horizontal axis includes a column for each knowledge group or subject. Because the matrix becomes large if there is a column for all subjects in both knowledge group, it may be easier to handle the evaluation in parts with a separate matrix for each group. Once the matrix is established, someone with knowledge of the contents of each course can mark each cell in the matrix to indicate whether a course covers a knowledge group/subject at all. One may want to refer to the course syllabus in deciding how to mark the cell. Gaps in the matrix suggest where knowledge needed for practice is not covered. A more detailed analysis may mark each matrix cell in terms of the amount of time a course devotes to a subject. In addition, a rating scheme might establish basic quantitative estimates of coverage. A rating scale might include ratings of not at all, a little, moderately, or a great deal. A rating scheme might also be based on level of difficulty, such as not at all, elementary, intermediate, or advanced. Gaps or marks indicating limited coverage may identify opportunities for improvement. A third approach is to evaluate specific courses in depth. In this case, the evaluation issue is whether a course covers specific knowledge statements. One can estimate whether a course covers each individual knowledge statement relevant and important for the course. Table 3 provides an outline for this type of evaluation. Table 3. Analysis of a Specific Course for Coverage of Knowledge Statements Page 5 of 35

10 D3R7K8 Data analysis and presentation D4R1K4 Formal and informal presentation techniques Suppose there is a course that include the subject: Inspections, Investigations and Audits. One could review each knowledge statement in this subject (see Appendix B) to see if the five knowledge statements are adequately covered. If not, the analysis will suggest content improvements for the course. In another example, a course may cover communication in the General knowledge group. One could rate the course by indicating that it covers communication knowledge a lot, but only at an elementary level. Further study of results might indicate that this course should be covering more advanced communication knowledge to complement the elementary level already covered in other courses. One can also use the job analysis data to identify whether skills are developed in the entire program of study. Again, a general analysis or a precise analysis can reveal opportunities for improvement. The general analysis will identify whether skill groups are developed in particular courses. The precise analysis will identify whether specific skills are covered in a logical order during the entire educational program or within specific courses. One might rate a course on both knowledge and skills. The example above relating to communication knowledge could be extended to communication skills. The analysis for skills may indicating that the course provides a limited opportunity to develop writing skills, but the skill development activities occur Page 10 of 35

11 at an advanced level and nicely complement other courses that provide greater opportunity for writing skills at an elementary or intermediate level. Whether one completes an evaluation at a general or detailed level, the results of the evaluation can identify potential improvements in specific courses or across the program of study for students entering or advancing in professional safety practice. Summary The job analysis studies conducted by BCSP to establish the examination blueprints for the Safety Fundamentals and Comprehensive Practice examinations that lead to the Certified Safety Professional designation are valuable resources for safety degree programs. The knowledge and skills defined in the job analysis studies are essential for effective professional safety practice. Schools offering a degree or courses in safety practice can compare the course contents to the knowledge and skills and identify whether the course and the entire program prepare students for effective professional safety practice. Appendix A. The CSP Examination Blueprints (Reprinted with permission of the Board of Certified Safety Professionals.) Domain 1. Safety, Health and Environmental Management (SAFETY FUNDAMENTALS - 37%! COMPREHENSIVE PRACTICE - 34%) Responsibility 1. Design comprehensive management systems by defining requirements and developing policies, procedures and programs to protect people, property and the environment. Page 11 of 35

25 Responsibility 2. Adhere to standards of professional conduct by limiting practice to areas of competence and avoiding conflicts of interest to minimize the potential for harm. Knowledge D4R2K1 BCSP Code of Professional Conduct D4R2K2 General business ethics D4R2K3 Conflict resolution techniques D4R2K4 Personal and professional limitations D4R2K5 Methods of facilitating teamwork D4R2K6 Competencies of other technical professionals with whom the safety professional interacts D4R2K7 Consequences of professional errors or omissions D4R2K8 Elements of a conflict of interest policy D4R2K9 Laws relating to conflict of interest D4R2S1 Applying BCSP Code of Professional Conduct D4R2S2 Applying team building and interpersonal techniques D4R2S3 Resolving conflicts through negotiation D4R2S4 Selecting consultants and outside resources and providing adequate support Page 25 of 35

33 Product safety [2] Applied SHE - Process/Petro-Chemical Process management, material flow and procurement Process safety [2] Applied SHE - Systems System safety [2] System and occupational safety Appendix C. Outline of Required for Professional Safety Practice ([xx] is number of times a skill statements appears in the blueprints.) General and Applied Interpreting Interpreting and applying regulations Interpreting and applying safety, health and environmental science data for process improvement Interpreting and applying laws, standards and regulations Interpreting plans, specifications and drawings [2] Interpreting laws, standards and regulations [6] Interpreting analytical results Interpreting data [2] Interpreting regulations to ensure a compliant and effective system Ethics Application Applying BCSP Code of Professional Conduct [7] General and Applied Mathematical, Analytical and Scientific Mathematical and statistical analysis [5] Sampling and making observations Utilizing basic science to explain safety, health and environmental issues Calculating accident and incident rates [2] Analyzing sampling results and other data to support decision making and prioritizing control recommendations Problem solving Creative Designing safe work practices for systems, facilities and equipment Managing safety through the design processes General and Applied Verbal Communication Interpersonal communication [3] Using interpersonal communication Communication and presentation Working with unions and management Consulting with subject matter experts [4] Consulting with equipment manufacturers/suppliers and construction contractors to assure safety control compliance Consulting with subject matter experts Meeting with federal, state and local officials Providing input during standards development Negotiating with political entities Negotiating compliance issues with government and other entities or affected parties Using presentation technology: hardware and software [2] Making effective graphics Communicating effectively using verbal and nonverbal skills [3] Using graphics, illustrations and other media Using teleconferencing, and other electronic media Interviewing people Integrating cultural norms into communications Communicating risks that are present and appropriate control measures to management. Applying risk communication strategies Delivering the information in the language and media appropriate for the audience General and Applied Written Communication Page 33 of 35

Competency Requirements for Executive Director Candidates There are nine (9) domains of competency for association executives, based on research conducted by the American Society for Association Executives

Career Paths in Safety The primary focus of safety practice is to prevent incidents and accidents that may lead to injuries, illnesses, damage to property and equipment, or harm to the environment. Many

Corporate Training Occupational Safety, Health, and Environmental Management Certificate Program extension.uci.edu/corporate Safety and health professionals play an important role in maintaining the quality

PMP Examination Tasks Puzzle game Here is a great game to play to test your knowledge of the tasks you will be tested on in the actual examination. What we have done is take each of the domain tasks in

ISO 14001:2015 Transition Information Commitment & Services Our Values When it comes to our people and the way we work, we possess seven core values. We demand the highest ethics and integrity from our

Career Service Authority IT Project Manager III Page 1 of 6 GENERAL STATEMENT OF CLASS DUTIES Performs advanced, specialized, professional project management work on one or more medium to large moderately

Master s in Educational Leadership Ed.S. in Administration and Supervision Austin Peay State University Professional Educational Standards, TILS Standards, and NCATE Standards Austin Peay State University

GLOSSARY Assessment - the evaluation process used to measure the performance or effectiveness of a system and its elements. As used here, assessment is an all-inclusive term used to denote any of the following:

Health Systems Management Courses *HSM 200. US Healthcare Organization and Delivery. Credit 3 hours. This course is an overview of how health care is organized, delivered and financed in the United States.

Content Area Standard Strand 21st-Century Life & Careers 9.4 Career and Technical Education: All students who complete a career and technical education program will acquire academic and technical skills

The following Cluster (Foundation) Knowledge and Skill Chart provides statements that apply to all careers in the Cluster. Persons preparing for careers in the Cluster should be able to demonstrate these

ADVANCED HIGHER EDUCATION ADMINISTRATION PROGRAM The Advanced Higher Education Administration Program is designed for international university administrators to enhance their knowledge and understanding

Human Resources Management Program Standard The approved program standard for Human Resources Management program of instruction leading to an Ontario College Graduate Certificate delivered by Ontario Colleges

OCCUPATIONAL GROUP: Information Technology CLASS FAMILY: Security CLASS FAMILY DESCRIPTION: This family of positions provides security and monitoring for the transmission of information in voice, data,

Page 1 AD-AUDIT BRANCH MANAGER CHARACTERISTICS OF WORK: This position involves professional accounting and auditing work at an administrative level within the Department of Audit. Incumbent reviews all

S24900B, page 1 Nothing in this job description restricts management's right to assign or reassign duties and responsibilities to this job at any time. FUNCTIONAL DUTIES Serves as the Safety Program Manager

Revised Body of Knowledge And Required Professional Capabilities (RPCs) PROFESSIONAL PRACTICE Strategic contribution to organizational success RPC:1 Contributes to the development of the organization s

Red River College Course Learning Outcome Alignment with BABOK Version 2 This alignment chart was designed specifically for the use of Red River College. These alignments have not been verified or endorsed

Position Description Cover Sheet In order to make an objective and accurate evaluation of a position, it is very important that the position description (PD) contain specific data. Therefore, please provide

OCCUPATIONAL GROUP: Inspection and Safety CLASS FAMILY: Safety and Loss Control CLASS FAMILY DESCRIPTION: This family of positions includes those whose purpose is to perform inspections on various entities

Page 1 of 11 REVISED JOB TITLE: REPORTS TO: SUPERVISES: DIRECTOR OF EARLY CHILDHOOD EDUCATION Assistant Superintendent for Curriculum and Instruction Students and staff assigned to the Early Childhood,

Faculty of Health Sciences Graduate Public Health Program Master of Public Health (MPH) Health Promotion and Community Health () Learning Objectives mapped to Competencies May 2012 Each MPH concentration

IAAP Certified Administrative Professional (CAP) Exam 2016 Body of Knowledge The IAAP Certified Administrative Professional (CAP ) is an assessment-based professional certification for office management

Fire Alarm Systems Certification The skills and knowledge listed under each task are suggestive of those involved in that task, but are not intended to constitute an exhaustive listing. 2.1 Submittal Preparation

Adopted by state board of education of ohio October, Ohio Standards for School Counselors Ohio Standards for School Counselors ii Contents Section I: Overview of the Ohio Standards for School Counselors...

Position Description February 9, 2015 Position Description February 9, 2015 Page i Table of Contents General Characteristics... 2 Career Path... 3 Explanation of Proficiency Level Definitions... 8 Summary

Career Service Authority Senior Human Resources Professional Page 1 of 8 GENERAL STATEMENT OF CLASS DUTIES Performs full performance level professional, technical, and administrative human resources work

Student Name: Date: PROGRAM COMPETENCY PROFILE FOR CAREER TECHNICAL EDUCATION NH Frameworks (statement that provides (what a student needs to know and be able to www.ed.state.nh.us/frameworks Understand

Health, Safety, Environment and Corporate Social Responsibility 15 Guiding Principles 2013 Message from President and CEO 0 New Gold has a complementary portfolio of assets in different countries and cultures

Master of Arts (Industrial and Organizational Psychology) M.A. (Industrial and Organizational Psychology) Objectives The program aims to enable graduate students to analyze, to carry on the research projects,

COURSE DESCRIPTIONS IN MANAGEMENT MIS 241. Introduction to Management Information Systems Credit 3(3-0) This course is an introduction to management information systems and their role in supporting and

Implementation of a Quality Management System for Aeronautical Information Services -1- Implementation of a Quality Management System for Aeronautical Information Services Chapter IV, Quality Management

Crosswalk: Core Competencies for Public Health Professionals & the Essential Public Health Services Introduction View crosswalks of the Tier, Tier and Tier Core Competencies for Public Health Professionals

CRITERIA FOR ACCREDITING ENGINEERING TECHNOLOGY PROGRAMS Effective for Reviews During the 2015-2016 Accreditation Cycle Incorporates all changes approved by the ABET Board of Directors as of November 1,

Analyst Position Description September 4, 2015 Analysis Position Description September 4, 2015 Page i Table of Contents General Characteristics... 1 Career Path... 2 Explanation of Proficiency Level Definitions...

About The IIA Global Internal Audit Competency Framework The IIA Global Internal Audit Competency Framework (the Framework) is a tool that defines the competencies needed to meet the requirements of the

Page 1 of 12 SCHOOL NURSE COMPETENCIES SELF-EVALUATION TOOL School Nurse School Date Completed School Nurse Supervisor Date Reviewed The school nurse competencies presume that some core knowledge has been