Institutional pass rates are used in three components of the accreditation system: as a precondition to accreditation, as a condition for meeting Standard 1 of the NCATE Unit Standards, and as a condition of national recognition in the program review process. This document identifies NCATE policies related to pass rates in these areas.

History

At its October 2002 meeting, the Unit Accreditation Board adopted three motions on the use of pass rates in the accreditation process, the Specialty Areas Studies Board adopted one motion on the use of pass rates in determining national recognition, and the State Partnership Board (SPB) adopted one motion on the use of pass rates when states recommend national recognition. These five motions were approved by the Executive Board at its October 2002 meeting. Unit heads, NCATE coordinators, state partners, board members, and BOE members were notified of the board actions via an email following the October 2002 board meetings. The summary of these actions is currently available in the archives section of the NCATE website.

At its March 2003 meeting, the Unit Accreditation Board modified the language of the two rubrics on the knowledge of content for Standard 1 to align it with earlier Board action. The Board also accepted a statement to guide the implementation of the new rubrics requiring an 80 percent pass rate on content tests.[1] The UAB actions and a summary of NCATE requirements related to the use of test scores were sent via email to unit heads, NCATE coordinators, state partners, board members, and BOE members before the end of March 2003. The summary of UAB actions from this meeting can be accessed on the home page of NCATE’s website.

Following the Executive Board meeting in May 2003, emails with specific details about the use of pass rates in the NCATE system were sent to institutions with visits in 2003-2004 and 2004-2005. Emails were sent to all precandidates with an explanation of the revised Precondition #7. The standards and preconditions on the NCATE website were changed to reflect the new language and procedures. A Q&A section on the use of tests and pass rates was added to the website in July 2003.

In October 2003, the UAB revised policies related to the use of testing in the NCATE system. The UAB clarified the type of condition and provision that would be applied to an institution that no longer meets the state pass rate as required in Precondition #7. The policy was revised to require the submission of written documentation to show compliance with the state pass rate within two years. The revision is included in the policy statement below and outlined in Appendix A.

In October 2003, the UAB also revised its policy related to the “80 percent rule” in Standard 1. The last paragraph of the “Guidance for the Implementation of the Rubrics with Test Scores” was revised to allow Board of Examiners teams to look for an 80 percent pass rate on content examinations during the year prior to the visit or aggregated data over the period since the last visit. The revised guidance is included in Appendix C.

Precondition #7

Precondition #7 now reads:

In states with a program approval process, the unit’s programs are approved by the appropriate state agency or agencies. In states with candidate licensing examinations and required pass rates, the unit’s summary pass rate meets or exceeds the required state pass rate. This provision does not apply to units in states without examination requirements or required pass rates for licensure.

The following procedure was adopted to guide the implementation of the new Precondition #7:

The Annual Report and Preconditions Audit (ARPA) Committee will review a unit’s summary pass rate to ensure that each unit is meeting or exceeding state required pass rates on a yearly basis. If an institution falls below the state required pass rate, it will have to write a report to be reviewed by the ARPA Committee. The ARPA Committee will consider the report and make a recommendation to the UAB. The ARPA Committee can recommend (1) that no further action be taken; or (2) that the UAB review the institution’s accreditation status.

The recommendation of the ARPA Committee Should Be Based On:

Supplemental state licensing examination data submitted by the institution demonstrating that the state required pass rate is being met

The strength of the institution’s arguments regarding the accuracy of the data reviewed

When the ARPA Committee recommends a review of the institution’s accreditation status, the UAB can take several actions. If the institution is accredited without qualification, and does not submit compelling evidence that the unit is meeting the state-required pass rate, then the UAB shall change the institution’s status to conditional accreditation. If the institution is already conditionally approved, and does not submit compelling evidence that the unit is meeting the state-required pass rate, then the UAB shall place the institution on probation. If the institution is provisionally approved, and does not submit compelling evidence that the unit is meeting the state-required pass rate, then the UAB shall place the institution on probation. If the institution is already on probation, and does not submit compelling evidence that the unit is meeting the state-required pass rate, then the institution will remain on probation. The condition and provision decisions, in these cases, require institutions to submit written documentation within a period not to exceed two years. The probationary decisions, in these cases, require institutions to submit written documentation within the previously established probationary period.

If the institution submits compelling evidence that it is meeting the state-required pass rates, then the UAB shall require no further action.

This policy became effective in fall 2003.

Timeline for Implementation

All current and future precandidates for accreditation must meet the revised Precondition #7 before an on-site visit is conducted. The test scores reported for Title II by all institution candidates and accredited institutions will be regularly monitored by NCATE to determine if those institutions continue to meet Precondition #7.

Steps for Monitoring Compliance with Precondition #7

NCATE will track the states that have established required pass rates for program approval, identify the tests to which the states apply the pass rates, and clarify the effective date for implementation of the pass rates. For example, since 2001, Maryland has required an 80 percent pass rate for institutions on state licensing exams; the pass rates include the combined Praxis 1 and Praxis 2 scores.

NCATE will review the most recent Title II data after it is updated on the Title II website.

For each institution in a state that has a required pass rate, the required pass rate will be compared to the summary pass rate[2] for the institution to determine the institutions that have summary pass rates below the pass rates required by the state.

Following this review of pass rates reported in the Title II data, NCATE’s president will send a letter to the unit heads of institutions identified as not meeting state required pass rates. The letter will ask the institutions to submit a report to the ARPA committee for consideration at its next meeting. The ARPA committee will consider the report and make a recommendation to the UAB following the procedure above.

If the ARPA committee recommends a review of the institution’s accreditation status, the UAB can take several actions as described in the procedures above.

Standard 1

A sentence has been added to each level of the rubric for the first two elements on content knowledge to require that “80 percent or more of the unit’s program completers pass the state licensing examinations in states that require examinations for licensure.” The revised rubrics and related policy and implementation guidance can be found in Appendices B, C, and D.

Timeline for Implementation

The revised Standard 1 rubrics became effective for visits in fall 2003.

Implementation Plan

The policy and guidance for the application of the revised rubrics are described in Appendices C and D.

National Recognition of Programs

At its October 2002 meeting, the Specialty Areas Studies Board (SASB) adopted the following policy related to the use of test scores in determining national recognition of programs:

Eighty percent of a program’s completers (as defined by Title II) must pass the content examinations in the given areas of specialization in states where such examinations exist in order for a program to gain or retain full national recognition. This requirement does not apply to units in states that do not have content examination requirements for licensure.

The SPB also took action to require those partnership states that conduct program reviews and have the authority to recommend national recognition for institutions’ programs must enforce the “80% rule” — that is 80 percent of the programs’ completers must pass the state’s licensing examination. The final Board action is as follows:

All states approved by the SASB to recommend national recognition of programs must include a requirement that 80 percent of the program’s completers pass the content examinations in the given areas of specialization in states where such examinations exist in order for a program to gain or retain national recognition. This requirement does not apply to states that are not approved to grant national recognition.

Programs that do not meet the 80 percent pass rate cannot be fully nationally recognized.

Programs that meet or exceed the 80 percent pass rate must meet the other elements in the standards in order to be recognized. Multiple measures, including data summarizing performance, will be required to show that candidates gain the knowledge, skills, and dispositions necessary to help all students learn, and in the case of non-classroom personnel, to create an environment that helps all students learn.

In other words, programs are expected, at a minimum, to meet or exceed the 80 percent pass rate as a necessary but not sufficient condition of recognition.

This policy will become effective for programs submitted for review in Fall 2003 and beyond.

Timeline for Implementation

Pass rates for a program must be submitted with program reports due on September 15, 2003 and afterwards.

Endnotes

1. Contents tests include state licensing exams in academic content areas such as English, mathematics, French, and music, as well as exams in early childhood education, special education, and elementary education. In addition, contents tests include state licensing exams for other professional school personnel such as school counselors and administrators.

2. Summary Pass Rate is defined in Title II as the proportion of program completers who passed all tests they took for their area of specialization among those who took one or more tests in their specialization areas.

Appendx A

UAB Actions and Responses related to Precondition #7

Institutional Status

Result of the UAB’s Document Review

UAB Action

Unit Response

UAB Finding in Response to the Unit’s Submission

UAB Action

Accredited

Unit Meets Precondition 7

No further Action by the UAB

Unit Does Not Meet Precondition 7

Accreditation Status is Changed to Conditions

Unit Must Submit Documentation Within Two Years

Unit Meets Precondition 7

Condition Related to Precondition 7 is Removed

Unit Does Not Meet Precondition 7

Accreditation is Revoked

Accredited with Conditions

Unit Meets Precondition 7

No Further Action by the UAB

Unit Does Not Meet Precondition 7

Accreditation Status is Changed to Probation

Unit Must Submit Documentation Within a Period Not to Exceed Two Years

Unit Meets Precondition 7

Probation Related to Precondition 7 is Removed; Conditions May COntinue

Unit Does Not Meet Precondition 7

Accredited with Provisions

Unit Meets Precondition 7

No Further Action by the UAB

Unit Does Not Meet Precondition 7

Accreditation Status is Changed to Probation

Unit Must Submit Documentation Within a Period Not to Exceed Two Years

Unit Meets Precondition 7

Probation Related to Precondition 7 is Removed; Provisions May Continue

Unit Does Not Meet Precondition 7

Accreditation is Revoked

Accredited with Probation

Unit Meets Precondition 7

No Further Action by the UAB

Unit Meets Precondition 7

Unit Does Not Meet Precondition 7

Accreditation Status Remains Probation

Unit Must Submit Documentation Within the Established Probationary Period

Teacher candidates have inadequate knowledge of subject matter that they plan to teach and are unable to give examples of important principles or concepts delineated in professional, state, and institutional standards. Fewer than 80 percent of the unit’s program completers pass the academic content examinations in states that require such examinations for licensure.

Teacher candidates know the subject matter that they plan to teach and can explain important principles and concepts delineated in professional, state, and institutional standards. Eighty percent or more of the unit’s program completers pass the academic content examinations in states that require such examinations for licensure.

Teacher candidates have in-depth knowledge of the subject matter that they plan to teach as described in professional, state, and institutional standards. They demonstrate their knowledge through inquiry, critical analysis, and synthesis of the subject. All program completers pass the academic content area examinations in states that require such examinations for licensure.

Content Knowledge for Other Professional School Personnel

Unacceptable

Acceptable

Target

Candidates for other professional school roles have an inadequate understanding of their field and cannot give examples of important principles or concepts delineated in professional, state, and institutional standards. Fewer than 80 percent of the unit’s program completers pass the content examinations in states that require such examinations for licensure.

Candidates for other professional school roles know their fields and can explain principles and concepts delineated in professional, state, and institutional standards. Eighty percent or more of the unit’s program completers pass the content examinations in states that require such examinations for licensure.

Candidates for other professional school roles have a thorough understanding of the central concepts, tools of inquiry, and structures of their fields as delineated in professional, state, and institutional standards and shown through inquiry, critical analysis, and synthesis. All program completers pass the content examinations in states that require such examinations for licensure.

Appendix C

Use of State Licensure Test Results in Standard 1

At the October 2002 meeting, the Unit Accreditation Board adopted the following statement about the use of state licensure test results in determining whether Standard 1 is met:

NCATE’s interim national benchmark for evaluating candidates’ acquisition of content knowledge in the unit review process will require that 80 percent of the unit’s program completers pass the state licensure examinations in their content areas.

Units in which 80 percent of the program completers do not pass their state licensure test in the content area will not meet the content knowledge element in Standard 1 at the acceptable level. A unit cannot meet Standard 1 without meeting the content knowledge element.

Units that meet or exceed the 80 percent unit content pass rate must meet the other elements in Standard 1 in order to meet the standard. Multiple measures, including data summarizing performance, will be required to show that candidates gain the knowledge, skills, and dispositions necessary to help all students learn, or in the case of non-classroom personnel, to create an environment that helps all students learn.

In other words, units are expected, at a minimum, to meet or exceed the 80 percent unit content pass rate as a necessary but not sufficient condition to meet Standard 1.

The BOE must make a recommendation about whether each standard is met. If the BOE finds that the unit meets or exceeds the 80 percent unit content pass rate and if the unit also meets most of the other elements of Standard 1, then the BOE should recommend that the standard is met.

If the BOE finds that the unit does not meet the 80 percent unit content pass rate, then the element on content knowledge is not met at the acceptable level, and regardless of whether the unit meets the other elements of the standard, the BOE must recommend that the standard is not met.

The 80 percent unit content pass rate does not apply to units in states that do not have academic content examination requirements for licensure. However, the units in these states must provide other performance evidence to convince the team that candidates and program completers have developed the content knowledge expected in professional, state, and institutional standards for their fields.

Program Review documents will continue to play an important role in the BOE's determination of the extent to which the unit is meeting this standard.

This policy went into effect for visits in fall 2003.

Appendix D

Guidance for the Implementation of the Rubrics with Test Scores

At its March 2003 meeting, the UAB adopted the following guidance for reviewing the elements on content knowledge in NCATE Unit Standard 1. This guidance is reflected in BOE training, Q&As on the website, the BOE Update, and other NCATE publications.

Institutions should present state licensure test results reported to the state since the institution’s previous NCATE visit, or in the case of institutions undergoing their first accreditation review, since the time when the unit achieved candidacy or two years, whichever is longer. In instances in which the tests have not been offered for three years, institutions should present data reported to the state since the tests have been available. If in the most current year, an institution has fewer than 10 scores to report, then the institution should present aggregated data over the period since its last NCATE visit or (for new institutions) since achieving candidacy or two years, whichever is longer.

BOE teams will begin their reviews with the results of the most recent reporting cycle. In the case of institutions with fewer than 10 program completers, the BOE teams will begin their reviews with the aggregated data over the period since the last NCATE visit or since the institution achieved candidacy or two years, whichever is longer. If 80 percent of the program completers across the unit do not pass the exam in the most recent annual reporting period, the content knowledge element of Standard 1 cannot be met at the acceptable level and the team should recommend that Standard 1 is not met. In institutions with fewer than 10 completers in a given year, if 80 percent of the program completers across the unit over the period since the unit’s last NCATE review (or since achieving candidacy or two years, whichever is longer) do not pass the exam, then the content knowledge element of Standard 1 cannot be met at the acceptable level and the team should recommend that Standard 1 is not met.

An 80 percent pass rate is one measure of content knowledge; institutions must present multiple measures to demonstrate that candidates have gained the content knowledge necessary to help all students learn. This evidence should come in the form of aggregated data from other content area assessments. These data can be derived from the following types of assessments:

Comprehensive exams, perhaps administered by another department on campus and given as a requirement for a major

Exams developed by and/or for the unit related to content knowledge

Essays that require candidates to demonstrate an understanding of relationships among elements of knowledge

Oral exams related to content knowledge

Semester projects

Ratings by cooperating teachers

Major or GPA in the subject area

The unit must make the case in writing that (1) the assessments are sufficiently aligned with the professional standards on content knowledge; (2) the assessments are broad enough in scope to sample mastery of the domain of knowledge being assessed; (3) the criteria for success are clearly specified; and (4) most of the candidates have completed the assessments successfully. The professional judgment of the BOE and UAB will be used to determine if the institution has successfully demonstrated alignment, accuracy, and candidate success.

The use of a major or GPA in the subject area is a common source of evidence related to content knowledge. However, for this evidence to be convincing, the unit must make the case that (1) the curriculum in the major is aligned with the professional standards on content knowledge; (2) candidates are assessed using aligned instruments; (3) grading standards are clearly specified; and (4) grading is consistent across multiple sections of the same course.

In 2004-2005, when the standards are expected to be fully implemented and state licensure exams are better aligned with standards, the BOE teams will be instructed to examine both current and trend data of state licensing test score results over the period since the institution’s last NCATE visit or, for new institutions, since the unit achieved candidacy or two years, whichever is longer. The trend data should show a pattern of program completers passing the state licensure tests. If an institution does not have an aggregate pass rate of 80 percent in the previous year or aggregated over the period since the last visit, then the element of the standard on content knowledge will not be met at the acceptable level and the team should recommend that Standard 1 is not met. In addition to test data, the unit must present evidence as discussed above to demonstrate that the standard is being met.*