B. Components of a Comprehensive School Reform Program

A comprehensive school reform program is one that addresses each of the following eleven components in a comprehensive and integrated design:

Proven methods and strategies based on scientifically based research - A comprehensive school reform program employs proven strategies and methods for student learning, teaching, and school management that are based on scientifically based research and effective practices and have been replicated successfully in schools.

Component one emphasizes the need for schools, in designing their comprehensive reform program, to employ proven strategies and methods that are grounded in scientifically based research in core academic subjects, especially mathematics and reading. Staying focused on academic achievement, building a comprehensive program that supports it, and emphasizing "what works" in the classroom are important elements of a successful comprehensive design.

Comprehensive design - A comprehensive design for effective school functioning integrates instruction, assessment, classroom management, professional development, parental involvement, and school management. By addressing needs identified through a school needs assessment, it aligns the school's curriculum, technology, and professional development into a plan for schoolwide change. The ultimate goal of this design is to enable all students to meet challenging State content and student academic achievement standards.

Professional development - The program provides high-quality and continuous teacher and staff professional development and training. The professional development involves proven, innovative strategies that are both cost effective and easily accessible and ensures that teachers are able to use State assessments and challenging State academic content standards to improve instructional practice and student academic achievement.

Well-designed professional development activities increase all teachers' knowledge of both the academic subjects they teach and effective instructional strategies that are grounded in scientifically based research. They include strategies such as partnerships with institutions of higher education and address such topics as the use of data and assessment; the use of technology; and improving the instruction of special needs children.

This professional development is intensive, sustained over time and classroom focused. Those who participate in professional development also help to design it, and the design is well integrated with school and district educational improvement plans. The professional development component is regularly evaluated to gauge its impact on increased teacher effectiveness and improved achievement. Strategies employed are consistent with high- quality professional development as described in Title II, Part A of the ESEA.

Measurable goals and benchmarks - A comprehensive school reform program includes measurable goals for student academic achievement and establishes benchmarks for meeting those goals. The Department encourages LEAs to link these goals to the State's definition of adequate yearly progress (AYP) in Section 1111(b)(2) of the ESEA.

Support within the school - Teachers, principals, administrators, and other staff throughout the school support the program in a CSR school. They demonstrate this support by, among other activities, understanding and embracing the school's comprehensive reform program, focusing on continuous improvement of classroom instruction, and participating in professional development.

Support for teachers and principals - A CSR program provides support for teachers, principals, administrators, and other school staff by creating shared leadership and a broad base of responsibility for reform efforts. The program encourages teamwork and the celebration of accomplishments. These and other means of support are part of the school's comprehensive design.

Parental and community involvement - The program provides for the meaningful involvement of parents and the local community in planning, implementing, and evaluating school improvement activities. In addressing this component, schools create strategies that are consistent with the parental involvement requirements of Title I, Part A. (See section 1118 of the ESEA.) Schools pay special attention to building parents' capacity for involvement and design ways in which parents can be brought into the instructional program and contribute to the academic achievement of their children.

External technical support and assistance - The program uses high-quality external support and assistance from an entity that has experience and expertise in schoolwide reform and improvement, which may include an institution of higher education. The CSR legislation requires that SEAs ensure that funded programs are supported by qualified technical assistance providers that have a successful track record, financial stability, and the capacity to deliver high-quality materials, professional development for school personnel, and on-site support during the full implementation period of the reform.

Annual evaluation - The program ensures accountability by including a plan for the annual evaluation of the implementation of school reforms and the student results achieved. The evaluation helps ensure that the school is making progress toward achieving its measurable goals and benchmarks and that necessary adjustments and improvements will be made to the reform strategies.

Coordination of resources - The comprehensive program must identify Federal, State, local and private financial and other resources that schools can use to coordinate services that support and sustain comprehensive school reform.

Strategies that improve academic achievement - The program must meet one of the following requirements:

the program has been found, through scientifically based research, to significantly improve the academic achievement of participating students;

or

the program has been found to have strong evidence that it will significantly improve the academic achievement of participating children.

Using all eleven components, schools must create a comprehensive, integrated reform program that affects all subjects, all teachers, and all children in the school.

Although teaching and learning activities are at the heart of every school program, the other CSR components define additional elements that are crucial to the successful day-to-day operation of every school. Attending to those details and their interrelationship with each other will have a positive impact on student achievement. Appendix B [MS WORD (45K) | PDF (154K)] offers additional guidance for assessing the comprehensiveness of school reform plans.

Therefore, it is critical that each component be part of a school's program and that the overall design be composed of proven strategies, methods and practices that either (a) have been found, through scientifically based research, to significantly improve the academic achievement of participating children; or (b) have been found to have strong evidence that they will significantly improve the academic achievement of participating children.

Scientifically based research, as defined in section 9101(37) of the ESEA, is research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs. To meet this standard, the research must -

Employ systematic, empirical methods that draw on observation or experiment;

Involve rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn;

Rely on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations, and across studies by the same or different investigators;

Be evaluated using experimental or quasi-experimental designs in which individuals, entities, programs, or activities are assigned to different conditions and with appropriate controls to evaluate the effects of the condition of interest, with a preference for random assignment experiments, or other designs to the extent that those designs contain within-condition or across-condition controls;

Ensure that experimental studies are presented in sufficient detail and clarity to allow for replication or, at a minimum, offer the opportunity to build systematically on their findings; and

Have been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review.

B-3. What evidence is critical in evaluating whether a body of research is scientifically based?

When reviewing research findings to determine whether they meet the criteria for scientifically based research, SEAs, LEAs, and schools should consider the extent to which the research meets each of the defined elements listed above. The following questions, as well as the information contained in Appendix C, will be helpful in evaluating research findings:

Systematic and empirical methods

Does the research have a solid theoretical foundation?

Were methodology, subject, and researcher clearly identified?

Was the study conducted in a consistent, disciplined, and methodical manner?

Were the data obtained using observation or experiment?

Was the research grounded in data that are factual rather than opinion-based?

Are the research findings supported by tangible, measurable evidence?

Rigorous data analyses

Did the research test the stated hypotheses and justify the general conclusions drawn?

Does the research report the sample size and the statistical procedures used?

Were the data analyzed using methods that were appropriate for the task?

Did the methods correspond to the nature and structure of the data?

Did the research minimize alternative explanations for observed effects?

Did the research findings present convincing documentation that the observed results were caused by the intervention?

Reliable and valid data collection

Did the data result from a study involving multiple investigators in a number of locations?

Were research biases minimized?

Were the data measured consistently? Did repeated measurements on subjects taken under similar circumstances produce similar results?

Strong research design

Does the design describe a random assignment experiment in which subjects are assigned to different conditions with appropriate controls?

Do the controls allow for the evaluation of the condition(s) of interest?

Was the study designed to optimize the investigator's ability to answer the research question?

Detailed results that allow for replication

Are the findings clearly described and reported?

Are the results of the research sufficiently detailed so that replication of the design is possible?

Can the findings be enhanced with additional research?

Results subjected to scrutiny

Has the research been accepted by a peer-reviewed journal or approved by a panel of independent experts?

Has the research been reviewed by unbiased experts who were not a part of the research study?

Have reviewers applied strict standards of scholarship and provided quality controls for the research they reviewed?

Has the research been subjected to external verification?

B-4. The CSR legislation requires participating schools to develop a comprehensive reform program that has been found, through scientifically based research, to significantly improve the academic achievement of participating students; or has been found to have strong evidence that it will significantly improve the academic achievement of participating children. What is meant by "strong evidence"?

"Strong evidence" defines a less rigorous standard than scientifically research based evidence. Practices, strategies and programs that demonstrate "strong evidence" of positive effects lack a broad research base that meets the criteria established in the definition of scientifically based research. (See B-2). Strong evidence is derived from a combination of high quality and reasonably high-quality research studies that demonstrate relevance, significance and consistency. In the absence of scientifically based research on the effects of comprehensive reform programs, schools are required to use the "strong evidence" standard by which to judge the quality of their programs. (See B-1 and Appendix C [MS WORD (173K) | PDF (345K].)

B-5. What assurance must an SEA make regarding the research base of funded CSR programs?

An SEA must ensure that each funded comprehensive reform program employs proven strategies and methods for student learning, teaching, and school management that are based on scientifically based research and effective practices and have been replicated successfully in schools. The program developed must have been found, through scientifically based research, to significantly improve the academic achievement of participating students or must be supported by strong evidence that it will significantly improve the academic achievement of participating children. The SEA is responsible for ensuring that funded programs are grounded in scientifically based research, best practice, and sound professional judgment.

B-6. Must a comprehensive school reform program contain all eleven components listed in the legislation?

Yes. A comprehensive school reform program supported with CSR funds must coherently integrate all eleven components, and its comprehensiveness is reflected in their interrelationship. Thus, in designing its program, a school should exercise care that the models, methods, and strategies that it incorporates create a coherent whole. It is critical that the program employ strategies and methods for student learning, teaching, and school management that are based on scientifically based research and effective practices and that have been successfully replicated in schools.

The eleven components embody the essential elements of successful schoolwide reform. They encourage analysis, planning, and the implementation of reforms that touch every part of a school program and all students in the school. Both nationally available and locally developed models are likely to address some of the eleven components, but not all of them. Even if a school adopts an external model, it must incorporate that model into a comprehensive program that addresses all of the CSR components. CSR funds may not be used simply to support randomly adopted or developed strategies; instead, the strategies much be integrated in a comprehensive design for effective school functioning, and the program must be supported by scientifically based research or have strong evidence of success. (See B-1.)

B-7. Is an SEA responsible for ensuring that only comprehensive school reform programs are funded?

Yes. An SEA must ensure that CSR funds support only programs that address each of the eleven components described in the legislation and that funded CSR programs have the capacity to improve the achievement of participating students in core academic subjects. (See Appendix B [MS WORD (45K) | PDF (154K)] for additional information on gauging comprehensiveness.)

B-8. To receive CSR funds, must a school implement a comprehensive school reform program that includes an externally developed reform model?

No. CSR schools must implement a comprehensive school reform program that addresses eleven components important to the successful functioning of a school. The design must employ proven strategies and methods for student learning and teaching based on scientifically based research and effective practice. The strategies for addressing the eleven components, and the integrated design of the program, must be found, through scientifically based research, to significantly improve the academic achievement of participating students or must be supported by strong evidence that they will significantly improve the academic achievement of participating students.

Schools may choose to adopt a nationally available reform model or a locally developed model that supports one or more components of its comprehensive design. However, the model remains only part of the overall design and must be integrated with the required components. Implementing a model of any kind is not required.