Promise to Practice Overview

Equity

Subgroup performance appears to be a focus only in Colorado’s compliance with the ESSA requirements for identification of targeted support and intervention schools—with no mention in the state accountability framework and school improvement materials. The state’s tools could be more specific about how it will address the needs of particular groups of students, and how it will help districts identify these focus areas.

Strengths

Colorado clearly presents and explains its menu of supports for districts and schools. The multiple routes districts may use to make improvement is also a strong example of local autonomy being balanced against state-level priorities. The state also differentiates levels of funding, depending on the district’s chosen path.

Improvements

Colorado’s use of dual accountability system raises issues with school improvement implementation. The state could also have a more robust plan for how it will monitor districts and engage in long-term evaluation of school improvement efforts, and provide more information and guidance regarding districts needing to be transparent about ratings for schools.

Coherent and Aligned Vision for Improving Outcomes:

Strategic Use of Funding and Alignment of Resources:

Rigorous Review Process:

Continuous Improvement, Monitoring and Evaluation:

Evidence-Based Interventions:

Capacity Building and Autonomy:

Engagement:

Sustainability:

Click through the tabs on the left to see how Colorado scored in each category.

Equity

How well does the state’s approach to school improvement include focused attention on supporting underserved students and closing the achievement gap? Does the state require LEAs to maintain an equity focus in their school improvement plans, activities and resource allocations?

Subgroup performance appears to be a focus only in Colorado’s compliance with the ESSA requirements for identification of targeted support and intervention schools. Meanwhile, the state accountability framework does not seem to consider subgroup performance at all. This lack of emphasis carries over into the state’s school improvement materials.

The state’s tools could be more specific about how it will address the needs of particular groups of students, and how it will help districts identify these focus areas. Colorado has different long-term goals for students with disabilities and they are very low compared to their non-disabled peer group. Moreover, English language proficiency is not part of their school-based accountability in the state model that produces four performance levels. Therefore, the progress of English learners is not represented in planning or guidance documents.

Strengths

How is the state thoughtfully leveraging ESSA’s flexibility to put in place the necessary policies and procedures that create an enabling environment for effective and sustained school improvement, and that consider state/local lessons learned from past efforts? What parts of the state’s turnaround strategy or guidance to LEAs were strongest or exemplary?

Colorado provides a very robust menu of supports for districts and schools identified for improvement. These resources are presented and explained clearly, packaged in a way that will enable local leaders to take advantage of them. The multiple routes districts may use to make improvement is also a strong example of local autonomy being balanced against state-level priorities.

Moreover, each of the pathways described in Colorado’s menu of supports entail differentiated levels of funding depending on the district’s chosen pathway. The state’s guidance materials delineate clear, specific funding bands for each of the available pathways, as well as a description of each. This approach to funding is unique among states so far in its clarity and specificity.

Improvements

How can the state improve its turnaround efforts? What parts of the state’s strategy or guidance to LEAs were unclear? What risks and challenges might the state face with its current approach?

Colorado’s use of dual accountability system raises issues with school improvement implementation. The state accountability framework places an emphasis on growth while ESSA emphasizes absolute achievement, and while there are merits to both, the perspective from an equity lens is quite clear, as a student may grow without ever reaching the potential expected of his or her peers.

The state could also have a more robust plan for how it will monitor districts and engage in long-term evaluation of school improvement efforts.

Colorado may benefit from communicating their long-term intent regarding the alignment of their state accountability school and district performance framework and their ESSA accountability system. In particular, subgroup performance as a potential component of the state accountability framework may drive district improvement and focus on students with disabilities and English learners, economically disadvantaged students, and those students in racial minority groups.

The state can provide more information and guidance regarding districts needing to be transparent about ratings for schools in both rating systems.

Coherent and Aligned Vision for Improving Outcomes

How well does the state articulate a coherent vision or theory of action that drives their school improvement efforts? Is this vision aligned with the state’s accountability system and goals for closing the achievement gap?

Colorado has a clear mission and vision statement, and its strategic plan includes five core values: strong foundations, all means all, quality schools, more options, and educators matter. “All means all” is clearly component that most directly targets equity issues. The state’s Unified Improvement Plan requires both specific identified root causes as well as strategies intended to address them, which is a strong practice. Equity appears to be a focus throughout the state’s materials. Colorado deserves credit for maintaining an equity strand throughout the documents available for review.

That said, the state has a dual system for school accountability, which undermines coherence and alignment and introduces some uncertainty in regards to Colorado’s equity focus. For example, English language proficiency is not part of Colorado’s school-based accountability system that produces the state’s four performance levels, though it is under the federal accountability system. The state accountability system also includes science and matriculation to college or a certificate program, and the federal system does not. There is not complete overlap between the identification categories and the comprehensive, targeted, and additional targeted support identification under Colorado’s federal accountability system under the two systems.

These inconsistencies may result in confusion about school improvement efforts. Nevertheless, given the reality of the two systems, the state seems to have done what it can to create an aligned vision with an equity focus.

Strategic Use of Funding and Alignment of Resources

Is the state allocating funding in a way that is strategic and maximizes resources? Are LEAs expected to prioritize improvement efforts that address the underlying performance issues?

Colorado has a strong menu of supports that contains district-designed and -led improvement strategies, state-offered services, and districtwide supports. These funds are requested through the state’s Empowering Action for School Improvement Grant, EASI, which also serves as the school improvement application under Colorado’s federal accountability system. Districts with comprehensive or targeted schools may choose what resources they need, and funding amounts are directly tied to the chosen pathway, which is a particularly clear and strong approach.

The menu of supports is differentiated by improvement tier and specifically indicates which districts and schools may apply for which pathways. It then describes each pathway, its duration, and the funds available. For example, a district selecting Diagnostic Reviews and Planning grant are eligible for a block grant of up to $50,000, whereas a district selecting Stakeholder and Community Engagement receives up to $20,000 per school.

The first guiding question that the state considers when evaluating a district level plan asks, “Does the plan investigate the most critical performance areas and prioritize the most urgent performance challenges?” This is a good indication that the state is mindful of the effort districts and schools are making to target their resources. That said, federal school improvement funds are reserved for comprehensive and targeted schools, not necessarily schools in the lower tiers of the state’s accountability system. This means that the state’s ability to target funds to need is complicated, if not mysterious.

Colorado may need to consider adding state funding to support those schools in the lower tiers of their state accountability framework. That way, the state would not need to ask districts to fund some of the supports that have made available in their menu of options for districts. A potential avenue may be using federal Title set-asides at the state level to provide a pool of resources for schools to opt-in to these services and supports at no additional charge.

Rigorous Review Process

Is the state applying rigorous criteria and review processes to ensure resources will be used to support effective school improvement efforts? Is the state prioritizing funding to LEAs who demonstrate the greatest need for school improvement funding (including LEAs with a high percentage of CSI and TSI schools) and the strongest commitment to school improvement?

It appears that Colorado has a mix between a competitive and a prioritization process for distributing funds for districts. Districts can choose to apply for a combination of five different routes: exploration supports, district designed and led, state-offered supports, and continuation and districtwide supports. The routes have different criteria and ways of awarding funds geared to a specific intent, and the funding levels vary greatly between them.

The school improvement application rubric covers four areas – needs assessment, implementation, targets & progress monitoring, and budget. Districts with plans scoring less than the designated number of points may be – but are not necessarily – asked to resubmit in order to be eligible for funding. The rubric is strong and seems to be focused on quality, not just compliance, though the degree to which multiple raters at the state calibrate their assessments of the district plans will be important in determining how well the rubric informs the development of high quality, useful improvement work.

The criteria and points are established such that only half of the total points result in a “meets” rating, which the state describes as an “adequate response” in its rubric. “Meets at high level” is defined as “concise, thoroughly developed response.” Based on these criteria, there is concern that Colorado will fund plans that are, on average, only adequate. Colorado may consider raising the minimum requirements for funding, and then providing technical assistance for districts to improve and re-submit plans that don’t meet criteria initially.

Continuous Improvement, Monitoring and Evaluation

Does the state have a robust, data-driven process to monitor LEAs’ implementation of the school improvement plans within their district? Did the state establish clear milestones to ensure improvement over time, and within four years?

This area is complicated for Colorado, as the state’s two accountability systems use different criteria for identification and movement out of lower performance tiers. The state has indicated that comprehensive and targeted schools will remain in their status for a minimum of three years in order to continue funding and ensure that schools are able to sustain any improvement efforts. Meanwhile, the state also noted in its strategic plan documents that 97 of 177 schools that were priority improvement or turnaround on their state accountability framework moved to the top two levels of their school performance categories from 2016 to 2017. This appears to mean that schools will simultaneously be able to move within the state accountability system tiers while also remaining identified for comprehensive or targeted support under Colorado’s federal accountability system.

Those considerations aside, the fifth guiding question in the state’s rubric for evaluating school improvement applications is, “Does the plan include elements to effectively monitor the impact and the progress of the action plan?” The state requires that districts set year on year performance targets, identify interim measures, and identify implementation benchmarks. In the state’s strategic plan, it has sections for each focus area on what progress looks like in 2022. The documents available seem to indicate that continuous monitoring is the responsibility of the district, not the state, but these components to planning are strong nevertheless.

Evidence-Based Interventions

To what extent is the state mandating LEAs use evidence-based strategies in their improvement efforts? Does the state provide guidance and supports to LEAs to help them identify and implement the most effective strategies based upon their needs?

In their school improvement application, districts in Colorado must use one of the four levels of evidence under Colorado’s federal accountability system and the intervention must fit with the need of the school. To ensure the best fit, selected strategies must also demonstrate an alignment to the contextual fit framework proposed by Horner, Blitz and Ross, which frames fit around need, precision, efficiency, skills, cultural relevance, resources, and organizational support.

Comprehensive and targeted schools may make use of the state’s diagnostic reviews and planning support, as well as engage in the turnaround network, which seem to be strong supports. However, if districts take advantage of state consultation to determine their evidence-based strategies, they must pay for the service. This practice is rare and is puzzling. Many states are making guidance around evidence-based strategies available to their districts and schools free of charge. Because all of the supports offered are optional, assessing fees creates a barrier to districts availing themselves to these supports.

Capacity Building and Autonomy

How well does the state articulate, delineate or set parameters around which interventions and responsibilities belong to the state, LEA and/or school? Does the state provide support or guidance to help LEAs identify and reduce barriers to school improvement? Does the state have a framework or process to support and monitor outside entities who partner with the state, LEAs or schools in school improvement efforts?

Colorado has clearly defined which interventions belong to the state or to the district. In fact, all responsibility for intervention is with the district. Districts and schools that have five consecutive years as “priority improvement or turnaround” on the state accountability framework are subject to the “accountability clock.” The accountability clock prescribes that, once time runs out, the state board of education will direct a local school board to pursue one of a defined set of actions as a more intensive intervention. These actions include a change in governance, a charter conversion, “innovation” status, school closure or district reorganization.

The state’s ESSA plan does mention the accountability clock in the discussion of more rigorous intervention for schools who remain in comprehensive status after three years, though Colorado should make more explicit the link between comprehensive schools under Colorado’s federal accountability system and the accountability clock.

There is no evidence of a framework or process to support and monitor outside entities who partner with the state, schools or districts in improvement work.

Engagement

Does the state require LEAs to engage with stakeholders such as parents and community members in the development and implementation of their school improvement plans? Does the state provide sufficient guidance and resources to LEAs to effectively do so, helping them foster local buy-in and promote sustainability?

Colorado’s school improvement application states that “Within all routes, geographic representation (i.e., representative distribution of districts across the state are awarded funds) and meaningful stakeholder engagement will be taken into consideration.” The application also states that “At a minimum in this application, all proposals will need to address how stakeholders are involved in a meaningful and relevant way in the development and implementation of proposed activities. Furthermore, stakeholder representatives are required to sign the proposal to verify support for the proposal.” The application could be strengthened by requiring representatives from a diverse set of backgrounds and roles. Still, requiring signatures is a great way to signal the importance of stakeholder involvement.

Engagement

Does the state have a plan in place to review the school improvement efforts statewide and evaluate the impact and effectiveness? Does the state have a process in place to support LEAs and schools by enhancing their capacity to maintain their improvement efforts upon exiting identification and intervention?

Colorado’s strategic plan sets a goal that 80% of the priority improvement and turnaround schools on their state accountability metrics will move to the top two levels of their performance framework by 2022, which is certainly a clear bar with which to measure success. The state’s ESSA plan does not mention any formal external or internal program evaluation for their school turnaround work.

The state is offering a robust menu of options for districts and schools, but those are divided among four pathways with some having more opportunities for capacity building than others. To the extent that districts are left to their own devices to select a pathway, there may not be uniform opportunities for districts to enhance capacity to sustain their work.

The notion of exiting identification and intervention is confusing due to Colorado’s dual systems of accountability. No schools can exit comprehensive or targeted status for three years, but the state accountability performance levels are being determined every year. In this environment, schools and districts may get mixed messages regarding what exactly it is that they should be sustaining.

Overview

Strengths

Colorado has clearly sought out and respected the feedback from many stakeholder groups, and its “hub and spoke” model for conducting outreach is a thoughtful and extensive approach.

The state makes a laudable effort at including more students in its accountability system by lowering the threshold at which it includes subgroups for accountability and reporting purposes – including additional ways to capture schools with low-performing subgroups – and creating a methodology for including subgroups when they separately do not meet the minimum number of students required.

It also provides a strong rationale regarding its content standards and the process used to validate alignment with higher education. This process should be replicated in other states.

Colorado’s plan for supporting schools recognizes that schools will be at varying levels of readiness to engage with the state and establishes a range of interventions in response. The plan should be commended for its focus on leadership as a target of school turnaround efforts.

Weaknesses

Colorado is proposing to shift to an entirely normative approach, where all indicators and the accountability system itself are based on relative performance, not a predefined standard. That approach may not be sufficiently clear to parents, educators, or other stakeholders, and it means the accountability system has no incentives aligned to the state’s professed goal of college and career readiness for all students.

The state also lacks coherent goals for its schools, and the ones provided are disconnected from the state’s long-term vision. As a result, the system does not provide schools clear signals about how they need to improve, and it’s particularly problematic for those students who have been historically sidelined as a result of their race, class, and/or life circumstances.

Colorado’s method of excluding students whose parents opted them out of state tests may undermine the state’s proposed school-rating methodology.

Colorado has also not finalized a number of decisions in its accountability system, leading to uncertainty about how exactly it will be implemented in practice.

Click through the tabs on the left to see how Colorado scored in each category.

GOALS

Colorado’s long-term goals are disconnected from the state’s vision.

The vision lays out the priorities of all students demonstrating readiness for school, third-grade reading proficiency, meeting or exceeding standards throughout their schooling years, and graduating high school ready for college and careers.

However, the state’s plan does not set long-term goals for students’ early preparation, includes average scale scores rather than a proficiency measure, and does not include any goals for college and career readiness.

The enumerated long-term vision is also disconnected from the state’s system for classifying school performance.

Colorado has not set clear long-term goals or interim targets to reach its vision.

The state has proposed a confusing percentile-based system that intends to raise the statewide performance from the 50th to the 53rd percentile. This is:

Difficult for parents, educators, and the public to understand

Does not set strong expectations for all schools and all groups of students to improve

May not be ambitious improvement because the plan does not provide any information about the percentage of students meeting grade-level standards at that performance level.

As written, Colorado expects children with disabilities, who currently score at the 1st percentile statewide, to score at the 53rd percentile in six years. That would be an impressive gain, but Colorado also expects Asian students, who currently score at the 82nd percentile statewide, to regress backward to the same 53rd percentile within six years.

In contrast, Colorado has set graduation rate goals using objective data on past performance. The state set a goal of increasing its graduation rate to 90.3 percent within a six-year time frame, based on its analysis of what the state has achieved in recent years. But unlike the percentile approach, which is normative, graduation rate gains are based on actual, observed changes over time against a predefined threshold. The state should apply a similar approach to its achievement goals.

STANDARDS AND ASSESSMENTS

Colorado has a high-quality assessment in grades 3-9.

Colorado has a high-quality assessment system, the Colorado Measures of Academic Success, in grades three through nine, which includes state-developed science and social studies assessments and PARCC English language arts and math assessments. Colorado also deserves credit for including its assessment in science into its accountability system. This decision may help ameliorate concerns about curriculum narrowing and expand the scope of what schools focus on beyond reading and math.

Using PSAT and SAT at the high-school level has pros and cons.

At the high school level, Colorado is using the PSAT and SAT as its accountability assessment. Absent an independent review, we don’t yet know if the PSAT/SAT is fully aligned with Colorado’s state academic standards. While offering the SAT as the state’s official test offers many benefits, some of those key benefits may not extend fully to all students who require accommodations and may not receive college-reportable scores.

Colorado has made efforts to accommodate all students.

Colorado has provided a detailed description of its accommodations for English language-learners, particularly Spanish-speaking students. However, Colorado could strengthen its plan by ensuring that it has a process in place to meet the 1 percent cap on alternate assessments for students with the most significant cognitive disabilities.

INDICATORS

Colorado has not determined how each indicator will be weighted.

While Colorado’s plan describes how each of the measures that might be used in the accountability system are valid, reliable, comparable across schools, and supported by research, it has not yet determined how each indicator will be weighted in its accountability system.

Colorado’s plan is lacking indicators that would better match its long-term priorities.

The plan could be improved through the inclusion of early childhood as well as postsecondary and workforce readiness indicators.

The state could also develop a menu of college- and career-ready predictors (e.g., Advanced Placement, International Baccalaureate, or industry credentials) to help ensure schools are equipping students for success after high school.

Colorado should have a stronger plan in place to ensure that parental opt-outs do not undermine the school-rating system.

As the state transitions to the SAT, it may be useful to include the percentage of students meeting the college-readiness benchmark.

Similarly, some of Colorado’s indicators could be altered or strengthened to better align with its goals.

Colorado has proposed an alternative measure for including proficiency on statewide annual assessments in its accountability system called mean scale scores. This measure merely reflects schoolwide averages and does not track the percentage of students meeting grade-level standards.

While Colorado’s plan includes four-, five-, six-, and seven-year graduation rates in its accountability system, it could strengthen its plan by placing greater weight on its four-year graduation rate.

The state’s use of a dropout rate as its indicator of school quality and student success for high schools is also questionable. The dropout rate is a measure of retention, not attainment, and it may send an inaccurate view to parents and the public of the effectiveness of the education system.

ACADEMIC PROGRESS

Colorado has yet to determine what emphasis to put on student progress.

Colorado has been considered a national leader when it comes to assessing academic progress. Its growth model, based on student growth percentiles, has been adopted by a number of other states. The state has also applied its growth model to its English-language proficiency assessment (WiDA ACCESS for English language learners). However, Colorado has not yet determined how much of an emphasis to place on student progress in its accountability system.

Colorado’s growth model lacks the proper benchmarks.

The state’s growth model compares low-performing students with other low-performing students, rather than setting a benchmark that all students should aspire to.

Combined with an achievement measure that also ignores the proficiency benchmark, this system may not create a sufficient incentive for Colorado schools to pay attention to raising the performance of low-performing students. Colorado could strengthen its plan by incorporating its “adequate growth percentile” calculation, which does look at a student’s progress toward state benchmarks.

ALL STUDENTS

Colorado is taking the right steps to ensure that all students are being counted.

The state will not be incorporating subgroup scores directly into its school-rating system, but it does have a plan to identify 5 percent of schools for each of the lowest performing subgroups as schools in need of targeted support. If any particular subgroup continues to flounder for four consecutive years, Colorado will shift those schools into more rigorous, comprehensive support status.

Colorado proposes creating a “super-subgroup” only when the individual racial/ethnic subgroups do not meet the state’s minimum threshold on their own. The plan provides compelling data behind the state’s rationale for using this measure, and limits its use only when necessary.

It is also lowering its n-size, the minimum group size used to determine if schools should be held accountable for the performance of subgroups, from 30 to 16 for achievement and graduation indicators and to 20 for growth indicators – which will ensure more schools are paying attention to more subgroups of students.

Colorado also has a strong plan in place for proposing accountability metrics for its Alternative Education Campuses and for inclusion of extended-year graduation rates to ensure that all students are served, and it could be further improved if the state determined an accountability approach for K-2 schools.

The state has clear plans to help students with disabilities, but could benefit from early screening.

Colorado has done a good job aligning its plan to the IDEA State Systemic Improvement Plan and its IDEA State Personnel Development grant. The state also plans to incorporate Response to Intervention and Multi-Tiered Systems of Supports in all schools. Finally, it describes integration of the Early Childhood workforce, but could benefit from specifically screening students for learning disabilities early to ensure educators incorporate interventions beginning in pre-k.

Colorado uses the same methodology to account for subgroups of students as it does to identify the lowest-performing 5 percent of Title I schools. A school with a subgroup in the lowest 5 percent for that subgroup will be identified for additional targeted support.

The state’s definition of a “consistently underperforming” subgroup as one that earns the “lowest rating” on at least three indicators is concerning for several reasons:

First, no information is provided regarding what level of performance will constitute the “lowest rating.”

Second, if a school has a subgroup underperforming in math and English achievement, it would still need to also earn the lowest rating on a third indicator to be labeled for targeted support.

Further, it is not clear in the plan the difference in definitions between “consistently underperforming” and “chronically low performing” subgroups.

More information is needed when it comes to establishing performance expectations.

Colorado’s performance expectations are normative, which means schools are compared to each other, not against set expectations. It is unclear if the performance bands they set stay static or change over time, and over what time period. If they change annually, that artificially “forces” schools to the bottom and the top of the spectrum, which could send inappropriate signals to stakeholders.

The plan’s proposal to exclude students whose parents opted them out of state assessments in accountability determinations could also undermine the state’s school-rating system. The state could strengthen its plan by identifying schools in need of comprehensive support and improvement using the four-year graduation rate, not the extended rate, as the state proposes.

SUPPORTING SCHOOLS

Colorado’s support mechanisms for low-performing schools are relatively vague.

The state’s plan cites a host of possibilities (e.g., performance management tools, professional learning) and describes a resource list, but lacks details.

The one area the plan should be commended for is its focus on leadership as a target of school turnaround.

Additionally, Colorado does have an “Accountability Clock,” embedded in state law, but the plan does not make clear connections between possible interventions and what’s statutorily required.

Colorado’s plan does provide a description of the tiered system of performance management and technical assistance.

The plan lays out an ongoing process for matching 7 percent of its Title I funds dedicated for school-improvement activities using what is called a “needs-based approach.”

However, the goals outlined by the state appear to be in tension, specifically the intent of “incentivizing innovation” while “providing predictability” to school districts.

Given the importance of the former, a competitive allocation process aligned to the state’s long-term priorities would likely better serve the state’s aspirations.

The state’s plan could be improved if the state took advantage of the Direct Student Services opportunity and articulated how it intends to use those resources to provide greater access to advanced coursework for traditionally underperforming subgroups of students.

EXITING IMPROVEMENT STATUS

Colorado’s plan for exiting low-performing schools is weakened by its poor system of identification and lacks detail.

Based on the state’s description, schools could exit improvement status by jumping in the state’s relative rankings without making any actual improvements (other schools could simply fall behind them).

While the state specifies that schools must remain in the Comprehensive Support classification for three years, it provides no information about ongoing performance expectations necessary to exit this list. It is unclear whether/how averaging school performance over three years may make exiting improvement status more or less challenging for these schools.

Further, Colorado provides no clear timeline for supporting targeted support and improvement schools.

CONTINUOUS IMPROVEMENT

Colorado’s plan was developed through significant consultation with stakeholders in and out of the education system. But the plan could be further strengthened by outlining the steps the state will take to continue to engage with noneducation actors, especially the disability community.

The state also intends to establish annual cycles for evaluating the efficacy of its implementation efforts. However, most of the state’s efforts are focused on school- and district-level support, when there’s an opportunity in ESSA to directly address student needs.