Promise to Practice Overview

Equity

Arizona sends mixed messages to improvement districts and schools with regards to equity. It’s theory of action makes no mention of equity, while materials for districts and schools include an equity focus; the state encourages districts and schools to break down assumptions and biases that could inhibit school performance, yet does not as this to be included in school improvement plans. Arizona could also benefit from monitoring improvement plans to ensure they are advancing outcomes for students.

Strengths

Arizona’s plan places an emphasis on local control and flexibility in implementing improvement plans, though the state provides strong guidance for comprehensive support and improvement schools. It also requires districts and schools to upload plans and documents into a database for transparency and to track progress.

Improvements

Arizona could benefit from a plan to ensure districts and schools remain out of comprehensive support and improvement once exited. Arizona could also better address causes of inequities in its school improvement materials, and provide greater guidance around evidence-based practices for comprehensive and targeted support districts and schools.

Coherent and Aligned Vision for Improving Outcomes:

Strategic Use of Funding and Alignment of Resources:

Rigorous Review Process:

Continuous Improvement, Monitoring and Evaluation:

Evidence-Based Interventions:

Capacity Building and Autonomy:

Engagement:

Sustainability:

Click through the tabs on the left to see how Arizona scored in each category.

Equity

How well does the state’s approach to school improvement include focused attention on supporting underserved students and closing the achievement gap? Does the state require LEAs to maintain an equity focus in their school improvement plans, activities and resource allocations?

Arizona’s theory of action makes no mention of equity. That said, the state includes some equity-oriented language throughout their comprehensive support and improvement guidance documents. This indicates a disconnect between the theory of action and the state’s school improvement materials, calling into question Arizona’s focus on supporting underserved students. For instance, Arizona does require schools and districts to address some of the factors of equity in their root cause analysis however, there is no follow up to this in the school improvement application calling for strategies for addressing difficult and often deep systemic inequities in areas like human capital, graduation rates, engagement, and discipline. Districts are asked to analyze data by demographic group and to review academic achievement gaps by these groups, yet there is no specific requirement to address them in the integrated action plans as required by the state.

To be clear, it is encouraging in the root cause analysis to see a specific tool for breaking down assumptions and preconceived biases that could be inhibiting student performance. The directions and guiding questions are very strong and should be effective for guiding district and school leaders through toward some conclusions. It is a missed opportunity, then, that the plan itself does not ask for those conclusions. Schools and districts would benefit from a strand of questioning that promotes equitable solutions and strategies.

Arizona needs to monitor plan implementation to ensure that the solutions are effectively addressing achievement gaps. Additionally, funding should be prioritized for improvement efforts that are obtaining positive results in addressing systemic inequities resulting in the closure of achievement and opportunity gaps. The state should highlight and celebrate successful approaches to equity as part of its attempt to encourage and scale equitable strategies.

Strengths

How is the state thoughtfully leveraging ESSA’s flexibility to put in place the necessary policies and procedures that create an enabling environment for effective and sustained school improvement, and that consider state/local lessons learned from past efforts? What parts of the state’s turnaround strategy or guidance to LEAs were strongest or exemplary?

Arizona’s school improvement guidance for comprehensive support and improvement schools is strong and thorough, taking local leaders through the process step by step. The processes for completing improvement plans is detailed and hits on many important components of meaningful school improvement. The state requires these plans and other documents to be uploaded into a state database for transparency and tracking progress, meaning the state has the ability to maintain a firm understanding regarding the field’s progress.

The state is putting a premium on local flexibility so that district plans match district and school needs, which can be a strength, especially if monitoring visits prioritize true dialogue to enable continuous improvement and not compliance. Arizona’s plan to create and support effective leadership, which is based on the findings of the state’s educator effectiveness equity plan, builds the capacity of local leaders, which is critical.

Improvements

How can the state improve its turnaround efforts? What parts of the state’s strategy or guidance to LEAs were unclear? What risks and challenges might the state face with its current approach?

Arizona does not appear to have a strategy in place to ensure that districts and schools remain out of comprehensive support and improvement once exited. Continuous improvement efforts must be central to the state’s vision and take a wider look at the issues and challenges preventing long-term and sustained improvement. As schools improve from being in comprehensive status, the state must plan to ensure that the resources are there to maintain and accelerate successful efforts. One component to this would be to strengthen the engagement of the stakeholders both at the local and state levels.

The state is currently not adequately addressing the issues and challenges causing inequities in its school improvement materials. In addition, the approach does not include a framework to identify strategies that work, how to replicate and scale them, and how to sustain them with limited resources, and the state provides no guidance or incentives around specific strategies.

Overall, the documents available focus on compliance and not on driving the type of school improvement work that gets significant increases in student outcomes. It could well be that work on the ground in Arizona tells a different story, but the state is nevertheless missing an opportunity to positively affect the discourse around school improvement efforts statewide.

Coherent and Aligned Vision for Improving Outcomes

How well does the state articulate a coherent vision or theory of action that drives their school improvement efforts? Is this vision aligned with the state’s accountability system and goals for closing the achievement gap?

Arizona’s theory of action leaves significant room for improvement. It uses language directly from ESSA requirements, which makes it feel based in compliance and not improvement. Further, it does describe the levers the state will use, nor clear goals, to guide improvement efforts at the state level and in the field.

Schools receiving an F and D in the state’s accountability system are required to submit comprehensive needs assessments, along with root-cause analyses to identify high impact strategies. However, the link between the theory of action and the accountability system is not clearly articulated, and while the theory of action states it will result in improved outcomes for all students, not all schools will be undertaking the activities it describes.

Strategic Use of Funding and Alignment of Resources

Is the state allocating funding in a way that is strategic and maximizes resources? Are LEAs expected to prioritize improvement efforts that address the underlying performance issues?

It does not appear that Arizona is taking advantage of the opportunity to encourage districts and schools to pursue a specific set of state-supported or high-leverage strategies, and instead relies on ESSA’s definition of evidence-based interventions. The state appears to be agnostic regarding the strategies submitted in school improvement applications, and there also does not appear to be a state-provided resource for districts to find even suggested strategies to choose from.

The application for school improvement funds is competitive, but it is unclear how the competitive use of funds is aligned with the states vision of improving outcomes. For instance, there seems to be little emphasis on the quality of the application as a whole or the extent to which the district has shown that it is addressing the needs determined through the comprehensive needs assessment. In addition, Arizona does not appear to consider the capacity of the district leadership team to create a competitive application. As a result, it is hard to see how funds will be distributed in a strategic manner or reach the schools and districts with the greatest need.

Rigorous Review Process

Is the state applying rigorous criteria and review processes to ensure resources will be used to support effective school improvement efforts? Is the state prioritizing funding to LEAs who demonstrate the greatest need for school improvement funding (including LEAs with a high percentage of CSI and TSI schools) and the strongest commitment to school improvement?

Arizona has developed a coherent set of guidelines and processes for comprehensive support and improvement schools, which are those schools that received a D or F accountability rating. The actions a school must take are logical and strong: comprehensive needs assessment; identification of three to four primary needs based on the comprehensive needs assessment; root cause analysis of the identified primary needs; the Five Why’s method for each of the primary needs. The documentation and artifacts from these processes are required to be uploaded into a state portal, along with measurable goals to track progress. Furthermore, districts are also required to upload integrated action plans on how they will support campuses in their identified support areas.

However, there is concern that the lack of attention to funding, both in the school improvement application and in these tools, will not result in effective use of those funds both at a state and local level. Further, there does not seem to be consideration given to non Title-I schools who received a D or F but cannot receive federal school improvement funds from the state.

Arizona uses an on-site support and progress monitoring review process to help gauge the effectiveness of the improvement efforts. It will be interesting to see whether the state has the resources and capacity to conduct these visits as a continuous improvement exercise and not simply a compliance review.

Continuous Improvement, Monitoring and Evaluation

Does the state have a robust, data-driven process to monitor LEAs’ implementation of the school improvement plans within their district? Did the state establish clear milestones to ensure improvement over time, and within four years?

While districts use the state’s system to upload their needs assessments and improvement plans, there is no evidence that this system is then leveraged to monitor the plans and help districts improve their approach. The monitoring activities that are described seem compliance-focused. While districts and schools do need to submit substantial amounts of information, the information is being checked for compliance, not quality or effectiveness. On the other hand, the data review performed with the school or district during the on-site and progress monitoring visits is potentially a high-impact strategy.

The comprehensive needs assessment requires measurable goals which are monitored by the state, however, the goals are set for the upcoming school year alone and are not year over year targets. “Improvement” is going to be defined differently in almost every case because goals are entirely determined by the local districts and schools, making it more difficult for the state to assess the impact of its school improvement system.

Evidence-Based Interventions

To what extent is the state mandating LEAs use evidence-based strategies in their improvement efforts? Does the state provide guidance and supports to LEAs to help them identify and implement the most effective strategies based upon their needs?

Arizona uses ESSA’s language around evidence-based interventions verbatim, which meets the letter
of the law but does little more to provide support to districts and schools, which could quickly result in practices that do not meet the legal requirements for evidence-based strategies. The state has missed an opportunity to designate the evidence-based strategies it would like districts and schools to prioritize. The strategies being employed are reported to the state, but there does not seem to be any attention paid to the merits of each approach aside from the tiers of evidence described in ESSA.

Capacity Building and Autonomy

How well does the state articulate, delineate or set parameters around which interventions and responsibilities belong to the state, LEA and/or school? Does the state provide support or guidance to help LEAs identify and reduce barriers to school improvement? Does the state have a framework or process to support and monitor outside entities who partner with the state, LEAs or schools in school improvement efforts?

Arizona does not clearly articulate the responsibilities for which the state, districts, and schools
are responsible as a part of the state’s school improvement system. The state has provided a clear framework for comprehensive support and improvement schools, however, and there are some activities assigned explicit to the state, districts, and schools. Even so, the state does not provide any guidance to districts regarding priority interventions or ways to reduce barriers to school improvement.

The state has implemented the “Elevate — Arizona’s Executive Leadership Network” program, which focuses on the “knowledge, competencies and skills of leaders are they work toward systemic change within schools and LEAs.” This opportunity seems to be an exception to Arizona’s otherwise compliance- driven approach and is designed for true systems change. The state subsidizes the cost for leaders to apply, and while this is promising, the program is certainly not touching all of the leaders it needs to be impacting, and especially not at the scale articulated in the state’s theory of action.

Engagement

Does the state require LEAs to engage with stakeholders such as parents and community members in the development and implementation of their school improvement plans? Does the state provide sufficient guidance and resources to LEAs to effectively do so, helping them foster local buy-in and promote sustainability?

The state’s theory of action does not include a mention of involving the community or stakeholders — not even educators. It is difficult to see how the level of community engagement Arizona asks of their districts will result in a healthy culture of improvement.

Additionally, it is clear from Arizona’s school turnaround process and documentation that community engagement is not a priority in the development of school improvement plans. The only stakeholder engagement required is a public presentation of the plan that was developed — it is important to note that this requirement is not even worded as a workshop or public comment period, but rather a presentation to the public on what has already been decided. This again is only a compliance exercise, not meaningful dialogue.

Engagement

Does the state have a plan in place to review the school improvement efforts statewide and evaluate the impact and effectiveness? Does the state have a process in place to support LEAs and schools by enhancing their capacity to maintain their improvement efforts upon exiting identification and intervention?

The state does not require districts or schools to describe or design interventions that are sustainable over time, and there are no concrete processes and practices in place to review and evaluate practices across the state. There appears to be no long term strategy to address the continuing support needs of schools and districts as they exit identification status, and there are no common metrics with which the state would define success. Evaluation of sustainable strategies is going to look different for each school, and that will make progress difficult to benchmark over time.

Arizona’s approach does not appear to leverage the district information and planning documents being collected in the state’s online system. The state should develop a process to evaluate the impact and effectiveness of the individual district and school improvement plans and strategies. This is another missed opportunity, since theoretically this repository could allow for the state to begin to support

the scaling of effective practices and strategies or at the least provide more guidance to districts and schools about strategies that are achieving positive results.

Check State Plans Overview

Strengths

The plan’s proposed balance of achievement and growth measures is a potential model for other states, and its innovative idea to incorporate a “menu” of school quality indicators is noteworthy.

Weaknesses

Arizona’s plan lacks details, especially on how it will meaningfully incorporate subgroups into its accountability system, how it will identify schools for improvement, and the criteria low-performing schools would need to meet to exit improvement status.

Overall, the plan Arizona submitted did not adequately address how it plans to use federal funds to help increase student achievement, increase options for students, or intervene in chronically low-performing schools.

Click through the tabs on the left to see how Arizona scored in each category.

GOALS

Arizona aims to have 90 percent of its students proficient in English and math and to have a 90 percent four-year graduation rate by 2030.

It has set interim targets for each of these measures in three-year intervals, but those will apply in very different ways for different subgroups of students, and, absent data suggesting what’s feasible, the vast majority of them appear very ambitious. Arizona’s plan would be stronger if it provided clear strategic guidance on how it intended to address these large gaps in the time period identified. In contrast, the state’s graduation rate goals appear more realistic, and the state has pegged those goals to leading states of today.

STANDARDS AND ASSESSMENTS

Arizona has presented only basic information about its approach to standards and assessment.

Arizona adopted college- and career-readiness standards in 2010 and recently made small revisions of a mostly technical nature. It appears as though the state intends to continue utilizing its assessment, AZMerit, but the plan does not provide significant details in this area. Arizona is planning to take advantage of the flexibility to use end-of-course assessments in eighth grade, which is a productive use of ESSA’s assessment flexibility. There was no mention of alternate assessments for students with disabilities.

Arizona will need to be clear about how it will support its large English learner population.

Arizona is an English-only state, so the state does not translate its assessments into any other language. Moreover, Arizona should strengthen its plan by ensuring that it has a process in place to meet the 1 percent cap on alternate assessments for students with the most significant cognitive disabilities.

INDICATORS

Arizona’s plan includes a relatively simple list of indicators with mostly straightforward explanations.

For elementary schools, its system will include student achievement (English-language arts, math, and science), student growth (both a normative and a growth-to-standard model), English-language proficiency, and measures of acceleration/readiness. The acceleration measure has 5 “menu items” that schools can choose from to accrue points on the state rating system: accelerated math scores, decreases in the percentage of 3rd graders scoring minimally proficient in 3rd grade, subgroup improvement, special education inclusion, or chronic absenteeism.

The indicators for high school are identical, except a menu of college- and career-ready measures instead of acceleration, plus graduation rates (four-, five-, six-, and seven-year rates). The menu of college- and career-ready options includes things like completing a CTE sequence, scoring high on the ACT, passing an AP test or the ASVAB, completing the FAFSA, or earning an industry recognized credential. Each of these are worth a different number of points per student.

While these “menu” items are an interesting way for the state to encourage schools to offer well-rounded curricula and meet student needs in a variety of ways, it could pose a challenge for educators and parents to understand how their respective schools compare. Moreover, Arizona should monitor its data to ensure that all of its options are comparable or if certain types of students are disproportionately likely to pursue certain pathways.

ACADEMIC PROGRESS

It proposes to include a simple measure of student achievement (percent proficient) and two different measures of student growth. For K-8 schools, 30 percent of their rating would be based on achievement, 25 percent based on a normative growth measure that calculates the progress students make compared to their similarly performing peers, and 25 percent based on a growth measures that tracks whether students are on track to proficiency within three years or by the end of 8th grade, whichever comes first.

Arizona’s achievement measure gives students more credit for advancing to higher performance levels, but does not allow students scoring at advanced levels to fully compensate for lower-performing students.

Arizona deserves credit for designing a system that balances incentives for students to meet grade-level targets and, for those students who aren’t near the proficiency benchmark, to make progress year-over-year.

ALL STUDENTS

Arizona’s plan includes little information about how it would incorporate subgroup performance into school ratings beyond the statutorily required information.

The state’s proposed n-size of 20 is encouraging, and the state presents data about how many different subgroups will be captured under a group size of 20 versus other alternatives. But it’s hard to tell what practical impact that would have because Arizona does little more than promise to meet the minimal federal definitions to identify schools with low-performing subgroups for targeted support efforts.

Subgroups are also barely included in the state’s A-F grading system.

They’re not included in high school ratings at all, and for elementary and middle schools they would play a very small role (schools could earn up to 6 points out of 100 for reducing school-level achievement gaps), but only if they chose that item on Arizona’s “menu” of school quality indicators.

Arizona proposes including former students with disabilities in the students-with-disabilities subgroup for two additional years after they are no longer receiving services.

Since exiting students tend to have higher performance, the state should monitor its data to ensure it is not masking the performance of students who are still receiving services.

IDENTIFYING SCHOOLS

Arizona does not describe how final letter grades would be awarded.

Arizona’s plan gives a list of indicators, and it specifies how those indicators would be weighted on a 100-point scale, but the plan does not appear to describe how final letter grades would be awarded. Arizona says it will meet the minimal federal requirements and identify the lowest 5 percent of all schools in need of comprehensive support and identify for targeted support any school with a subgroup that, on its own, would qualify for the bottom 5 percent overall. Still, Arizona’s plan does not provide any specifics on how many schools might fall into that category.

SUPPORTING SCHOOLS

Schools identified for comprehensive and targeted support can apply for grants using a needs assessment and action plans.

Prioritization is based on schools that show evidenced-based decision-making, selection of interventions, root cause analysis, greatest need, and strongest commitment to using funds for student achievement and outcomes. Arizona also provides a fiscal review process and technical assistance as needed.

Arizona’s school improvement plans are thin.

The state mandates that low-performing schools perform a comprehensive needs assessment that will look at leadership capacity, instructional practices, curriculum and assessments, and organizational climate and culture. Arizona says this will involve a number of stakeholders, including school staff, families, and community members, but the state does not provide any additional information on how it would target assistance.

The needs assessment description is largely high level and provides little insight into how it would actually work, on what timeline, and if there were any clear guideposts or benchmarks. The state also provides a long list of tools for support and improvement, but how they all tie together is unclear.

Arizona also provides no description of how the state will use its 7 percent set-aside dedicated to school-improvement activities.

The state provides little detail around what evidence-based strategies low-performing districts and schools will need to pursue.

If the state is going to devolve a lot of improvement and support to its districts, it should be clearer about that. However, it does appropriately address its involvement in school and district improvement, including stepping in when there is systemic failure to improve.

Exiting Improvement Status

Arizona’s plan has only the most rudimentary of information regarding schools’ exiting improvement status.

Schools may exit comprehensive support if they no longer meet identification criteria. For schools in need of targeted support, the same rule applies, but without a timeline. The state is not detailed, clear, or rigorous about what schools would need to accomplish to exit improvement status, and it has not outlined criteria that would ensure schools are progressing and succeeding before being exited from improvement status.

Continuous Improvement

Arizona does not provide sufficient description about how it will foster continuous improvement for all schools.

Moreover, Arizona’s plan does not signal that the state is requiring or working to achieve meaningful engagement with the disability community, or parents in general. This is a missed opportunity to create a stronger, more useful, and more targeted plan.