Promise to Practice Overview

Equity

Indiana addresses equity in its theory of action, though there is room for a greater connection to the state’s ESSA plan, as well as a more explicit focus on how the state expects districts to address the needs of particular student groups. Indiana also does not include subgroups in its accountability system, which is concerning.

Strengths

Indiana’s use of the 5 Essentials and the state’s articulated theory of action are well developed, thoughtful, and research based. The state has also committed to greater cross agency coordination to streamline practice and centralize resources.

Improvements

Indiana’s standards for allocation of competitive funds for implementation are unclear and potentially too subjective. The state should also address sustainability and continuous improvement earlier in the process. Indiana’s use of dual accountability systems can also cause confusion about which schools are being identified and how to prioritize them.

Coherent and Aligned Vision for Improving Outcomes:

Strategic Use of Funding and Alignment of Resources:

Rigorous Review Process:

Continuous Improvement, Monitoring and Evaluation:

Evidence-Based Interventions:

Capacity Building and Autonomy:

Engagement:

Sustainability:

Click through the tabs on the left to see how Indiana scored in each category.

Equity

How well does the state’s approach to school improvement include focused attention on supporting underserved students and closing the achievement gap? Does the state require LEAs to maintain an equity focus in their school improvement plans, activities and resource allocations?

Indiana’s theory of action includes an equity focus. However, the state school improvement application process does not specifically call attention to many of the areas identified in the state’s ESSA plan. For example, Indiana’s ESSA plan identified inequality in educator equity differences. The school improvement plans do not address this area.

Indiana should be more specific about how the state expects districts to address the needs of particular groups of students, and how it will help districts identify these focus areas. The state does not include subgroups in its accountability system, and this raises concerns of a lack of commitment to subgroup accountability, support, and improvement.

Strengths

How is the state thoughtfully leveraging ESSA’s flexibility to put in place the necessary policies and procedures that create an enabling environment for effective and sustained school improvement, and that consider state/local lessons learned from past efforts? What parts of the state’s turnaround strategy or guidance to LEAs were strongest or exemplary?

Indiana’s use of the 5 Essentials and the state’s articulated theory of action are well developed, thoughtful, and research based. They require adherence to moderate-to-high standards of evidence for interventions while simultaneously recognizing that some school-specific customization will likely be necessary, and the rubric is extremely robust.

The state has committed to greater coordination across state agencies and worked to identify areas in which they can streamline practice and centralize resources in order to free up districts and schools to focus on the school improvement work. The state is also encouraging districts with more than four identified schools to apply as a district, which will further support shared learning and economies of scale.

Improvements

How can the state improve its turnaround efforts? What parts of the state’s strategy or guidance to LEAs were unclear? What risks and challenges might the state face with its current approach?

Indiana’s standards for allocation of competitive funds for implementation are unclear and potentially too subjective – more information on the how the qualitative and quantitative indicators will be used in funding decisions would be helpful and the inclusion of a readiness or needs assessment could help districts better use the funds. Interventions must meet one of the top 3 levels for strength of evidence, and Indiana allows for customization of interventions beyond that, rather than requiring a more rigorous use of the comprehensive needs assessment.

The state should address sustainability and continuous improvement in the early stages. The state appears to be organizing itself in a way to sustain the efforts and to have a coherent and aligned strategy to support school improvement. It would benefit school improvement efforts across the state if the work to redesign the state education office occurred across districts as well. Districts could benefit from the approach the state has undertaken to create a structure to support school improvement.

Indiana’s use of dual accountability system raises issues with school improvement implementation as it can cause confusion about which schools are being identified and how to prioritize efforts.

Coherent and Aligned Vision for Improving Outcomes

How well does the state articulate a coherent vision or theory of action that drives their school improvement efforts? Is this vision aligned with the state’s accountability system and goals for closing the achievement gap?

Indiana appears to be incorporating the University of Chicago’s 5 Essentials as the driving force for the school improvement process, which is intuitive and based in research. That said, the state’s updated school improvement grant application and guidance do not reference or incorporate references to the 5 Essentials or seem to encourage districts and schools to use it as a framework for improvement. The state also references the Universal Design for Learning and Multi-Tiered Systems of Support on its school improvement webpage, but it is unclear how those are further incorporated as well. On the other hand, Indiana has created a guidance document that crosswalks the components of the state’s comprehensive needs assessment tool with the relevant areas in the grant application.

The state appears to have developed a set of practices at both the planning and implementation stages that prioritize streamlining. For example, Indiana has created several school improvement grant applications – for planning. Implementation, and single or multiple schools – that districts can select from based on their situation.

Strategic Use of Funding and Alignment of Resources

Is the state allocating funding in a way that is strategic and maximizes resources? Are LEAs expected to prioritize improvement efforts that address the underlying performance issues?

Indiana will provide planning grant funds to first year comprehensive support and improvement schools, and will subsequently award implementation grants after districts complete their plans. Districts must apply for both grants, which will be awarded on a competitive basis. In addition, districts that can demonstrate that they have sufficiently planned for school improvement already may skip the planning stage entirely and move straight to their implementation grant, which may cut down on repetitive processes and wasted funds. The state appears to be using the planning grant application as a readiness assessment that must be completed to satisfaction before distributing planning grants in order to help districts and schools maximize the funds for improvement.

The state will use one of several rubrics – each of which match one of the options districts have in applying for school improvement funds – to determine which districts and schools receive competitive funding. The rubric appears to combine compliance and quality, and because there is no description of how competitive funds are awarded it is unclear whether districts and schools that have effectively targeted their proposed interventions to their determined areas of need will necessarily be awarded funds over schools and districts that have simply completed their application thoroughly.

The state has also committed in its ESSA plan to greater coordination across education agencies, such as with the Indiana STEM Council, Department of Child Services, and Interstate Migrant Education Council, in order to streamline interventions across educator effectiveness, school improvement, and monitoring. This is a very promising practice which could be implemented by more states. Indiana is encouraging district-wide applications for districts with four or more comprehensive support and improvement schools, and applications include a comprehensive needs assessment requirement. These efforts are intended to increase efficiencies and manage costs while still requiring appropriate interventions for each school.

Rigorous Review Process

Is the state applying rigorous criteria and review processes to ensure resources will be used to support effective school improvement efforts? Is the state prioritizing funding to LEAs who demonstrate the greatest need for school improvement funding (including LEAs with a high percentage of CSI and TSI schools) and the strongest commitment to school improvement?

Indiana has a set of well-developed, evidence-based rubrics that extensively covers key areas of school improvement. The state describes that only reviewers external to the state education agency will score the grant applications. However, there is not enough information regarding how the state will apply the rubric to applications in order to determine the highest need while also factoring in the quality of the plans, not just completeness.

Continuous Improvement, Monitoring and Evaluation

Does the state have a robust, data-driven process to monitor LEAs’ implementation of the school improvement plans within their district? Did the state establish clear milestones to ensure improvement over time, and within four years?

Indiana uses a continuous improvement process that begins with a comprehensive needs assessment and includes monitoring at the district level. Districts will be expected to use templates developed by the state to monitor school improvement plans. At the state level, however, there is not much information describing how quantitative and qualitative information will be combined to understand and make decisions based on a school’s needs and progress.

That said, the state has mapped out how it will support comprehensive support schools and districts based on the number of years a school has been identified. This information seems to take into account how the state’s role changes in improvement depending on time elapsed, but not necessarily the progress a school is making. Available information suggests that schools will share their plan with the state, but it is unclear what the state will do with that information, how they will intervene if the plan is unacceptable, and ways they will support for effective implementation. Indiana’s approach could be strengthened if there was detail describing how the state will help the district in response to its plan.

Evidence-Based Interventions

To what extent is the state mandating LEAs use evidence-based strategies in their improvement efforts? Does the state provide guidance and supports to LEAs to help them identify and implement the most effective strategies based upon their needs?

Indiana meets the letter of the law in its use of evidence-based interventions, and improvement plans must use only identified interventions that meet the top 3 tiers of evidence (strong, moderate, and promising) with the possibility of school-specific “purposeful” customization. The state does provide some guidance regarding suggested evidence-based interventions with the resource hub on its school improvement webpage. Interestingly, the resource hub is aligned to the 5 Essentials, but other existing school improvement documents are not. However, the comprehensive needs assessment and school improvement grant application reference the evidence tiers and not the 5 Essentials. In addition, the tool that crosswalks the needs assessment with the plan template does not mention evidence-based interventions. These factors make the support that the state provides less coherent than it could be.

The state held a provider fair to assist districts in identifying providers who offer evidence based practices. This is a promising practice for the state and would be very useful for districts who often don’t have the capacity to analyze differences between providers. A question remains regarding the extent to which Indiana vets participating vendors for quality.

Capacity Building and Autonomy

How well does the state articulate, delineate or set parameters around which interventions and responsibilities belong to the state, LEA and/or school? Does the state provide support or guidance to help LEAs identify and reduce barriers to school improvement? Does the state have a framework or process to support and monitor outside entities who partner with the state, LEAs or schools in school improvement efforts?

Indiana is clearly committed to streamlining and improving how it builds district capacity and supports districts and schools in implementing its school improvement process. The state has done extensive work to build a library of resources and implementation tools for schools and districts that will support building-level improvement, and is providing exemplar completed needs assessments to guide schools and districts, which is a strong approach Indiana’s plans for the provision of technical assistance, online course offerings that walk local staff through the school improvement process, and commitment to supporting local partnerships with community partners all suggest that schools and districts should be able to assume leadership and ownership of school improvement efforts beyond the terms of their improvement period. One promising practice is that the state has provided several workshops with follow up coaching for their districts on how to implement the framework and more information about the framework.

More publicly available information regarding how the state plans to actively engage the field with information would be helpful, but Indiana appears to be off to a promising start.

Engagement

Does the state require LEAs to engage with stakeholders such as parents and community members in the development and implementation of their school improvement plans? Does the state provide sufficient guidance and resources to LEAs to effectively do so, helping them foster local buy-in and promote sustainability?

Indiana’s school improvement applications require that a planning or school improvement team be formed, which must include representatives from the community. In addition, the state requires that improvement grant applications include SMART goals for ongoing stakeholder engagement, along with strategies to reach them. Family engagement and supportive school environments are two of Indiana’s 5 Essentials for improvement, and the state has provided resources related to these elements on their resource hub. It is evident that engagement is of interest to the state, but again the state can do more to link the 5 Essentials with the relevant components in the school improvement grant application.

Engagement

Does the state have a plan in place to review the school improvement efforts statewide and evaluate the impact and effectiveness? Does the state have a process in place to support LEAs and schools by enhancing their capacity to maintain their improvement efforts upon exiting identification and intervention?

At the state level, Indiana appears to have sustainability plans and practices under development. Indiana has articulated a commitment to increasing coordination within the state department in order to increase efficiency and effectiveness across divisions, which will improve sustainability at the state level. Indiana also plans to centralize key supports for schools and districts, which also increases the likelihood of sustained efforts, as these resources not depend on disparate areas within the bureaucracy to maintain.

The state’s school improvement grant application prompts districts to think about sustainability, including requiring a SMART goal for sustaining improvement and asking questions related to sustainable budgeting. In addition, Indiana specifically directs districts to consider how non-Title I funds can be leveraged after it stops receiving 1003a funds for a school. This could be improved if the state provided districts with guidance related to how existing state funding streams might apply to frequently used school improvement activities or strategies.

Overview

Strengths

Indiana’s plan deserves high marks as a model for other states—not only for its content but also for its ease in readability and clarity.

Indiana’s A-F school rating system is clear and straightforward, and it relies on a small number of important, proven indicators of student success. It effectively balances growth and proficiency, and it goes beyond test scores in reading and math.

Indiana’s school improvement strategy and exit criteria are potential models for other states. Through those sections, Indiana has committed to taking strong, evidence-based actions in schools that continue to suffer from low performance, and it has set ambitious, high bars for low-performing schools to demonstrate that they’ve made sustained progress.

Weaknesses

Much of Indiana’s plan is up in the air as it transitions to new standards and assessments and builds the results of those new assessments into its formal accountability system.

In the next few years, Indiana will revisit nearly all parts of its accountability system—its academic achievement and graduation rate goals, plus its calculations for achievement, growth, English language proficiency, and college and career readiness. Given those changes on the horizon, Indiana’s plan could have done more to explain how the state will evaluate its new data, incorporate stakeholder feedback, finalize its decisions, and maintain continuity over time.

Another large concern with Indiana’s plan is that the state does not include individual subgroup performance in its letter grade calculations, which creates a significant risk of schools receiving high ratings while underserving particular groups of students.

Click through the tabs on the left to see how Indiana scored in each category.

GOALS

Indiana’s overall goal creates high expectations for all stakeholders and focuses on what every parent wants for their child and what taxpayers and patrons expect from public education: “All Hoosier students will be college and career ready, allowing them to successfully embark on their chosen path in life.”

To achieve this, the state expects improvement from all subgroups and expects the greatest improvement from the subgroups that are performing at lower proficiency rates.

Specifically, the state established proficiency targets based on current performance of each subgroup and expects larger improvements in the same timeframe from those groups with lower baseline proficiency rates.
Indiana’s plan includes historical data on past progress, although this data suggests that its goals would require larger increases for most student groups than they have achieved in the past.

Additionally, the state’s plan notes that due to changes in the state’s assessment system and the state’s high school diploma and only a limited amount of data on English learners’ English proficiency, the goals and interim targets in the plan will have to be reconsidered in the future. Indiana also discusses its State Systemic Improvement Plan for students with disabilities, which includes a specific goal for improving third grade reading achievement for students with disabilities.

Indiana’s plan would be strengthened by aligning the accountability system with the goals more clearly, as the goals don’t appear to be used in the broader A-F school rating system.

STANDARDS AND ASSESSMENTS

Indiana is transitioning to a new computer-adaptive summative assessment, aligned to Indiana’s Academic Standards, beginning in the 2018-19 school year.

The new assessment, called ILEARN, will measure proficiency for English Language Arts, mathematics, social studies, and science.

The change in assessments means the state has little longitudinal data available. The state could strengthen its plan by providing additional detail on its plans for alignment and coherence going forward and ensuring the new assessment will be ready in time for statewide implementation. Although Indiana’s description of its assessments and the way assessment results will be used to enhance student learning opportunities are impressive, it remains to be seen what actual benefit the new assessment will bring to students.

Indiana is taking appropriate measures to assure accommodations are made for non-English-speaking students.

Indiana has adopted WIDA ACCESS for English learners as the state’s annual English language proficiency assessment. With regard to subject matter tests in other languages, Spanish will be the only language offered. Although the state considers Spanish to be a “significant” language, it does not provide a specific threshold as to what defines a significant language. The state will also offer accommodations, such as translation guides.

Indiana’s plan also mentions its alternate assessments for students with the most significant cognitive disabilities, but the plan would be strengthened if the state provided steps it would take to ensure that it does not exceed the 1 percent cap on participation in that alternate assessment.

INDICATORS

Indiana’s indicators center on important outcomes, which should focus schools on helping students meet academic targets and goals. However, some of its choices in how it plans to implement those indicators will weaken their effectiveness.

Indiana’s indicators center on outcomes including proficiency, growth, English language proficiency, graduation rate, and college and career readiness.

But while the use of chronic absenteeism is supported by research, the specific indicator Indiana has chosen—giving schools credit for improving a student’s attendance by 3 percent a year—may not provide a sufficiently clear incentive for schools to focus on supporting every student to attend regularly.

It is also unclear whether the state will count absences due to suspensions/expulsions in this calculation, but doing so would strengthen the plan. Additionally, the plan would be strengthened if the English language proficiency indicator provided more information about the student-level growth trajectories.

At the high school level, Indiana deserves credit for attempting to incorporate both four- and five-year graduation rates, but the description of the calculation is somewhat unclear.

It appears that the state is prioritizing the four-year rate and giving bonus points for improvements in the five-year rate, which is a strong approach. However, it’s unclear whether only the five-year rate would be reflected in a school’s ultimate rating, which would not be as strong.

The state’s college and career readiness measure is positive and covers high-quality indicators of readiness, including passing an Advanced Placement or International Baccalaureate test, earning at least three college credits, or earning an approved industry certification.

However, Indiana could strengthen its plan by incentivizing schools to offer a wider variety of college- and career-ready opportunities and by shifting this measure to include the entire ninth grade cohort, not just graduates, which would ensure schools were not rewarded by having higher percentages of their lower-performing students dropping out.

Additionally, Indiana should monitor its data to ensure that all of its options are comparable or to determine whether certain types of students are disproportionately likely to pursue certain pathways.

The state deserves credit for acknowledging the need to reassess its indicators going forward, especially in light of a new assessment, but the current lack of information leaves some uncertainty about how exactly this process will be implemented.

ACADEMIC PROGRESS

At the elementary and middle school levels, Indiana plans to give equal weight to growth and proficiency, and the plan includes a progress measure at the high school level.

These decisions will send a strong signal about the importance of improvement.

Indiana also deserves credit for placing a substantial emphasis on students reaching grade-level standards. The state’s proposed measure of academic achievement—the percentage of students proficient in reading and math—is clear and easy to understand.

Indiana also proposes using a relatively simple growth model called a “value table,” which gives students credit for advancing performance levels. This model is not as precise or sophisticated as some other alternatives, but it provides clear, upfront signals to students, parents, and teachers about what constitutes sufficient growth in a given year.

While not explicitly a growth-to-standard model, this growth measure does implicitly reward schools that help students make continued progress every year.

The downside of this model is that it creates rigid cut points at the borders of the performance levels, and schools may choose to focus on students right on the cusp of a given level. Moreover, Indiana’s specific value table measure is somewhat concerning, since in certain cases it appears to give less credit for lower-performing students who make progress relative to higher-performing students, which could potentially create unfortunate incentives.

ALL STUDENTS

The way Indiana plans to include subgroups will likely allow schools with low performance for individual groups of students to still be rated highly in the state’s accountability system.

Indiana is planning to base its A-F school grades on school-wide averages, and it will not include the performance of individual subgroups as a separate weight in those calculations. The state will report on subgroup performance and use those scores to identify additional schools in need of targeted support, but the scores will not factor into school grades.

Indiana does use subgroup performance to identify additional categories of schools. In particular, it will define a school with persistently underperforming subgroups as a school with one or more subgroups with an overall accountability score at or below the lowest-performing 5 percent of schools for at least two consecutive years. However, the state has not provided data suggesting how many schools this rule might capture.

Indiana’s decision to only consider students enrolled at the school for 90 percent of the school year as part of the proficiency rate calculation could end up excluding significant numbers of students from this calculation.

Indiana deserves credit for a few of its decisions affecting subgroups.

For one, it will lower its minimum threshold for subgroups from 30 to 20 students. Additionally, the state includes a strong rule on participation rates. Indiana will multiply a school’s academic achievement results by its participation rate. If a school meets at least 95 percent threshold, the multiplier is one. Anything less than that reduces the school’s percentage of students attaining proficiency, which sends a strong signal that schools should make every effort to include all students. The plan would be even stronger if it included consequences for schools that miss the 95 percent participation threshold, overall or for particular subgroups.

Indiana is planning to slowly phase English learners who have recently arrived into the United States into the formal accountability system.

Those students will be tested in year one, but the results will not count for accountability purposes. In year two, their scores will only count toward the state’s growth measure, and those students will be fully included in their third year. The state is also proposing to include former English learners in the English learner subgroup for up to four years after they are no longer receiving services. Since exiting students tend to have higher performance, the state should monitor its data to ensure it is not masking the performance of students who are still receiving services.

IDENTIFYING SCHOOLS

Indiana’s identification system is likely to capture the lowest-performing schools overall, but it is possible that schools with low subgroup performance will still receive high ratings and may not be identified for support.

Since Indiana will identify both schools with F grades and those in the bottom 5 percent overall, as well as high schools with low graduation rates, the system is likely to capture the lowest-performing schools, even if they are not rated F.

Indiana has a clear method for identifying targeted support schools, but because Indiana does not include subgroup performance in the A-F grades, schools with low subgroup performance may still receive high ratings and not be identified for support.

Additionally, the state’s accountability and assessment systems would allow schools with low subgroup performance to avoid serious consequences until the 2022-23 school year. That seems too slow, and the state would strengthen its plan by adding more urgency to the system.

Indiana will also create a list of “at-risk” schools that fall outside the lowest-performing 5 percent of Title I schools and secondary schools with graduation rates between 68 and 70 percent.

This will provide a warning to schools that, though they face no consequences immediately, are on the verge of being identified for improvement activities. Indiana could further strengthen its plan by asking schools to draft improvement plans, or by making additional technical assistance available to these schools to prevent them from slipping into comprehensive support status.

SUPPORTING SCHOOLS

Indiana has a strong plan to support its identified schools and should be a model for other states.

The plan includes specific state- and district-level roles and responsibilities, from needs assessment to planning to selection of interventions and supports. Indiana’s criteria for making grants for school improvement are also clear and rigorous, and the state’s strategy for supporting districts with multiple low-performing schools is particularly promising.

The state seems to have well-reasoned courses of action for comprehensive and targeted support and improvement schools.

The plan outlines the state’s efforts to help schools and districts, and it appears to be comprehensive and helpful (e.g., providing templates, outlining evidence-based approaches, providing models on how to use data to improve schools, offering targeted professional development, and facilitating technical assistance from partners). Additionally, the state has built in stakeholder engagement as a key element of the comprehensive needs assessment.

Indiana will first use its 7 percent of funds set aside for school improvement activities to award planning grants to all comprehensive support schools in their first year of identification.

The planning grant funds will be used to conduct a comprehensive needs assessment, develop a school improvement plan, and “ensure the required conditions (e.g., leadership at all levels, academic strategy, student supports) are in place to enable successful implementation” of the school improvement plan. The state will develop a model comprehensive needs assessment aligned to an evidence-based framework for school improvement; provide a recommended protocol for planning and conducting the comprehensive needs assessment, including strategies for stakeholder engagement; and define an optional menu of supports for districts and schools to support their planning and/or implementation of one or more sections of this recommended protocol.

After the initial year of identification, additional grant funds will be awarded on a competitive basis to applications using strong, evidence-based interventions. The state also will run a special school improvement grant competition for districts with four or more comprehensive support and improvement schools, which will help districts with particularly intense challenges.

For schools that continue to be identified for comprehensive support for four years, Indiana has a set of statutorily required steps directed by the state board.

The board will hold public hearings and hear testimony concerning the following options for school improvement: merging the school with a nearby school that is in a higher school performance category under Indiana’s school accountability model; assigning a special management team to operate all or part of the school; approving the school district’s plan to improve the school through the creation of a transformation zone; approving the school district’s plan to improve the school through the creation of an innovation network school; implementing the state department of education’s recommendations for improving the school; examining other options for school improvement expressed at the public hearing; and closing the school.

The board of education has the authority to decide which intervention a school must pursue and when it should pursue that intervention. It can delay for one year if students are making progress.

The state could strengthen its plan by indicating if and how it intends to provide direct student services using the optional 3 percent of funds set aside, which provides an additional opportunity for the state to align school improvement activities with its statewide goals.

EXITING IMPROVEMENT STATUS

Indiana’s plan for schools to exit improvement status is a potential model for other states.

Indiana’s exit criteria are quite rigorous, requiring that comprehensive and targeted support schools reach at least a C grade for the entire school or the low-performing subgroup, respectively, for two consecutive years before they are eligible to exit improvement status. This is likely to represent significant improvement, and maintaining that level for two years should help avoid the risk of backsliding after only one year of improvement.

Additionally, the state says the school must “demonstrate a strong plan for sustainability.” This is a good idea, but the state could strengthen its plan by further defining how it would monitor what constitutes a “strong plan for sustainability.”

CONTINUOUS IMPROVEMENT

Indiana has outlined a plan to modify and add to its accountability indicators over time based on feedback from stakeholders, including piloting culture and climate surveys.

The state seems very involved in assisting schools and districts in the improvement process, both at the beginning of the planning process and during plan implementation. The state appears to have a number of built-in supports and information for the schools and districts in need of support.

However, Indiana could do more to describe how it plans to adjust its statewide system based on data, the schedule, and how their analysis be reflected into a continuous feedback and improvement loop. Much of Indiana’s plan is in flux until the 2018-19 school year or beyond, so regular and ongoing review and adjustment will be of critical importance.