Some members of the Jeffco School Board frequently state that they are making decisions based on data. On more than a few occasions, we’ve heard the board’s majority, along with their supporters, cite test scores in the district as a core issue and a fundamental driver for pushing a “value-added,” or better known as “pay for performance”, compensation plan for teachers. Unfortunately many of these key assumptions are based on misleading data and logical leaps that are difficult to substantiate or support, and put into doubt the efficacy of a new compensation plan. More importantly, by attributing blame to teachers for student achievement on test scores, we fail to look any further at the underlying root causes, like poverty. The end result is that we all lose: good teachers will leave, student achievement won’t improve, political divisions will fester and grow, and nothing will be solved.

Remediation Rates

“In 2013, Jeffco’s high school graduation rate was 82%. However, 29% of Jeffco graduates attending public colleges in universities in Colorado had to pay for non-credit remediation courses, which usually don’t work. In Colorado, only 22% of students who take a college remediation course graduated with a bachelor’s degree within six years (see, “Remediation: Higher Education’s Bridge to Nowhere” by Complete College America).”[1]

The assertion is true… to a point. First, it is true that 29% of Jeffco graduates attending public colleges and universities in Colorado required additional remediation courses. A little more context: the remediation rate for all Colorado graduates attending the same colleges and universities was 39.2% for 2011. When compared against other districts in the area for the same year, Douglas County was slightly lower at 28.1%, Cherry Creek was 3.6 percentage points higher at 33.4%, and Boulder Valley was 6.7 percentage points lower than Jeffco at 23.1%. Even with this, it’s not a complete picture yet. Some further context. How does remediation rates compare with previous or subsequent years? The Colorado Department of Higher Education provides remediation rates from the years 2009 – 2012 for each school district within the state (subsequent years are not yet available)[2]

Jefferson County

Douglas County

Cherry Creek

Boulder Valley

State of Colorado

2009

31.7%

25.9%

31.5%

19.4%

38.1%

2010

32.7%

28.1%

31.8%

24.9%

40.5%

2011

29.8%

25.6%

33.4%

23.1%

39.2%

2012

27.2%

27.0%

28.6%

18.2%

36.0%

Using the table above, we can see that Jeffco’s remediation rate is consistently 7 to 10 percentage points lower than the state and only slightly lower than Douglas County. Secondly, both the district’s and the state’s remediation rates have dropped year-over-year since 2010. Jeffco’s drop is higher than the state’s: 5.5% versus 4.5% – a full percentage point higher than the state average. Only Boulder Valley showed a larger drop at 6.7% over the same period. This suggests that Jeffco’s teachers are doing a good job of addressing the gaps. Of course, we’re not where we want to be, but we’re making good progress.

Also interesting is the variation in remedial rates between districts. Compared to Cherry Creek and Douglas County, Jefferson County remediation rates are very similar. The only standout is Boulder Valley. To understand why, we can look to a growing body of research that shows strong linkages between poverty and learning[3]. Using annual free or reduced data for the same districts over the same time shows the following[4]:

Jefferson County

Douglas County

Cherry Creek

Boulder Valley

State of Colorado

2009

29%

8%

24%

17%

38%

2010

30%

11%

26%

18%

40%

2011

32%

11%

26%

19%

41%

2012

34%

11%

25%

19%

42%

Jeffco’s FRL rate is noticeably higher than the other districts, yet our remediation rates are dropping faster than Douglas County or Cherry Creek. In that context, Jeffco is doing more.

To the point about graduation rates, the percentage of students completing with a bachelors (22%) are certainly a concern. Yet, the inference, again, that our schools are culpable requires the reader to make a large logical leap. Why didn’t they finish? While are the values are certainly low compared to the national average of roughly 53% of all college students completing a Bachelor’s degree in 6 years, why does Massachusetts have a 51% completion rate for students requiring remediation, why is New Mexico’s only 7%[5]? Are there differences in the structural supports at the higher education level that vary between different states’ higher education systems? What are they doing differently to retain these students? How do these numbers compare with previous years? Are they up? Down?

The bottom line is that the referenced document in question provides no evidence relating the performance of Colorado’s public schools with completion rates either, so while there appears to be correlation, I see no evidence of causation.

Let’s consider something else that might factor into this equation: By 2011, the state of Colorado reduced funding for its colleges and universities by 69 percent from $10.5 billion in 1980 to $3.2 billion in 2011 – in 2011 dollars[6], that $10.5 billion is equivalent to $28.6 billion[7], more like a nine-fold decrease adjusted for inflation. Meanwhile, the cost of attending has gone up. For example, since 1980, the tuition rate at CU Boulder has increased thirteen-fold from $694 per year in 1980 to $9,048 in 2014[8] – even if we adjust for 2014 dollars, tuition has gone up roughly 450%[9] during that period. Compare that with surrounding states like New Mexico, Utah, and Nebraska, where their in-state tuition has less than doubled since 2005[10] adjusted for inflation. While I wholehearted agree that remediation is an obstacle for completion, Colorado’s higher education institutions aren’t necessarily helping to remove obstacles either. Certainly more research will be need to determine if cost is a factor, but I am skeptical to draw conclusions based on one data point without additional evidence.

TCAP Results

“The state of Colorado has established academic standards for different subjects and grade levels. Every year, our children’s mastery of those standards is assessed using the TCAP tests. If they achieve proficiency at each grade level, they should also meet the college and career readiness standard on the eleventh grade ACT, and not need to take remediation courses in college. Unfortunately, the percent of Jeffco students who are proficient tends to steadily decline from grade to grade. For example, in 2013, 74% of all Jeffco third graders were either proficient or advanced in math; by tenth grade, this had fallen to 43%. This was not a function of poverty: for students not eligible for free and reduced lunch, the fall was from 83% to 52%. For F&R students (about 34% of Jeffco students), it was much worse: from 57% to 20%. For special education students with IEPs (about 10% of Jeffco students) the fall was from 36% to 4%. And for gifted students with ALPs (about 11% of Jeffco students), the percentage scoring advanced in math fell from 79% in third grade to just 35% by tenth grade.”[11]

Again, this statement is partially true. It is indeed true that TCAP Math Proficient/Advanced Proficient percentages drop from 74% in 3rd grade to 43% in 10th grade, but it’s cherry-picked from other relevant data. Let’s compare that with state numbers, and comparable districts[12]:

District

3rd Grade Math Proficient/Advanced

10th Grade Math Proficient/Advanced

Percent Change

State of Colorado

72.3%

38.8%

33.5%

Jefferson County

73.7%

42.7%

31.0%

Denver County

59.3%

24.6%

34.7%

Douglas County

81.7%

45.3%

36.4%

Not only does Jeffco see a drop in proficiency from 3rd grade to 10th grade. The entire state experiences a big drop, as does Denver County, a similar-sized district. Likewise, even districts like Douglas County with a dramatically lower poverty rate than Jeffco has the same kind of drop, but at a rate 5 percentage points higher than Jeffco. Context matters.

Pointing Fingers

“This pattern has not changed in the eight years for which we have public data, which includes the period before Jeffco implemented budget cuts. Money does not appear to be the problem – and we’re still spending a billion dollars per year. To be sure, most of us have heard of individual student success stories in Jeffco, and that some of our schools have won awards (though some of these, like the James Irwin Award, seem to reflect favorable student socioeconomic circumstances, and not true school value added). However, these favorable anecdotes pale in comparison to the systematic data that describe Jeffco’s excruciatingly painful eight-year achievement track record.”

I don’t think I am misinterpreting when I read this as suggesting that teachers are the core problem. However, arriving at this conclusion requires evidence that just isn’t there. We’ve already seen that simple statistics comparing one grade to the next is problematic. Yet the reliance on assessment scores as a measure of our teachers’ effectiveness is fundamentally flawed on many levels.

First, there is a growing body of research that suggesting that the relationship between teacher quality and student test scores is tenuous at best[13],[14]. Additional research questions the efficacy of standardized assessments ability to measure what children have learned compared to what they remember[15], and others have raised questions to the reliability and validity of the tests[16]. Finally, TCAP and other standardized assessments, ACT included, often have built-in biases that are reflected in test results. For example, minority status, English language proficiency and family income all impact a student’s test scores[17],[18],[19].

Underneath the aggregated statistics for the district test scores is a much more detailed and richer story that provides substantially more context. Jeffco is very large, very diverse and encompasses a wide range of racial and social-economic circumstances that are reflected in the demographics of our schools. There are schools that are nearly 90% minority, have over 50% of students whose native language is not English and have over 90% of the students who are from low income families. Conversely, there are other schools in the district that are complete opposites. All of these socio-economic factors have significant impacts on test scores. Does a school’s minority, non-English proficient and/or lower-income student demographics impact test scores? If so, how much?

Using data from the Colorado Department of Education’s Data Lab[20], I collected TCAP scores from every school in Jefferson County for grades 3 through 10 for the 2009 through 2014 academic years. Additionally, I enhanced the exported data with demographic data from Jeffco Public Schools[21] for each school including the following:

Percentage of Minority Students

Percentage of Students receiving Free and Reduced Lunches to be used as a proxy for low-income students

Percentage of ELL Students

Percentage of NEP/LEP Students

Articulation Area for each school – I use this information to group results since neighborhood schools are located in the same proximate geographic area, which is also closely aligned with other socio-economic variables. For schools not assigned a particular High School articulation, such as Option and Charter schools, these were grouped into their own categories: Option and Charter respectively. Similarly, Jeffco also operates and provides education services for students with non-traditional needs, where data is available, I include these schools in a “Special Services” category”

For each row entry, I identify the school and enter the demographic data specified by the variables above. Additionally, for each row, I calculate a new variable called Not Proficient Percentage the adds the entry’s Unsatisfactory Percentage and Partially Proficient Percentage. This will be used to contrast the CDE’s Proficient/Advanced Proficient Percentage which adds the Proficient Percentage and Advanced Proficient Percentage values.

There’s quite a bit of data here making it difficult to identify clear patterns. To simplify, here’s a graphic representation that charts the percentage of students who did not achieve a proficient or above score against the average free and reduced lunch rates, NEP and LEP rates, and percentage of minority students by each articulation area:

There is a clear pattern between achievement results and socio-economic variables like free and reduced lunch, NEP/LEP rates, and the percentage minority students in that articulation area. The red bars represent the percentage of students who did not achieve a proficient rating or higher on any of the assessment subjects. The data is sorted by the articulation area’s FRL percentage. For example, the Jefferson articulation area has the highest percentage of students eligible for free and reduced lunch; Ralston Valley has the lowest. Based on these values, we can see a clear pattern where higher FRL rates produce similarly high “Not Proficient” rates.

Going a step further, I found that each of these socio-economic variables show a strong negative correlation with the percentage of students achieving proficient or above on any TCAP test:

Minority %

-0.562

NEP/LEP%

-0.447

FRL%

-0.647

Further analysis of these factors using ordinary least squares regression methods[22] shows that the percentage of students eligible for free or reduced lunch in each articulation area significantly impacts the percentage of students who achieve proficient or higher on any TCAP assessment. Interestingly, the percentage of students who are identified as minority and whose native language is not English do not significantly impact test scores when free and reduced lunch rates are included in the model. However, when FRL percentages are removed, the percentage of minority students does significantly influence test performance, which indicates that the rate of poverty and the percentage of minority students are highly correlated[23]. The following table shows that results of the regression analysis:

Variable

Composite

Math

Reading

Writing

No FRL%[24]

FRL% Only[25]

Free and Reduced Lunch%

0.708***

0.839***

0.665***

0.621***

—

0.535***

Minority %

-0.290

-0.041

0.235

-0.078

0.871*

—

NEP/LEP %

0.054

-0.431

-0.362

-0.031

-0.693

—

Intercept

0.143***

0.155**

0.083**

0.189***

0.105

0.122***

Adjusted R2

0.896

0.871

0.914

0.901

0.486

0.875

N

20

20

20

20

20

20

*p<.05; **p <.01; ***p < .001

What Do The Results Mean?

Jefferson County is a very diverse school district encompassing a wide range of socio-economic circumstances. For example, some articulation areas like Evergreen are performing extremely well while others like Jefferson in Edgewater have a large majority of students who are falling behind. The demographics of these areas, combined with test scores are quite stark and revealing: Evergreen has a very low poverty rate as measured by the percentage of students eligible for free or reduced lunch (13%) while Jefferson has a free or reduced lunch rate hovering around 90%. This is reflected in the test scores as well. Using the model above, we can expect that an increase of 10 percentage points in the FRL rate will result in a 7% decline in the number of students who achieve proficient or above on TCAP assessments. Not only does the data show that poverty significantly predicts test performance, the results show that poverty is a substantial factor – over 80% of the variation in test score performance can be explained by free or reduced lunch rates. This shows that students’ personal circumstances, like poverty, play an enormous role in test outcomes. These results clearly show that assigning culpability to teachers’ effectiveness for students’ test results is unwarranted.

Conclusion

The board’s majority repeatedly use statistics to defend their positions and to drive their political agendas and policies. Many of these statistics they have based their conclusions on are partially true at face value, but are taken out of context. For example, remediation rates for a single year can be quite alarming by themselves. Put into context with other years, we see a completely different picture: They’re dropping, and they’re dropping faster than the rest of the state, and the trend appears to occur nationally also. Similarly, TCAP test values are also taken out of context. Certainly, 10th grade test scores are much lower than 3rd grade test scores. However, when these scores are compared statewide and with comparable districts, the story changes – the pattern isn’t unique to Jeffco, and it’s generally less severe. According Tom Loveless, a senior fellow at the Brookings Institute Brown Center on Education, the answer for these drops is unclear:

“”(The reason) could be something very profound,” Loveless says. “It could be something cultural about the way we school kids in the United States, or maybe in terms of our expectations for teenagers that they’re just not as academic as they should be. But all of those are guesses. We don’t really know.””[26]

More importantly, members of this board’s majority have used this data as a wedge issue for using assessment data to evaluate teacher effectiveness. They make conclusions about teacher quality by attempting to connect the dots between scant pieces of data, but provide no tangible, significant evidence of any relationship between teacher effectiveness and Jeffco student achievement on TCAP/CMAS assessments. The results here show there is strong evidence that the root cause can be explained by external circumstances like poverty. Anecdotally, I’ve spoken to teachers with students eating their only hot meals at school. This is a profound reality for some kids in Jefferson County. For example, food insecure families at Molholm Elementary, one of most economically disadvantaged schools in Jefferson County, fast 17.5 hours a day[28]. Recent research is providing insight to how nutrition plays a powerful role in cognition and learning[27], that may potentially reflect how well students perform on standardized tests.

These results have significant policy implications. The net effect of assuming teachers are the root cause of our schools’ test performance is misguided. Teachers aren’t the problem. Yet, the board’s current majority continues to pursue policies that puts teachers in the cross hairs for student test score achievement. This has resulted in creating deep political divisions in the community, some of which may leave permanent scars. In a recent poll taken by the Denver Business Journal found that 56% of businesses in Jefferson County have been adversely affected by recent actions of this school board[29]. McKinsey and Company, a prominent business consulting firm, found that if the gap in performance between low income students and the rest were narrowed, our nation’s GDP would have increased as much as 3 to 5 percent, or $400 – $670 billion dollars[30]. These aren’t trivial numbers. Instead of attacking teachers, and more recently parents and students, the board should refocus its efforts by leading to find real solutions.

Imagine if this board worked with local government and business leaders to find ways to raise up the poorest families in our district rather than tear down our biggest asset, our teachers. Imagine if this board took a proactive approach to helping principals, teachers, parents work together to address local school concerns rather than assaulting them financially by reallocating our tax dollars we agreed should go to the schools. Imagine if this board listened to and collaborated with the community, we would do the heavy lifting. Imagine how good our schools could be. Sadly, right now, few of us see this vision becoming reality with this board’s majority, and we end up paying for it when good teachers leave.

About the Author

I am a software professional with 20 years experience living in Jefferson County for over 14 years. I am also a public school advocate, a volunteer and a parent. I have a Masters degree in Sociology with an emphasis on Political Economies and have taught courses in quantitative analysis and research design.

5 Responses

Thank you for your deep dive into the student data especially around the roadblocks to success that our students in the Jefferson Articulation Area experience.

We have already started on building a broad coalition of community stakeholders in the greater Edgewater area to do just what you explained in the last paragraph. Teachers and school staff are an integral part of the solution that we are all working toward.

Joel, this exactly the kind of thing I’m excited about. It needs to be modeled and duplicated across the entire district. Your work with the Edgewater Collective should spur our board to work more closely with community, government and business leaders.

You are assuming, incorrectly, that everything that is wrong with Jeffco schools is attributable to the new board majority. This has been a 12 year slide with control exerted over jeffco schools to direct them in the wrong direction. What is now being taught in 3rd grade used to be taught in 2nd grade. We have dumbed down our kids ourselves by falling asleep at the wheel. The new board majority is trying to right the ship, but it is almost impossible when they have to fight the other two members of the board constantly. They refuse to consider compromise. Then there is the union inciting teachers and children to fight them on non-existent issues. The disgraceful behavior by teachers at school board meetings shouting abusive phrases at the board is disturbing, disruptive and antagonistic, to say the least. I have sat next to teachers who have done this and been congratulated by their friends. Maybe this is what is wrong with out schools, if this type of behavior is acceptable for teachers, what are they teaching in the classroom. What power there would be for everyone, teachers, students, parents and school board. If you people would put that energy in working WITH the board to find solutions. Instead of fighting them at every turn and assuming, incorrectly, that they are to blame for all of the problems that have built up over the past 12 years. Of course you are not taking into effect the impact of Common Core. Kids will all achieve at the same level. Bright kids will assist the more challenged students. Teachers will all be paid the same across the country, you won’t need any kind of performance measures, you will get a raise when the government says so. Maybe you should focus your energies on defeating Common Core instead of the school board.

The core thesis of this article is very clear: The board’s decision to implement a value-added compensation model is fundamentally flawed. It assumes that all students, ceterus parabus, will perform the same, and that teachers are the root cause for diminished achievement. The research cited here, along with data from Jefferson County shows that children from economically disadvantaged families have significantly lower rates of achievement than children from more affluent families. The basic premise for linking test performance to teacher effectiveness is misguided at best and insidiously intended to push effective teachers out of the classroom (and district) at worst.

There is nothing in this article that speaks to Common Core. It’s not relevant to the discussion since the underlying issue of pervasive poverty will continue to impact student achievement irregardless of curriculum format or testing frameworks.

With respect to Common Core. Let’s make a clear distinction between the ~standard~, and curriculum and assessment components. Would you agree that third graders should have a solid grasp of multiplication and division? Would you agree that 8th graders should clearly understand integer math and linear equations, and that 10th grade, have full command of binomial equations and geometric proofs?

If you say yes to any of these, then I think you’ll find alignment with the Common Core State Standards (see http://www.corestandards.org/). With regard to _implementation_ of the standards, we certainly can have more discussion. Nevertheless, let’s be clear that Common Core, per se, is not the issue.

Back to my original thesis: You can have the best curriculum and even the best teachers in the world in the classroom. However, if children are going to school without having the most basic needs met, then no amount of heroic effort by teachers will affect the ability to learn until we can fix the litany of issues associated with poverty including food and housing insecurity, and the most important part: self dignity. So I would urge you and others who are quick to point fingers at teachers to see for yourselves what it means for some kids who won’t have another hot meal until next Monday. Ask them why the 2nd grader can’t finish their Tumble Books. Ask the 8th grader why they didn’t complete their IXL math. Ask them how many jobs their parents or guardians have and why they’re not home to help with their homework. This winter, ask a kid why he’s wearing a only sweatshirt in 15 degree weather. This is a reality for quite a few children in Jeffco. So if you want to see student achievement rise, address poverty, and overcrowded schools, like Stein Elementary, which is at 172% of capacity. Until that happens, no amount of sanctimonious scorn toward teachers will fix anything – it just makes it worse.

YourHub on Twitter

YourHub on Facebook

YourHub is The Denver Post’s online local news source and weekly print section. This website is for community contributions. For YourHub staff content, go to our Denver Post website at denverpost.com/yourhub.