Follow me on Twitter

The Signal and the Noise

There are the known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns–there are things we do not know we don’t know. –Donald Rumsfeld, Secretary of Defense under George W. Bush

Donald Rumsfeld made these comments in response to the possibility of weapons of mass destruction in Iraq. He got a lot of flack and mockery for it (the flip-floppery of the words and their utter ridiculousness on the surface seemed to encapsulate the Bush White House response on everything, even if the core idea was valid) from an American people that were post 9/11 afraid, sitting on the cusp of a new, violent world. As Iraq proved, there were plenty that we did not know we did not know.

Nate Silver, the founder of the great sports, politics, and economics data site FiveThirtyEight, also wrote a pretty good (a bit long) book The Signal and the Noise. It’s worth the read as it will make you think about data in new ways, and really question the assessments we do (and the assumptions we can and cannot make). In the chapter “What You Don’t Know Can Hurt You” he looks at intelligence failures, including 9/11 and Iraq, and delves into the idea of the “knowns”. The breakdown is interesting and important:

Known Knowns: You know the problem and have the answer. You know you need enough chairs for your class to sit. Thanks to a class list, you know how many kids will be in your class.

Known Unknown: You know the problem but do not have the answer. This is the first day of class. You know the material you have to cover. Unknown are the skills students bring into the classroom: Can they even read the text you are counting on? Unknown are the personalities: Do they like to learn or are their “issues”? These questions x 1,000. Schools combat this unknown with assessments and data; a good administration will give teachers access to databases or just include basic data in your class list (my first job included DRP scores and IEP designations with the list). Elementary schools spend a lot of time crafting classroom balance when moving kids grades, and reporting on each child before the new teacher takes over. High schools have more informal information exchanges, in the teachers’ room over coffee or, later, in a bar over drinks. Schools recognize this problem and use data to solve it as much as they can (caffeine and alcohol are mere balms).

Unknown Unknown: “A contingency that we have not even considered,” writes Silver. “We have some kind of mental block against it, or our experience is inadequate to image it; it’s as though it doesn’t even exist.” (421) There is a reason we pay experienced teachers more. If you’ve had a student teacher recently, or mentored a new one, you can see they have no idea what lies ahead. Not only do new teachers not realize it can take fifteen minutes for a student to find a pencil, they have NO idea what the home life of many are and how unimportant Poe’s “The Murders in the Rue Morgue” is at that moment in time. Who knew? There is a reason so many teachers quit in the first three years.

* If we could get parents, school board members and others to teach for a month the entire tone around teacher negotiations would change. There is nothing more frustrating (I would argue a microagression) than a non-teacher suggesting a lesson. “If you would just….” Thanks, no.

Unknown Known: As veteran and studied educators, we know the results but not the problems (we do make judgments, though–I used to blame middle school teachers for what my 9th graders couldn’t do, until I became a middle school teacher (I try not to blame the elementary teachers)). I remember a few yeas ago, I got data showing that nearly 25% of my incoming students were illiterate. Easily, one-third are below grade level. That state is known. The problem is not.

Silver does not talk about this–and I have no doubt I’m getting the binary wrong and this state of mental organization doesn’t even exist–but for me the “unknown known” seems to be the blind spot in education. Instead of looking at a known problem (kids can’t read) and the unknown solution (Why? How can we fix it?) we should be looking at how we got here.

The Difference and How It Helps

In our district, we are pushed to look at where the student is and solve the problem. A student can’t read (known) we teach them how (unknown how, but solve it over 180 days). That’s the school year in a nutshell. Next year, repeat.

The problem is that it is reactive. Ten (more?) years ago our district went full RtI (Response to Intervention). For at least two school years (a lot time for many initiatives to last) we talked about using data (then, a new idea) to drive instruction (an even newer idea). If Johnny did not know his alphabet, someone would take him aside and drill it; then he’d be with the class and ready to push on as an equal. It is a great idea. It reminds me of herding stray sheep to keep the whole alive. We even got these great laminated folders that detailed much of the philosophy and protocol (I kept mine–it is so clear–while I’ve dumped my share of other such initiatives and supporting materials). Unfortunately, RtI got watered down by the differentiation push that followed it. Plus, because PBL (Performance Based Learning) had not yet happened, those teachers in the upper grades complained that their content was too complexly woven together to do a simple intervention (PBL and targets takes some of that argument away–just teach a focused Evidence group, for example, if that’s their weak spot).

The unknown known is not about the student in front of you. It is about the path that brought them to this moment. You know the result: one-third of my class struggled with literacy. I don’t know the problem: For some reason, a large number of students could arrive at middle school without being able to read, but I don’t know why. We have good teachers in the younger grades. We have resources. It is unknown how we got here.

In putting the unknown first (unknown known) we focus on the system, not the individual. In this instance, I am not looking at my students but those who are coming up. In theory, if I can know that unknown those coming into my class in future years will not have this issue–they will be able to read, and I can focus on bringing them up even higher.

The Power (and Blind Spots) of Linear Thinking

Semantics? Perhaps. But there is a lot to be said about linear thinking. Our school is dogged by linear thinkers that cannot see the complex interconnectedness that is life (and teaching). They often hold back discussions and real change because they cannot see how fixing C before B helps get to M–and we all get bogged down. But linear also clarifies. In thinking about what lead up to this moment, our solutions look to the future.

The question to ask is simple: How did we get here? It is one we rarely have time to address because, teaching. Those one-third in my classroom right now need me. Those two-thirds need me, too. Someone, though, needs to be thinking about how we got here.

But we know the unknown unknown (confused or just meta?). Ten years ago, I sat in a Literacy Committee meeting and heard from the kindergarten about this group. The next year, the first grade told us about them. By the time the third grade teacher reported out about “this group” I asked what we were doing about it. Nothing. Blank stares. Crickets. Then, we threw extra resources at it. They got better. To the fourth grade teacher, this was problem: solution. For me, though, that group was known but I had no idea why. When they got to me, I knew plenty.

Root out those linear thinkers who bog down every other discussion and put them to work. One a wall in a conference room put two lists: Cause and Effect. The latter is what we know (literacy). Charge them with solving the cause.

Two years ago our administration told us not to use gender in our classroom. It had come up when students were asked to pack-up the room for the day–boys putting away materials, girls stacking chairs. It was one of many daily sorts we do, and students self-designated their gender and responsibility, but one student who was questioning their gender felt stuck. We were asked to look at other data when we make groups, be it classroom chores or placement in classes. It was a solid decision that moves us forward in a number of issues–read my post about “the boy problem” here.

When told that gender was no longer being used for groupings, I asked if this was going to be true for sports, too.

It is a complicated issue, but it gets at the heart of problem with using gender as a designation–it offers no path to a solution, except if the problem is gender discrimination.

For the sake of argument, let’s use the stereotype that “boys are physically bigger and stronger than girls” because this has a basis in data. We line up everyone who wants to play basketball and find that for 80% of boys that is true. If a co-ed team was created, 80% of the A-team players would be boys. What, then, to do about the other 20%?

If we stick to gender, we are going to fill the team with sub-par specimens instead of the best players–period–filling up those roster spots. From a tactical standpoint, the coach would want that 20% of stronger girls. A co-ed team.

Let’s add some complexity–skills. Teams have smaller, quicker players with skills that trump size and strength. If the roster was filled with the best players–if the coach was able to evaluate without taking gender into consideration–the team would probably be a diverse group, physically. From a tactical standpoint, a good coach would want the best fifteen players on their squad–size and strength being only one factor.

Note that in that last scenario, other than a concern about discrimination, gender has moved to the side.

What, then, is the issue? Let us take plain bias in evaluating talent off the table–it is a huge one, but this will allow us to look at other, overlooked issues.

The first is equity. Our school has four basketball teams–Boys A, Boys B, Girls A, Girls B. If we went co-ed we could simply have A, B, C and D. Extending the above, let’s assume Team A has an 80:20 split of boys: girls. Let us further assume that Team B has a more equitable ratio, if not the former Girls A 80% taking up the majority of the Team B spots.

Does Team A being mostly boys and Team B being mostly girls create inequity? Typically, our Team A goes to more tournaments, gets the new uniforms, and has a more committed coach. Team B is more developmental–and I would assume Team C and Team D would be more so. As the majority of girls are on the lower teams (even on Team B), the majority of girls would get less. At least with a Boys A and a Girls A schools can easily count dollars spent, games played and the like.

The real issue here is the purpose of the sports program in the first place: The eternal debate–winning vs. participation. For those hoping to be the best, they need to play the best. On the court, you want the best players regardless of gender. Those who are not in the top fifteen need development.

Equity means respecting development. Players on Team B should be striving to earn a spot on Team A. Instead of focusing on winning games, though, that program needs to focus on development of the player. This requires participation and good instruction. Equity falls away when players are no longer pushed. Team C and Team D should be the same, even as they are even more elementary in terms of skills and development.

When players are on the team where they are, the system is equitable.

Bias. Of course, this is only possible when bias in evaluation is taken off the table. But parents get ugly when it comes to sports. When teams are by gender, parent after parent still finds a reason their kid is being held back, not on Team A or riding the bench too often. One coach a town over, after a win, was confronted by a spreadsheet wielding parent, recording time played by each player and a quibble over a two minute variance (the coach scheduled roles and times prior to the game, except the fourth quarter so he’d have flexibility if the game was close). He moved to absolute equity the next game, lost, but no parents complained (the players were not as happy). Whoa the burden coach’s kid actually being good, but constantly being told they got their spot because of bias.

Add gender and the result is explosive.

In the data world they call this issue “the signal in the noise.” The signal is the problem–finding the best players and playing them to win–while biases are the noise. We identify gender as an easy way to categorize people. We notice it. Evolutionary, we are built to recognize patterns as a means of survival. But our intuition can cause us to fall to, create and reinforce stereotypes. We create more noise, and lose the signal. Every stereotype has some truth at its core, but it ultimately binds the person it is being done to–we put the person in a box. And, it makes others blind to the real problem and its solution.

In the case of those 20% of boys who do not get spots on Team A because there are girls who are better, gender now becomes a factor. A battle to be fought. This is where cries of political correctness and reverse discrimination become issues, not what is best for the players.

The issue, then, is dealing with this bias. Notice how, over several paragraphs, we have moved away from the stereotype of boys being more physically able than girls and are now talking about bias and equity. Dealing with equity is hard. It requires education and community support. It requires a commitment, so that it becomes, over time, the norm. “You are on Team B because you need to develop skills X, Y and Z.”

That we shy from implementing this tells us something about our values as a school and community. Schools need to set the standard against bias in all forms. They cannot do this when their institution underscores this in the group representing them to the larger community–wearing uniforms with the logo across their chest and being photographed for the paper.

The way forward. There are plenty of obstacles to move forward. Looking at basketball, girls use a different ball. But people adapt. Until seventh grade girls playing recreational lacrosse did so with boys, with full contact rules. When they were segregated, playing by girls’ limited contact rules, most felt it was a step back. These girls were ready to hit.

One step is to desegregate those sports without such conflicts. There is no reason I know to have separate cross country races for boys and girls. Wrestling, golf and typically single-gender sports (football, field hockey) should be gender-free, too.

It should be noted that, through most of schooling, the physical size and natural abilities waxes and wanes. A small kid one year comes back from summer break having grown half a foot. The kid with no balance suddenly catches up. As educators, we should be embracing a growth mindset. If only K-8 schools embrace a gender-free athletic process it will create a foundation for growth. There is no reason not to.

Finally, schools should focus on both winning and development, but the second part is key. Having a Team A, with the understanding that it is competitive, is important in giving an aspirational goal for all. Those on the team need to accept that, in being on that team, they might sit. But practices should be developmental. And Teams B, C and D should be levels of development. There is room for both those who compete and those who just want to play. The emphasis is on work and commitment, and from that comes growth and joy.

While a bunch of us were chatting, our administrator noted how all of the discipline cases he is dealing with are boys. He then noted how most of our failing students are also boys, and that most of our top students are girls. Not alone in this observation, he pondered what is perennially proposed: A different program for boys than girls.

Bad idea.

The problem of using gender to model programs is that it offers no path to a solution, except if the problem is gender discrimination.

In the data world they call this issue “the signal in the noise.” The signal is the problem–behavior and academic achievement–while classifications (gender, race, age) are the noise. We identify the boy problem because gender is an easy way to categorize people. We notice it. Evolutionary, we are built to recognize patterns as a means of survival. But our intuition can cause us to fall to, create and reinforce stereotypes. We create more noise, and lose the signal. Every stereotype has some truth at its core, but it ultimately binds the person it is being done to–we put the person in a box. And, it makes others blind to the real problem and its solution.

To say that our school does not serve boys is to say that there is only one way to be a boy. It’s a box we put people in. For the sake of argument, let’s use the stereotype that “boys need movement.” And let’s say that 80% of boys need movement. That leaves us missing 20%. And if the needs are reversed for girls–only 20% need movement–that leaves that group missing out, too Everyone is in a box.

Instead, we might build a program around students needing movement, and another program(s) around something else. Now, those student who need it (half) get movement. That 20% of non-movement boys are now free to pursue their needs, along with 80% of the girls. Win-win.

We track gender, race, SES and the like for two reasons. First, some elements of an identified group we can address–SES kids, by definition, need to be fed. But that use is limited. Second, we identify groups because some have been discriminated against, historically. That was the reason NCLB required those designations. When a group comes up short, this provides a place for schools to start the conversation–is the cause discrimination?

After that has been answered in the negative (hopefully) the use of such designations should then move to characteristics of the individual students in question and their needs. Why are these students getting into trouble? How large an issue is that, and how can it be addressed? In looking back at NCLB the one area that using the data made a big difference was with SES students. Interestingly, the solution was like the 80:20 above–many needed something the school was not providing (but not all), and some non-SES students benefited from those same programs. Win-win. It was not because of discrimination, but in starting there schools took a fresh look at a problem and identified the true root cause. Focusing on gender instead places the needs of some on the larger demographic while excluding others–it is not an efficient solution and can create new problems.

So what is our school’s problem? We are too indulgent. In our desire to provide to students what they need to succeed we have failed to hold them accountable. We take off limits but do not demand responsibility as part of the bargain. For example, I allowed music last fall because students benefited as it canceled out distracting noise. Now, it is the distraction. And music has creeped into other activities and classrooms. Snacks have become meals. Fidgets have become toys. Water and bathroom breaks are a right. In the end, I should have a product. Not always. In opening the barn door first I have now set up my enforcement of academics as conflict, not an inspired goal. Too often, the work reflects this shift in tone from support to scold. The exception is the rule. We indulge.

We have reached the tipping point: As we approach 20% of students being an exception, the exception becomes the rule. When music left the classroom, using during essay writing only, it became an exception. As 20% of classrooms had exceptions (hats, music, no lines, snacks) it becomes harder to hold to the rule.

There are two fixes for this. The first is to hold the rule. No hats. No music. Lines. The second is to add responsibility and accountability to the privilege. Currently, we ask nothing in return for privilege. Why? Because monitoring it is difficult and it sets up conflict. But that’s exactly how we creep towards the tipping point–we let those with the least amount of respect for others redefine the rule. When those 20% change expectations the number of rules pushed grow and the number of students breaking them grows, too.

Those kids are defending by packs of adults, all justifying why they need it and apologizing for transgressions. It’s not the exception that is the problem, but the transgression and accountability. We are starting with an assumption that this thing bestowed–music, gum, movement–is a right to be taken away, not a privilege to be earned. This is the exact opposite of how the adult world works–the most responsible gets the privileges while those lacking control either get few rewards or confinement. And we excuse them in academics, too, for the same reasons. We are doing these kids a disservice.

All of this is hard. You can’t penalize a kid for the lack of structure provided up until the moment they cross a line. The Responsive and Developmental Design programs offer those systems and protocols, but they require time and commitment from the group.

I would argue that it appears boys are the problem because success involves awareness and impulse control. Boys seem to have more of a problem with this, but they are not alone. Because society excuses much of it based on stereotypes (“boys will be boys” and “you can’t expect a boy to sit for an hour”) they hear that and internalize it. When we lean on punishment, though, we are teaching students not to be caught. Our data does not account for those who appear to follow rules but who skate the line constantly–take a census of how many are not where most of their peers are, or doing outlier behaviors, and you’ll find “they have permission.” Plus, we have behaviors that are more personal, and do not affect others. Even with academics, Tier II is filled with those who flail openly and dramatically. When we stop looking at the major behavior data, but instead account for minor behaviors and any deviations from the rule, a true picture of our ailments become clear. The solution is not movement but accountability to norms and earned privilege.