August 2, 2008

SES and Rotten Instruction

The only thing that the [standardized] tests are sensitive to is SES and racial/ethnic characteristic. If those two variables were partialled out statistically, the results would show that schools are pretty feckless instructionally.

That's not "news." It's been around since the Coleman Report of the 1960's. But the popular conclusion is that we have to "change society." The "obvious conclusion" has been overlooked--change instruction. When you do, you find that the correlation with accomplishments and SES is near-zero. That's empirical reality, not a statistical manipulation.

At least at the elementary level and if we don't include comprehension with uncontrolled vocabulary, and if we teach the higher-SES kids like we do currently, but his larger point is valid. SES matters quite a bit as long as instruction is rotten.

35 comments:

Eric
said...

Here's the Schutz quote I liked: fed bureaucrats in the social science agencies ... quasi-incestuous enterprise makes for politicized research and creates a government-academic axis that is civically unhealthy. It does not make for solid national educational R&D capability building. ... academy as much as the government should be dinged for the situation ... teacher training is a cash cow ... the farce is maintained

The important question is "how do we develop and shepherd an educational technology into classrooms for student benefit on a large scale?"

This is complicated by an "interlocking directorate of professional educationists" that benefits from the status quo.

It's as if the Better Mousetrap Corporation never anticipated the Mouse Hunter Union influencing accreditation of State University's Doctor of Moustrapology program that trains the employees of the state Mousetrap Evaluation Department...

> The "obvious conclusion" has been overlooked--change instruction. When you do, you find that the correlation with accomplishments and SES is near-zero.

We should call this the 'Disney Thory' of instruction. Where all the impacts of poverty and low socio-economic standing can be alleviated with a magical teacher, a la the Dead Poets Society or Dangerous Minds or whatever claptrap the movie industry is serving us now.

To be more evenhanded, envision the Miracle Max School of Education versus the Bialystock & Bloom School of Education. To get the teachers we need, all we have to do is unaccredit Bialystock & Bloom. Teachers are prepared by Miracle Max. They excel. Their students thrive. Taxpayers rejoice.

Just one problem: from which of the two schools did the accreditors get their credentials...

1. thinking that improved instruction can substantially improve student achievment given that there replicated large-scale longitudinal experimental research exists on this point, or

2. thinking that feeding kids more, giving them better access to health and dental care, and giving their parents a monetary hand-out will substantially improve student achevement given the lack of replicated large-scale longitudinal experimental research exists on this point.

Just when I open another fresh pack of earth-tone crayolas, someone lets loose with this: a pumpkin and two mice short of a stagecoach (no google hits for this; I checked with my daughter and she's never heard it but it will be her new favorite.)

I'm still soliciting help with educating a Governor's education deputy on the need for proven approaches. How, for example, can we achieve the outcomes identified on this Social Studies Map unless kids have effective literacy and critical thinking instruction (e.g. either Reasoning and Writing or HOTS)?

I thought I made that one up on the flyApparently you did. My daughter and I thank you.

achieve the outcomes identified on this Social Studies Map ... That's probably impossible.

Then we need to get the Presidential candidates to add Reasoning and Writing (or HOTS) to their education platforms.

Do you think NEA will lend support, or will they just let the Partnership for 21st Century Skills raise expectations without helping teachers gain access to the curricular materials that could help them succeed?

Eric is a past member (2005) of a state-level Malcolm Baldrige Award Board of Examiners.

Why don't we work together and get 10,000 local school districts to endorse Senator Obama's "discussion" on the NEA's "roadmap" Great Public Schools For Every Student by 2020. Then we can ask NEA for the data that would allow high school seniors to examine education policy using the 21st century outcomes from this social studies map. That would surely resolve Rod Paige's concerns about the NEA's commitment to the principles of democracy and ethics.

Otherwise we have the ugly image of the first black president of the Harvard Law Review playing stupid (as if on command) while the NEA dangles a $50 million incentive likely to be distributed 90% to Democratic candidates. ("Beg; roll over; play stupid") How dare they.

Pardon this intrusion. I'm almost certain I'm not thinking about these issues at the same level or from the same angle as the discussants here.

But . . .

"The obvious conclusion has been overlooked--change instruction."

To quote yet again from my favorite movie: That's damn right.

Without ever having tested this idea myself, I can say with confidence that pedagogy is more pliable than any student's reality of poverty, disease, abuse, and neglect.

Does that mean that public education institutions--very human organizations, as any--should turn their backs on the under-fed, the abused, . . . migrants, homeless children (yep, they're there) . . . those with severe learning disabilities, those with severe (and severely disruptive) behavioral maladies, persons with handicaps, gifted children, etc, and focus exclusively on "education science"?

No.

But the activity of educating our children is--must be--bounded in some way. Otherwise, it loses most of its meaning.

I think some would like to make the "definition" of education so small as to be able to "drown it in a bathtub." Others would like to make its reach so powerful that it comes to define even the very nature of what students are learning.

I'm in more agreement with the former, but I'd like to know that there is a potential for larger bathtubs.

The NEA document isn't a "roadmap"--not even close. The fatal flaw is the blinkered vision 2020. That dismisses a whole generation of el-hi kids. And the "Social Studies "map" isn't a map--not even close. The only thing "higher" about "higher order thinking skills" is that they're out of teachable reach. The example activities at selected grade levels don't add up. The grades could be scrambled and you'd get about the same effect. The mice are running the theme parks, and they appear to have been lobotomized.

There are any number highly functional and generlizable skills that can be taught--google searching, outlining, summarizing, expanding, storyboarding, flowcharting, propaganda techniques, negotiation techniques and so on, and on. These are "fun" if you know how to do them and they enable/empower you to independently/interdependently do things you couldn't do before.

My favorite definition of "education was coined by the philosopher Josiah Royce: "Education is learning to use the tools which humanity has found indispensable."

Those "tools" are becoming more complicated but at the same time they're becoming simpler. The symbol systems of mathematics and written language are the indispensable, indispensables. Parents, employers and professors recognize this, and have been begging responsible agents to stand and deliver on the tools if they do nothing else.

The only obstacles in doing so NOW are at the top not the bottom of the EdChain. Where's "accountability" when we really need it?

An interesting exercise: investigate the (Capitol Hill) backgrounds of "Partnership for 21st Century Skills" insiders (and their supporters); Speculate on possible motives of their members; compare and contrast with the Intel/OLPC dustup.

It's just amazing what you find when you start out with the question: "How many dollars per student do I need to fix a dropout factory."

Anyone have a roadmap? OK, anyone other than a tow-truck company have a roadmap? Better yet: anyone have a roadmap and satisified, long-term customers?

There are any number highly functional and generlizable skills ... google searching, ... storyboarding, flowcharting, ... and so on, and on. ... they enable/empower you ... The mice are running the theme parks, and they appear to have been lobotomized.

Pretty clever of lobotomized mice to ensure their wards stay unempowered and their critics silenced or marginalized. I'm not so sure that mice are running the park--uhhh, racket?.

All the dollars in the world wouldn't do it. They'd go largely for personnel costs and the personnel would do same ol, same ol.

Neither INTEL nor OLPC (or anyone else with any clout) is taking seriously "software as service" "instruction as service" is the way of the future. An amazon Kindle type device, souped up with more interactive capability is one possible configuration worth considering.

Both INTEL and MIT are trying to foist off "big people" general purpose configs off on instruction. No thought given to serious R&D to make the application optimal for kids. Or to testing before dumping a whole lot of money into production.

The "Tangled Web" is that education isn't "governed." It's "administered." And it isn't "policy" that's needed. That's competing ideologies. What's needed are product/protocols that can be used to reliably deliver specified instructional aspirations. We have a few of these. But more are needed.

Actually, public schools are institutionally weak only in the area of instruction. When we get better educational intelligence, the slack can be tightened and economies effected.

The scenario/roadmap for effecting better ed intelligence is clear. It's less intrusive and less expensive than current testing practice and illuminates responsibility at all administrative levels.

Q:How many dollars per student do I need to fix a dropout factory?A:All the dollars in the world wouldn't do it. ... No thought given to serious R&D ... What's needed are product/protocols that can be used to reliably deliver specified instructional aspirations. ... less expensive than current testing practice and illuminates responsibility at all administrative levels.

Clearly Negroponte/OLPC is doing R&D and putting (lower level) protocol stacks in place. (We could flowchart, program manage and Venn digram them. We could learn about the ethnographics of ed tech transfer) Clearly, we're more concerned with higher level protocols where education actually happens (Framing Issues for Public Deliberation: A Curriculum Guide for Workshops and scripted programs could be seen as examples.)

So what protocols are missing and need to be in place before the stated cost reductions occur? How can we encourage research along these lines (a keystone protocol crafted (by whom?) passed by a state legislature and signed by a governor, perhaps?)?

This is a serious question. Where should money go if we want results? What is the return on (educational) investment of a proposed R&D effort? Have Negroponte, Stan Pogrow, and Zig made contributions to this effort or not? What have we learned?

This is a good time to let Presidential education platform wonks know that a bait-and-switch is in the works. (Actually, they already know that, they just can't address the details--and we aren't giving them alternatives by which to assess posibilities of success.)

You say "What's needed are product/protocols that can be used to reliably deliver specified instructional aspirations" and "When we get better educational intelligence, the slack can be tightened and economies effected."

Could you be specific in describing exactly what you mean by "product/protocols" and "better educational intelligence"?

That's true. But the MIT folks have very strong views regarding how education should go. If you like their views, you'll like their rig. If you want to teach kids how to read, do math, and use the internet, it's not optimally adapted to those applications.

Negroponte and Zig have certainly made contributions, in my view. I don't know enough about Pogrow's work to comment on it.

I use the term "instructional product/protocols" instead of "instructional program" because that term has lost all meaning. It's all the kit and kabboddle that provides the wherewithal to reliably accomplish a specified instructional aspiration.

I described the product/protocols I'm aware of in reading awhile back at Ken's request in an earlier thread. I'll do it again briefly:

Zig's DI/Mastery gear would certainly qualify, if Zig or someone cranks out a set of 5-9 Key Performance Indicators to mark the acquisition of reading expertise.

BRI-ARI was initially derived from R&D sponsored by US Feds, who walked away from it, as they did with the Becker-Engelmann program.It's been updated in intervening years to reflect subsequent research on the Alphabetic Code. There is information on the current gear at www.piperbooks.co.uk/index.htm(See particularly the research papers in the research pull-down menu.)

The popular UK programs are termed "synthetic phonics"--(an inapt designation, but we're stuck with it.) The UK govt has an elaborate set of Guidelines:

www.standards.dfes.gov.uk/local/clld/las.html

Schools and teachers can either cobble together their own gear or use a published program. The Guidelines plagiarize Jolly Phonics, which has been copied by a few other program authors. Chris Jolly doesn't mind because he has a commanding market share and the govt support increases his volume.

A very recent entry is the computer-based International Phonics Program authored by Debbie Hepplewhite, President of the Reading Reform Foundation, the professional association that honchoed the UK Govt's official legislating SP beginning in Reception (our Kindergarten).

www.phonicsinternational.com

I don't know of any math product protocols. I've contended that the work of the National Math Panel makes the construction of the requisite gear very tractable.

Re"Educational intelligence." It's an extension of "business intelligence" and "dashboards" to instruction. Googling for these terms will tell you more than you want to know. The extension to instruction is routine IF you can generate Key Performance Indicators. If you can't, you can't say where you're going to start, how you're doing along the way, or how you'll know when you've completed the job. In StreetTalk, you're bullshitting.In EdTalk you have "gaps" which the "gains" never close.

The logic for increasing instructional productivity is that instructional performance is a function of student, teacher, and instructional product/protocol. KPI's can be aggegated by these variables, by higher administrative units, and by the "usual" demographic characteristics.

Previous experience has been that there are some teachers and schools who "march to their own drummer, and they are easily flagged. Demographic characteristics are not consequential. The "big differences" are in the reliability of effect and the time and cost of implementing the alternative product protocols.

This is the common experience in sectors other than eduction. As Ken stated in originating the thread, it's not the rotten kids or teachers. It's rotten instruction.

So, to paraphrase, K-12 education would benefit from research-proven instructional programs, such as DI, that combine a focus on specified skills with explicit instructional guidelines.

In addition, educational institutions (primarily schools and districts) should create and implement consistent assessment tools that provide ongoing feedback on the extent to which students are mastering (or not mastering) specified skills and content standards.

Don't forget that the integrated Education Information System has lights and a siren to alert the district curriculum director to any slowdown in the KPI production quota (or so it will be feared).

Those teachers who've worked in a district headed by Toxic Bozo--with direct reports Dragon Lady, curriculum director and Hatchet Man, business manager--might prefer something else. They might prefer good-smelling, teacher-friendly snake oil that levels the field by not improving student achievement for anyone--especially if data is so soft that the ruse can be rationalized.

John Conyers; District 15 (Palatine, IL): You cannot address NCLB with random acts of improvement. ... Attention was focused on pushing for leading indicators, and not lagging indicators. In other words, focus less on results data and more on quarterly and even daily assessments. We were ahead of the game when NCLB finally arrived because we had been practicing a tougher self-assessment strategy with more leading indicators (real time). We had learned that we had to provide the teachers with data and internal assessment packages that would help them drive instruction based on data they received from students.

This is where a field-tested instructional sequence provides real advantages. Once you have a good sequence in place, you can start making accurate predictions as to where students should be in the sequence 100, 200, 300 lessons out. This also allows plotting the instructional trajectory and use that daily and weekly data to respond quickly to the teaching performance where the trajectory falls below a grade level pace.

This is where a field-tested instructional sequence provides real advantages.

Yes. We have successfully connected a governance thread with a DI blog. But if governance is allowed to go unfixed, DI won't get the attention it deserves. I really think we need to fix governance first. (This shouldn't be too hard: "Dear LCCREF, do you want blue ribbon schools that fail or gold medal (Baldrige) schools that succeed. Your choice. Be sure to let your candidates know.")

Ron Maurer, Pearl River: Now with the NCLB, there are added assessments mandated. For a building principal the responsibility for improving every student's performance on every assessment can be enormous. In fact, no principal can maintain such dedicated energy in this regard. Baldrige presents the building principal with a process that adds value to the task. The strategic planning focus, for example, allows the principal to focus his or her resources and energies on improvement for a select few objectives. Rather than saying improvement will occur in all academic areas (and therefore probably in none), the Baldrige strategic planning process allows the focus to be a narrow but important spectrum of student achievement. The odds of improvement in these areas rise significantly. The value added for the principal is that he or she knows what the objective is, what resources are being committed, what the plan is, what the implementation strategies are, and what formative evaluative checks will be in place. The work of the principal is focused.

But here's the rub: Creating that system is incredibly complex, especially when you're a small district with limited resources.

Yes, when you bring in a pre-packaged program with built-in assessments and remedial structure, such as DI, it makes life much simpler. But what does it look like when you get past foundational literacy or math skills? What does that type of program look like for World History, British Literature, etc.?

I work in a high-performing district that has attempted to do this at the middle and high school levels: daily, structured lessons that are consistent across the district, coupled with regular, standardized assessments and quality reports. It is a difficult and complex process building the system, much less creating fidelity of implementation across 100+ schools.

That's not a reason not to attempt it, just a recognition that it is far from simple or easy.

Well, we're not quite all on the same wave length here. I don't see the Baldridge model as doing anything for the schools who need it the most. And as far as predicting whose going to do best on the next lessons in the future, who cares?

I'm contending what would be useful is simpler. Information on instructional status. Where students are with respect to the delivery of being able to to read.--Reading any text with understanding equal to that were the communication spoken. The KPI logic provides that information. I don't know of any other logic that does. Do y'all know of any?

parry graham said: "But here's the rub: Creating that system is incredibly complex, especially when you're a small district with limited resources."

Me: It all depends upon how you go about it. If you do it right, it can be more economical than what you're now doing. Course you'll have to displace the junk "programs" and tests. But you won't get complaints from teachers, kids, or parents.

Parry:Yes, when you bring in a pre-packaged program with built-in assessments and remedial structure, such as DI, it makes life much simpler.

Me: All instructional materials are "prepackaged." Most are prepackaged junk. A few "do what it says on the can."

Parry: But what does it look like when you get past foundational literacy or math skills? What does that type of program look like for World History, British Literature, etc.?

Me: Well, if kids can read, you have a wide latitude. You can also consider "What should be in kids' heads and what should they be able to search-and-find on demand?" Schools are operating in a pre-internet era. "Google capability" goes a long way in providing the "higher order thinking skills." And outfits like google, amazon, and such are getting better and better at interacting with you to help you delve into matters you didn't even know you were interested in. It's really seductive and fun. Not yet up to the level of sex, but they're working on it.

Parry: I work in a high-performing district that has attempted to do this at the middle and high school levels: daily, structured lessons that are consistent across the district, coupled with regular, standardized assessments and quality reports. It is a difficult and complex process building the system, much less creating fidelity of implementation across 100+ schools.

Me: Agree. Doing it a district at a time and carrying the baggage of such things as "standardized assessment" and "fidelity of implementation" isn't sustainable.And by the time kids get to middle school and high school they're so strung out by inadvertent mal-instruction that teachers are inescapably dealing with a dog's dinner.

I don't see the Baldridge model as doing anything for the schools who need it the most.

I'm trying to make the point that school governance can impede adoption of effective protocols/technologies, and Baldige can help diagnose such governance problems.

At a high level, Baldrige is just:"Do the best you can with the resources you've got; Don't let leadership and strategy get out of touch with the classroom; Make sure your staff have the tools and data they need to do their job; Ensure that changes will be for the better."

So an urban district might do balanced scorecard because Council of Great City Schools thinks that's a good idea. A Baldrige district will do balanced scorecard because it helps keep administration, staff, and strategy in sync and in touch with the classroom and students. Which balanced scorecard implementation is more likely improve student achievement and community trust?

If you're using "21st Century Skills" for "problem-solving" with "diverse teams," your're probably doing Baldrige (even without knowing it.) Of course, if you think you're doing the above yet failing miserably, that's not Baldrige.

So the hazard of various election-year education manifestos is they will displace the hard work required to fix real problems. No doubt our schools have prepared their graduates to recognize such a threat to foundational democratic institutions. :-(

Participatory democracy is one approach to fixing governance; David Mathews' Reclaiming Public Education by Reclaiming Our Democracy has details. National Issues Forums has a high school program, NIF in the Classroom. Combine that with the Partnership for 21st Century Skills Social Studies map, and maybe high school kids can fix public education's problems.

Three questions, Eric--How are you going to get the top of the EdChain to vacate their positions of power? Do you really think "high school kids will fix public education's problems. What exactly is it that you expect them to fix?

My understanding is that the kids who are dissatisfied with high schools are dropping out. Those who expect to go on to college may grumble now and then, but school as-is is all they know.

The thread is about fixing "rotten instruction." Being clueless and resistant to change is not equivalent to being "rotten." The "rotten" part is imposing mandates that are statistically impossible and then boasting of "gains" and "closing the gap" based on nothing more than a(below the mean)cut-score definition of "proficiency.

I happen to have faith that if the feds will stop beating kids and teachers over the head with NCLB sanctions, at least some states and/or LEAs will take initiative to begin reliably teaching kids how to read and do math. As things currently stand, there is no chance of this happening.

if the feds will stop ... some states and/or LEAs will take initiative ...

Some? Leadership Coalition for Civil Rights won't like "Some"! Good project for high school students to investigate under-representation of minorities in Advanced Placement. They could use Freedom of Information requests to investigate root cause (poor reading skills?). Actually, all they need is to threaten to do that, and the LEA might start paying more attention.

From the Partnership for 21st Century Skills Social Studies Map: "Using sound reasoning and relevant examples, students analyze the historical evolution of a contemporary public policy issue, place it within an historical context, and use a digital publishing tool to report their work. ... Students identify a public policy issue in their community, preferably related to school policy, and work as a whole class or in small groups. Students create a wiki for mobilizing others who are interested in the issue. This would require agreement on the primary positions taken by the group and coordination among students for the purpose of articulating a persuasive message."

"Leadership Coalition for Civil Rights won't like 'Some!" I don't like it either. But even "one" would be a huge increment beyond the "none" we have now.

"This would require agreement on the primary positions taken by the group and coordination among students for the purpose of articulating a persuasive message."

Good luck! Has the "Partnership for 21st Century Skills" actually done this anywhere with any "policy issue?" I don't think it's in the cards, but if it's happening, I'll eat the deck--with a few condiments and a beer or two on the side.

Has the "Partnership for 21st Century Skills" actually done this anywhere

No, but they are the NEA single source preferred vendor for "unified, collective vision for 21st century learning that can be used to strengthen American education," and clearly the NEA would exercise due dilligence to avoid discrediting educators throughout the nation.

OK, so those folks really are delusional foils of union hegemony. But since they volunteered, why not? At the very least, they could help us spot the 21st century skills (latently lurking somewhere) within NEA's NCLB Reauthorization policy papers.

And your criticisms are right: they have no map. They have no technologies and protocols. They probably can't name professional development to support their outcomes (unless it's for the "To:" line of a blank check!).

I do believe high school students can be empowered to be part of the solution--maybe tutoring or improving school climate. Certainly "how to cast an informed vote for school board" is appropriate for senior government classes.

So maybe that brings us full circle. I wanted a dialog with someone who sees a technology transfer problem underlying improved student achievement. I really believe we fix governance first, but I'm not at all averse to any district teaching kids to read.

But the core of my perspective is being responsive to rule of law. And maybe the message to the Governor needs to be build commitment to public education, starting with educators. Because when educators become informed advocates for public education, we'll fix rotten instruction.

But as it stands now, we're pretending educators will ask more of our students than they will of themselves or their (union staff) employees. That leaves us all two mice and a pumpkin short of a horse-drawn carriage. Not to overlook the fairy godmother.

P21 concerns me as it's rolling into Massachusetts this year. This bunk from their mission statement:

To successfully face rigorous higher education coursework, career challenges and a globally competitive workforce, U.S. schools must align classroom environments with real world environments by infusing 21st century skills.

About D-Ed Reckoning

The primary problem with K-12 education today is the problem of dead reckoning--an estimate based on little or no information. We don't know what a good K-12 education system is because we've never seen one operating. A good education system is one that is capable of educating almost every child.