April 23, 2007

Reading First Hearings

I watched all four hours of the Reading First hearings on Friday night and only managed to doze off once. (That's because I was assembling a new computer or the kids as I listened, so I was somewhat occupied.)

I'm not in the habit of watching too many of these congressional hearings, but I have witnessed a fair number of courtroom proceedings. There is no comparison. The congressional hearing is more like a kangaroo court than a legal proceeding. Most of Congressmen and women had no idea how to ask questions to elicit information from the witness. Many seemed content to mug for the camera and read speeches. It was clear that most of the Congressman on the education subcommittee had no idea of the issues surrounding Reading First.

Let me single out Congressman Miller as being particularly inept. This is especially surprising because according to Miller's bio, he's a lawyer. A lawyer that doesn't know how to take a deposition. Miler's idea of taking a deposition consists of reading a portion of the Inspector General's report in as dull and monotonous a voice, ask a, "so what do you think about that?" kind of question, give the deponent an opportunity to speak (sometimes at least), and then give his own opinion of the "facts" in an angry voice. We got more testimony from Miller than we did from any of the deponents.

Sadly, we are no closer to the truth now than we were before the hearing. We did, offhandedly, learn a few things, however.

The Inspector General, Higgins, admitted that he had never looked into the question of Scientifically Based Reading Research (SBRR) or the Essential Components of Reading Instruction (ECRI). This is a critical omission. Reading First is a statute that was designed to exclude reading programs that were not based on SBRR and which did contain the ECRI. Reading programs were, in fact, excluded. That was the whole point of Reading First to fund the right programs and exclude the wrong ones. So, were any programs improperly excluded? We don't know because no one looked at the critical cut-off point, i.e., whether the excluded programs were based on SBRR and had the ECRI.

Starr Lewis, Kentucky’s associate commissioner of education, was trotted out to give her tearfully story that she couldn't scam funding from Reading First for the controversial Reading Recovery reading program (and other whole language programs) and the fraudulent DRA testing instrument designed to show that Reading Recovery "works" even though the kids going through the program often can't read. This goes to the SBRR and ECRI issue. No one denies that Kentucky was denied Reading First funding. The issue was whether Kentucky was improperly denied. We don't know because no one has looked into the issue yet. Like, I said, kangaroo court.

Then Lewis recounted how Kentucky was denied funding for their use of the bogus DRA testing instrument. Kentucky responded by adding DIBELS to the list of permitted testing instruments. DOE said not good enough, we're not funding DRA. Kentucky finally withdrew DRA and was funded. Does this represent a violation of the law? We don't know because no one has looked into the SBRR and ECRI issue yet. Do you see a pattern emerging? The sequence of events also doesn't necessarily represent an instance of DIBELS being forced on Kentucky. DIBELS didn't serve to remedy the problem caused by including DRA and we don't know if the inclusion of other testing instruments would have satisfied DoE because Kentucky never presented other testing instruments for consideration. If Miller was looking for a poster child for the Reading First scandal, Kentucky wasn't it.

Then we have the conflict issue. Right off the bat this is a non-issue because Congress failed to included a conflict provision in the Reading First statute. But, DoE did, in fact, screen for financial conflicts. No financial conflicts were found. Higgins affirmed this. But, in his report Higgins substituted his own judgment for DoE's judgment by suggesting that DoE should have screened for "professional associations" with reading programs. This is a ridiculously broad standard which would have eliminated all the reading experts from judging what programs should have been excluded from reading First for failing to be based on SBRR and having ECRI. All the reading researchers have some professional tie to either phonics-based reading programs and whole-language reading programs and under Higgin's contrived non-statutorily-based conflicts standard all the reading researchers should have been excluded. Yeah, that's the conclusion we want to reach, exclude all the reading experts from being able to determine which reading programs qualified for funding.

To avoid this problem, DoE made certain that none of the reading researchers would review the programs with they had some tangential affiliation. According to the former Reading First director, they modeled this system off of a previous reading program funding scheme which passed Congressional muster. And bear in mind, that for prior federal programs that doled out money, the old standard was to specifically exclude the collection of data. That didn't work out so well in hindsight, so we tightened up the rules a bit in Reading First. Apparently, Congress didn't realize what it was getting itself into. fair enough, they have the right to change their mind to avoid the "appearance of impropriety" that they believe that resulted from DoE's scheme for complying with the Reading First statute. But, such an ex post facto change doesn't exactly show that DoE was guilty of any wrongdoing.

Making this conflict issue an even bigger non-issue, it came out of the hearing that less than 10% of the states even specified the actual names of reading programs in their Reading First applications. So even if there was a conflict with the reviewers, they had no way of knowing which programs the states were actually selecting. How could the "biased" reviewers have pushed specific programs on the states when practically none of the states even specified reading programs in the first place. There's a name for this in the law. It's called "harmless error."

Finally, there is the related problem that many states choose not to use Reading First funds to purchase many reading programs. Even if a reading program has SBRR and ECRI out the wazoo (Hello, SfA) no state was obligated to select the program for funding. Even if DoE did improperly force state's to adopt specific reading programs (which there is no real evidence of them actually doing), less than 10% of states actually listed specific programs in the first place and many of them went and chose programs on their own after the application process. So we have this large disconnect between what DoE could influence and what state's could actually do. Supposedly, there are a few instances in which state's tried to pull a fast one by submitting an application that appear to show they would abide by the SBRR and ECRI requirements, but then funded whole language programs. You can't resolve this dispute within looking into the SBRR and ECRO provisions, and no one has yet to do that.

Those are the big issues and no light was shed on any of them in the hearing. There are some remaining minor issues that indicate sloppiness of the part of DoE but it's not clear if they represent violations of any law that amount to anything. In all the OiG reports you don't see any real violations anywhere just "potential" violations and "appearances of violations." This is probably why this scandal hasn't gained much traction outside of the sour-grapes whole language community which has been largely excluded under Reading First. It is this exclusion which has caused the raising of scores under Reading First.

The Justice Department is conducting a probe of a $6 billion reading initiative at the center of President Bush's No Child Left Behind law, another blow to a program besieged by allegations of financial conflicts of interest and cronyism, people familiar with the matter said yesterday.

The disclosure came as a congressional hearing revealed how people implementing the $1 billion-a-year Reading First program made at least $1 million off textbooks and tests toward which the federal government steered states.

Too bad even the OiG failed to find any real financial conflicts and that there was no evidence of actual "steering" of state selections over and above the proper exclusion of programs as required by the statute.

When Wapo does give us an example of a purported financial conflict, it turns out not be a valid one:

One official, Roland H. Good III, said his company made $1.3 million off a reading test, known as DIBELS, that was endorsed by a Reading First evaluation panel he sat on. Good, who owns half the company, Dynamic Measurement Group, told the committee that he donated royalties from the product to the University of Oregon, where he is an associate professor.

Two former University of Oregon researchers on the panel, Edward J. Kame'enui and Deborah C. Simmons, said they received about $150,000 in royalties last year for a program that is now packaged with DIBELS. They testified that they received smaller royalties in previous years for the program, Scott Foresman Early Reading Intervention, and did not know it was being sold with DIBELS.

I haven't been following the testing instrument part of the story as much as the reading program side, but my understanding is that Good, Kame'enui, and Simmons were panelists judging testing instruments. Apparently, the Early Reading Intervention which two of them are affiliated with didn't include a packaged DIBELS component until recently. SO clearly, there was no violation at the time of review in the absence of some knowledge that DIBELS was to be included in the future. That evidence has not yet been adduced.

Then we have the lurid innuendo concerning Good, who created DIBELS. Apparently, he's made some money off of his invention. When you enact a law designed to give out a billion dollars of funding, lots of people who own products that meet the law's requirements will benefitby the law. There is no evidence of record that shows that Good forced any testing instrument on any state. The best the OiG found was this summary of the testing instruments that listed DIBELS as one of 24 possible testing instruments that were believed to be valid instruments.

All three Oregon panelists said they had not ranked their own materials’ fitness for use under Reading First, and so had avoided any conflict of interest. And they said it was the quality of their programs, not direct or indirect pressure from Education Department officials, that explained their popularity.

All we are left with is the testimony of Miller who does not appear to understand that his quips don't count for much besides headlines.

In an interview after the hearing, Mr. Miller said: “This hearing made it pretty clear that there was a very incestuous relationship among a small group of people in the Education Department and among contractors. They were very clearly using this program … for profit.”

Even if all this were true, it still doesn't mean the law was violated. The law was designed to profit a small group of reading programs for the benefit of children who are struggling to read. The fact that some profited as intended does not mean there was a violation. Potentials and appearances don't necessarily rise to the level of violations in the absence of proof. Proof that Miller has so far failed to obtain.

Today's big news is that the Justice Department may get involved. We'll have to wait any see if some real lawyers can cobble together a coherent scandal from the shoddy facts we've been given so far. I'm predicting in the absence of any new findings, the existing "facts" are insufficient to show any real violations of law.

DRA is the Reading Recovery non-satndardized test developed by Marie Clay that Kentucky wanted to use in conjunction with Reading Recovery. Talk about being unscientific and a scam both at the same time. The irony is that Lewis sat there with a staright face and came right out and told the committee that she wanted to pull off this scam and they were not only completely oblivious but were sympathetic to her plight. So much for safeguarding the public trust.

I wonder why your analysis doesn't make any mention of the facts surrounding Doherty's expert panel, which absolutely was not created in compliance with the law. Nor do you address the various ways in which he mislead states through the fake panel comments, his manipulation of the law when it served his needs, and his "off-radar" phone calls and e-mails.

If DRA and Reading Recovery and others are not SBRR, didn't Doherty and USDE have a responsibility to formally and publicly announce that fact? Why wasn't there a list of non-SBRR programs and assessments on the ed.gov website? Why did Doherty refuse to provide Kentucky anything in writing? Why is Kentucky still using Reading Recovery -- Doherty told them not to, they called his bluff and asked for it in writing, and he backed off. Why the sneakiness? Is it possibly because if he were to make public what he chose to do in private, then the public would have been outraged by his zealotry?

It's rather convenient to say that DRA and Reading Recovery are frauds and leave it at that, but the reality is that a great deal of reading experts would disagree with that assessment. And so would the Department's own What Works Clearinghouse.

Anon, welcome to the present. Tired of papering old posts with comments?

I put all the things in your first paragraph under the group "minor issues" since they are.

The panels were clearly not formed in accordance with the law. but without a specific showing of the harm done, the error was harmless. When you can locate a harm, let me know.

I see no prohibition under the law regarding summarizing panel comments by DOE. Again, you'd have to show a misleading misrepresentation amounting to DoE making curricular demands for this to be an issue.

Re RR and DRA, find me the provision in the Reading First law requiring Doherty to have done what you want him to have done and then we'll discuss. We are looking for statutory violations for purposes of this scandal.

For his zealotry to be a violation of law, we need to first have am underlying violation of law. I do consider it a potential violation for him to have funded RR and DRA, but other experts may disagree and I'd have to show that the basis of their diasagreement to be wrong.

You mean a great deal of "reading experts" as opposed to real experts. The WWC is not infallible either. Most researcher would not have permitted a non-standardized measure to have been included in the evaluation. take that away and what do you have? One study -- showing that RR would be greatly improved by adding a phonics component. Yet another reason why RR shouldn't have goten RR funding it lacks at least one ECRI, i.e., phonics.

Anon, you need to go read the RF statute before trying to opine on these issues. And, why do you keep avoiding addressing all the points I've raised? Are you trying to hide something or are you just conceding those points? You use a great deal of words to say very little of substance.

And back to your points. Answer this: Isn't it just a little bit fishy that Doherty only contacted state officials by phone? He left no paper trail and refused to put his comments in writing -- whether the law did or didn't allow it is not the point. The point is that he would have better served the public if he had done his deeds in a more transparent manner. Can you at least agree with that?

Oh, but then everyone would have known what he was doing, wouldn't they? If it weren't for the OIG reports, we would know none of this stuff.

Using the phone is not a violation of any law I know. If there is a requirement that Doherty put everything in writing, then I'm not aware of that mandate. It may be that you think that that's what the law should have said, but that's not it actually said. And, we are concnered with violations of actual laws, not ones that don't exist.

And, every time he spoke on the phone to someone, that person is potentially an aggrieved person who could have been located and could have provided testimony. It is telling that no one but states wishing to adopt whole language programs have come forth to testify. DOE has sufficient cover to have denied these programs under the law.

Some of us in the UK are watching this with interest. If Reading Recovery can drive a hole throught the Reading First requirements, then they have a stronger case for their penetration in the Uk, where similar requirements for 'evidence based' programmes are now in place.

I was amazed to see the WWC endorsement of Reading Recovery as it was obvious that its 'success' was based on the results of Clay's own test - designed to demonstrate that, duh, Reading Recovery is effective!

Clay's latest text, Lessons in Literacy Part 2, seems to have been written to try and show that RR conforms to Reading First requirements. There is a review of the book on the UK Reading Reform Foundation web site:http://rrf.org.uk/messageforum/viewtopic.php?t=2774

It was written with the UK requirements in mind, but they are similar to yours in the US. I could find no evidence of much change to the programme. It certainly doesn't conform to UK requirements.

In response to Kderosa's comment: "DRA is the Reading Recovery non-satndardized test developed by Marie Clay that Kentucky wanted to use in conjunction with Reading Recovery."

The Developmental Reading Assessment (DRA) is not part of RR. DRA was developed by Joetta Beaver and is a reading assessment used by (K-3) classroom teachers. The assessment in RR is the Observation Survey (OS, developed by Marie Clay. The DRA is similar to and based upon the leveled text component of OS. However, they're separate, administered and analyzed differently and used for different purposes. KY may have wanted to include DRA as part of their proposal as a tool for diagnosis or progress monitoring in the classroom setting, but OS is the assessment used in conjunction with RR.

These arguments and discussions will continue interminably until each literacy programme is accurately delivered to a reasonably large sample of children whose progress in literacy is carefully measured over 4 to 5 years. (ie longitudinal studies.) Only then will we be able to make really informed judgements about their effectiveness, or otherwise.

The real question is why someone like Miller - who professed during the Hearing to be concerned about even the 'appearance' of impropriety - was so willing to accept (and with a straight face)Kentucky's proposal for using Reading Recovery along with (you guessed it!) an assessment tool developed by one of its biggest supporters and promoters. Now that's more than just an appearance of impropriety. I'd say it's the real thing! And I don't mean Coke.

I'm sure that Anonymous already knew that Joetta Beaver is a Teacher Leader for Reading Recovery. She has been described by one family on the internet in the following way:

"Joetta Beaver had been our daughter’s classroom teacher. She was also a teacherleader in Reading Recovery. We thought she knew what she was doing. We considered Joetta and the other educators at Barrington as the “experts” in deciding what to do about Joe’s reading difficulties.

Today, I have learned that Reading Recovery was probably the worst thing that we could have done to Joe. Due to Joe’s dyslexia he had no natural phonological awarenessand a poor visual memory. Instead of an intensive, systematic, phonological method likeOrton-Gillingham, the Reading Recovery method teaches the child to use picture and context cues, and tries to get the child to memorize sight words. Every day, Joetta taught Joe to guess at words based upon what he saw in the pictures and his understanding of the sentence context.

Joe did not learn to read with the Reading Recovery method, but he did practice guessing at words day after day. This has had a devastating effect on Joe’s learning to read.......

I have learned that Joetta had an above average interest in proving the success of Reading Recovery. I believe this is why she specifically wanted Joe in the program and why she so aggressively used the method with him. The Ohio State UniversityDepartment of Education was instrumental in bringing Reading Recovery to the UnitedStates from New Zealand. Joe was one of the first students, in this country, to use thissignificantly modified New Zealand approach. I consider Reading Recovery, at the time itwas used on Joe, to be experimental. We were never informed of this.

Joetta was one of the very first Reading Recovery teachers. She was a Reading Recovery teacher leader. She worked closely with Carol A. Lyons at The Ohio StateUniversity. After Joetta taught Joe, she published research on Reading Recovery. Sheparticipated in speaking engagements regarding Reading Recovery. She received peerrecognition for her work with Reading Recovery. I believe Joetta had a professional andemotional stake in proving the success of Reading Recovery. I do not know to what extent, if any, she has benefited financially from her role with Reading Recovery.However, I perceive that Joetta had a conflict of interest when it came to deciding whatwould be the best method to use with a dyslexic child. I keep asking myself was Joe usedas a guinea pig for Joetta’s research?"

Well, Anonymous, I suppose that you'd say that if one mid-level bureaucrat from Kentucky says so, it must be true? Maybe you wouldn't - but Chairman Miller certainly did.

So, if we agree that surveys of one can indeed be misleading, let's discount the testimony of Starr Lewis as well while we're at it. Especially since Lewis was perfectly willing to risk shredding her cedibility by proposing only one assessmant tool (DRA) which, curiously, was developed by a major proponent of the program to be tested! At the very least this rises to Chairman Miller's bad odor test of appearance of impropriety.

It's one thing for Miller to question the conflict of interest procedures instituted by Doherty and the panels. But my question is (and remains) why wouldn't he apply this same standard to Kentucky? After all, a federal taxpayer dollar is a federal taxpayer dollar wherever the money is spent - even if in Kentucky.

Regarding zealotry, I think even Ken agrees that Doherty was probably over-zealous, consequently counter-productive - and therefore properly fired. I don't know what evidence (of RR's effectiveness) Kentucky submitted but I assume Observer would agree that, at a minimum, if it was produced using either OS or DRA it would have been properly rejected? But, is this what you call 'ignoring the evidence provided'?

Finally, criminalizing the Reading Wars is the worst kind of zealotry of all. That is what I saw as I watched the Hearings.

I've just come across an interesting letter sent in 2002 to "policy makers, educational leaders and researchers and federal research organizations" about Reading Recovery - which further confirms my view that Chairman Miller was, at the very least, negligent in not questioning Kentucky's 'insistence' on using DRA and Reading Recovery.

Here are some relevant quotes (apologies for the cutting and pasting - but verbatim is better):

"While research distributed by the developers of Reading Recovery indicates a positive effect of the program, analyses by independent researchers have found serious problems with these conclusions."

"Reading Recovery efficacy studies do not use standard assessment measures. Most evaluations are restricted to the Reading Recovery developers’ own, nonstandard measures. These same measures are used to determine which students will be considered as part of the sample (continued versus discontinued students). Thus, outcomes are inflated and unconvincing to the research community."

"Reading Recovery developers have been and continue to be resistant to integrating the findings of independent, scientifically based reading research into their program and making it more cost effective. The failure to attend to research in modifying the program is its major downfall."

"In New Zealand, where Reading Recovery was developed, the programme has been independently examined on two occasions. Both studies found shortcomings. In essence, the programme is failing to meet the claims regarding its objectives and success. Senior Reading Recovery administrators have also overtly blocked attempts by graduate students to independently examine aspects of Reading Recovery."

And finally:

"Reading Recovery is not a productive investment of taxpayers’ money or students’ time and is a classic example of a “one size fits all” method. No single method works with all students. Methods like Reading Recovery that are rigidly implemented and limited in the number of components of effective reading instruction will not work with all students."

The senders describe their letter as "unsolicited" - "voicing concern over the widespread use of an educational approach whose claims are not supported by the scientific evidence". Furthermore they go on to say that they sent the letter because "Reading Recovery has undertaken a campaign to persuade Congressional members to ensure that Reading Recovery programs will be supported by Federal funds. The letter has been provided to these and other policy makers to offer an objective analysis of the programmatic and cost effectiveness" of RR.

About D-Ed Reckoning

The primary problem with K-12 education today is the problem of dead reckoning--an estimate based on little or no information. We don't know what a good K-12 education system is because we've never seen one operating. A good education system is one that is capable of educating almost every child.