Pseudo Critical Thinking in the Educational Establishment

A Case Study in Educational Malpractice

Abstract

Unfortunately, there is not simply good and bad thinking in the world, both easily recognized as such. There is also bad thinking that appears to be good and therefore wrongfully, sometimes disastrously, used as the basis of very important decisions. Very often this “bad thinking” is defended and “rationalized” in a highly sophisticated fashion. However flawed, it successfully counterfeits good thinking, and otherwise intelligent people are taken in. Such thinking is found in every dimension of human life and in every dimension it does harm; in every dimension it works against human well-being. Very often it is generated in a structural way, as a likely or probable by-product of how we have arranged and ordered things. This is illustrated in the American educational establishment.

The manner in which it is structured and operates makes likely the continuous generation of more bad, albeit highly sophisticated, thinking: pseudo-critical thinking, in short. However, because the educational bureaucracy is a powerful shaping force in education, bad thinking at the bureaucratic level leads directly to bad teaching at the classroom level.

In this chapter, Paul illustrates this destructive pattern using the California Department of Education (as his model of educational bureaucracy at work) and the new California State Reading and Writing Assessment instrument (as the resultant bad practice). He argues that this poorly designed assessment tool leads directly to bad teaching practices and the exacerbation of a profound problem in instruction: the failure to teach students to reason well in every subject they study. If the educational bureaucracy doesn’t understand what reasoning is and how to assess it in reading and writing, argues Paul, is it likely that higher order reading and writing will be taught? No, he claims. And thus the educational bureaucracy creates a deep and serious problem in education.

Introduction

Sometimes when people think poorly, they do so out of simple ignorance. They are making mistakes, they don’t know they are making mistakes, but they would willingly correct their mistakes if they were pointed out to them. Often mistakes in thinking are quite humble. No one is apt to take them for models of how to think.

Such thinking may be quite uncritical, but is not pseudo critical thinking. Pseudo critical thinking is a form of intellectual arrogance masked in self-delusion or deception, in which thinking which is deeply flawed is not only presented as a model of excellence of thought, but is also, at the same time, sophisticated enough to take many people in. No one takes a rock to be a counterfeit diamond. It is simply other than diamond. But a zircon mimics a diamond and is easily taken for one and hence can be said to be a pseudo diamond.

There is much “sophisticated” but deeply flawed thinking which is presented as a model for thought. This is nothing new in the history of thought and knowledge.

Medieval philosophy and theology, for example, was used as a sophisticated tool to resist, quite unknowingly of course, the advance of science. When deeply flawed thinking is embedded in teaching, then the development of thought and knowledge in the student is retarded or arrested. Teachers at every level of education, for example, tell students how to think. They point out thinking which they in effect encourage students to emulate. When what they point out as a model is deeply flawed, and yet sophisticated enough to take many in, it is a form of destructive pseudo critical thinking.

When deeply flawed thinking is embedded in teaching, then the development of thought and knowledge in the student is retarded or arrested.

Pseudo critical thinking is everywhere in the world, for everywhere there are people who take themselves to be models of good thinking and who are engaged in influencing others by their model. Sometimes they foster an approach to thinking quite explicitly — by, for example, designing a program that purports to foster critical thinking. But more often they simply implicitly propagandize for a form of flawed thinking, not aware of the thinking that they are modeling. In any case, it is a rare person, one who really does think critically, who recognizes fundamental flaws in his or her own thinking. Most people are victims of their bad thinking. They do not know how to analyze and assess thinking. Consequently, most believe that their thinking is instinctively and naturally of good quality. Most believe, in other words, that his or her own thinking is that of a fairminded person who judges persons and events in an impartial and accurate way. Often people, then, inadvertently buy into one or more kinds of pseudo critical thinking: in business, in politics, and, of course, in personal, emotional, and family life. The pseudo critical thinking that I propose to concentrate on in this chapter is pseudo critical thinking in the educational establishment.

I will use as my major illustration, the California State Department of Education’s new assessment tool for reading and writing. Its development and nature provide an illuminating example of how deeply-flawed thinking is generated and worked into the system, from the statewide to the classroom level. Of course, we must remember that there is local and statewide bureaucracy and that they exist in symbiosis, each feeding the other. And teachers themselves have learned to think the way they do in bureaucratic settings, so very often they are in effect asking for, from the system, what the system by its nature is ready to give them. It is therefore somewhat misleading to say that the flawed thinking at the statewide bureaucratic level is the cause — it is rather a cause — of flawed thinking in the classroom.

Before we proceed to our “exemplar,” I would like to set the stage for what we shall do by providing the reader in advance with one — hopefully intuitive — example of why it might be that flawed thinking is regularly generated in the educational establishment. It is important that the reader comes to see why the blunders and mistakes of the California reading and writing assessment, which I shall presently document, are not exceptions in a generally good record, but rather representative examples of a typically bad product in a system that, like many others, typically generates bad products.

Consider one way in which the educational environment invites flawed thinking. It is an environment in which many whose education may in fact have been quite narrow and flawed, (see “Research Findings,” p. 19) take themselves to be experts in one form of knowledge or another, and of course, not only in a form of knowledge per se but in the kind of thinking that has created or discovered the knowledge.

. . . most people recognize that there is something incoherent about saying that one is well educated but thinks poorly.

These experts — called teachers and administrators — are presumed to be qualified to tell the young not only what to think but how to think about mathematical, scientific, social, and literary questions, for example. It would be odd for someone to say, “I’m a qualified teacher but my thinking is deeply flawed.” That is to say, most people recognize that there is something incoherent about saying that one is well educated but thinks poorly. Imagine someone saying, “Jack is very well educated, but with just one minor exception; his thinking is unclear, imprecise, inaccurate, irrelevant, narrow, insignificant, and shallow. Other than that, he is well educated.” Clearly this would be absurd. Hence to believe oneself an educator is pretty much tantamount to believing oneself a critical thinker, at least in some academic domain. Chemistry teachers take themselves to be experts in sound chemical thinking. Math teachers take themselves to be experts in sound mathematical thinking, and so on.

Yet many educators have been miseducated. Many are poor reasoners. Many confuse issues and questions, are easily diverted from the relevant to the irrelevant. Many lack a comprehensive educational philosophy. Many do virtually no serious reading. Many cannot speak knowledgeably outside a narrow field. And many are not even up-to-date in their own field.

. . . many educators have been miseducated. Many are poor reasoners. Many confuse issues and questions, are easily diverted from the relevant to the irrelevant.

Furthermore, the educational environment dominant in the schools is not traditionally conducive to critical thinking or to the development of further learning on the part of teachers and administrators. Much of the in-service is episodic, intellectually unchallenging, and fragmented. At most schools there is very little discussion on or about serious educational issues, and when there is such discussion it is often simplistic. And that is not all. The kind of instruction that is prevalent at all levels is didactic instruction. The kind of testing that is prevalent is multiple-choice focused on recall. Most students pass their courses by relying on rote memorization. Most teachers, even college professors, passed most of their courses in the same way. (see “Research Findings,” p. 19) It is the thesis of this chapter that the models for thinking and the assessment of thinking presented in the schools are generally deeply flawed, and that the reason why this is so is systemic. I will also make recommendations at the end of this chapter as to the kind of action that is called for.

The Bureaucracy Ignores Reasoning & Intellectual Standards

Much of the pseudo critical thinking derives from the lack of a coherent understanding of the role of reasoning and intellectual standards in disciplined thought. What do I mean by this? Consider that as soon as we set our minds to the task of figuring anything out — a poem, a book, our bank account, a problem in our personal relationships, whatever — we are engaged in the task of reasoning, and reasoning can be done well or poorly. It can be assessed. And to assess it, we need intellectual standards.

The California Department of Education English Language Arts Assessment (ELAA) committees are not clear about the role of reasoning in reading and writing, and therefore they are not clear about the role of intellectual standards in the assessment of reasoning in reading and writing. Unfortunately, when one is confused on a basic point such as this, the confusion inevitably spreads to other matters as well. And so we should not be surprised to find a variety of confusions in their work.

I will enumerate for your convenience some of the major ones just below. In the next section, I list flaws characteristic of the educational establishment in general. Each item in this second list I analyze in detail, to provide a background set of understanding in preparation for an in-depth analysis of the California reading and writing assessment.

The general point, running through-out, is that the ill-constructed California reading and writing assessment is not an anomaly.

The reason for this is simple. If one understands the general pattern of misunderstanding, then specific instances of the pattern are much easier to see. A third list of flaws follows the analysis of the test. This final list makes clear the significance and instructional implications of the flawed character of the test. The general point, running throughout, is that the ill-constructed California reading and writing assessment is not an anomaly. The mistakes it makes are painfully predictable, mistakes being made all over the country in any number of ill-designed tests, in any number of ill-conceived curricula, in any number of ill-thought-through assignments.

You shall read, then, three lists of flaws. Remember that each has a somewhat different, but related, purpose. Now, the first list.

The California Department of Education English Language Arts Assessment materials, as we shall show below, contain all of the following flaws:

Its treatment of intellectual standards is confused and erroneous.

It confuses recall with knowledge.

It confuses subjective preference with reasoned judgment.

It confuses irrational with rational persuasion.

Its key terms are often vague.

Some key terms are dangerously ambiguous.

It inadvertently encourages “subjectivism.”

Its scoring is arbitrary.

It is both invalid and unreliable.

But before we look at the detail of these manifestations of pseudo critical thinking in the California Department of Education’s assessment materials, let’s make clearer what some of the common confusions of pseudo critical thinking amount to in the domain of educational assessment and why they occur. With these understandings in hand, it will be easier to explain what precisely is wrong with California’s reading and writing assessment.

What Does Pseudo Critical Thinking Look Like in Educational Assessment?

The advance of knowledge has been achieved not because the mind is capable of memorizing what teachers say but because it can be disciplined to ask probing questions and pursue them in a reasoned, self-critical way. Scholars pursuing knowledge submit their thinking to rigorous discipline, just as the discipline within which they think must itself submit to the broader discipline of more encompassing intellectual standards. Each academic discipline, in other words, develops special standards in virtue of its specialized concepts, procedures, and assumptions, but each also must submit to general standards that enable it to share its knowledge with all disciplines and enable all genuine knowledge to be integrated comprehensively and tested for coherence.

Pseudo critical thinking is revealed in educational assessment when the assessment theory or practice — or the approaches to teaching, thinking, or knowledge that follow from it — fails to take into account fundamental conditions for the pursuit or justification of knowledge.

All research must be put, therefore, into a form of reasoning taken seriously in a field and the reasoning must then submit to the reasoned critique of others, both within and ultimately without the field, who share not only its standards but the standards of good thinking generally. Every field must be intellectually accountable to every other field by demonstrating its commitment to clarity, precision, accuracy, relevance, consistency, depth, and coherence.

Pseudo critical thinking is revealed in educational assessment when the assessment theory or practice — or the approaches to teaching, thinking, or knowledge that follows from it — fails to take into account fundamental conditions for the pursuit or justification of knowledge. The result is the unwitting or unknowing encouragement of flawed thinking. What are some of the common ways, then, that the assessment of thinking or, indeed, any approach to the teaching of thinking might be flawed?

Here are three. These are not by any means the only ones, but they are very common, very basic, and very important.

First Basic FlawThe Lack or Misuse of Intellectual Standards

This is one of the most common flaws. It derives from the fact that though all of us think, and think continually, we have not been educated to analyze our thinking and assess it. We don’t have explicit standards already in mind to assess our thinking.

We may then fall back on “mental process words” to talk about good thinking, words such as analyzing, identifying, classifying, and evaluating.

. . . though all of us think, and think continually, we have not been educated to analyze our thinking and assess it.

These are words that name some of what thinking does. We use our thinking to identify things, to classify them, to analyze them, to apply them, and to evaluate them. It is tempting, then, to think of critical thinking as merely thinking engaged in identification, classification, analysis, application, evaluation, and the like. But it is important to remember that responsible critical thinking requires intellectual standards. Hence, it is not enough to classify, one must do it well, that is, in accord with the appropriate standards and criteria. Misclassification, though a form of classification, is not an ability. The same goes for analysis, application, and evaluation.

It might be helpful to remember that all critical thinking abilities have three parts: a process, an object, and a standard. Here are various critical thinking abilities which can serve as examples. As you read them see if you can identify the intellectual standard in each.

the ability to evaluate information for its relevance

the ability to accurately identify assumptions

the ability to construct plausible inferences

the ability to identify relevant points of view

the ability to distinguish significant from insignificant information

The standards used in these examples are “relevance,” “accuracy,” “plausibility,” and “significance.” Each of these standards would, needless to say, have to be contextualized. Nevertheless — and this is the key point — there can be no critical thinking without the use of intellectual standards.

Hence, if an approach to teaching or thinking focuses on the use of mental processes without a critical application of standards to that use, and persuades many to do the same, then, it is an example of pseudo critical thinking.

There can be no critical thinking without the use of intellectual standards.

There are in fact many such approaches in use in education today.

Second Basic FlawMisconceptions Built Into the System

Flaws occur when thinking or an approach to thinking embodies a misconception about the nature of thinking or about what makes for excellence in it. I will explain just two of the most common misconceptions. The first involves confusing reasoned judgment (which is one of the most important modes of thinking leading to the possibility of knowledge) with subjective preference (which is not a basis for attaining knowledge). The second misconception involves confusing recall (which is a lower order use of the mind) with knowledge (which requires higher order thinking). Here are the explanations in brief. See if you can follow the examples and relate them to your experience.

Reasoned Judgment Confused with Subjective PreferenceMany pseudo critical thinking approaches present all judgments as falling into two exclusive and exhaustive categories: fact and opinion. Actually, the kind of judgment most important to educated people and the kind we most want to foster falls into a third, very important, and now almost totally ignored category, that of reasoned judgment. A judge in a court of law is expected to engage in reasoned judgment; that is, the judge is expected not only to render a judgment, but also to base that judgment on sound, relevant evidence and valid legal reasoning. A judge is not expected to base his judgments on his subjective preferences, on his personal opinions, as such. You might put it this way, judgment based on sound reasoning goes beyond, and is never to be equated with, fact alone or mere opinion alone.

Facts are typically used in reasoning, but good reasoning does more than state facts. Furthermore, a position that is well-reasoned is not to be described as simply “opinion.” Of course, we sometimes call the judge’s verdict an “opinion,” but we not only expect, we demand that it be based on relevant and sound reasoning.

Here’s a somewhat different way to put this same point. It is essential when thinking critically to clearly distinguish three different kinds of questions: 1) those with one right answer (factual questions fall into this category), 2) those with better or worse answers (well-reasoned or poorly reasoned answers), and 3) those with as many answers as there are different human preferences (a category in which mere opinion does rule).

When questions that require better or worse answers are treated as matters of opinion, pseudo-critical thinking occurs.

Here are examples of the three types: 1) What is the boiling point of lead? 2) How can we best address the most basic and significant economic problems of the nation today? and 3) Which would you prefer, a vacation in the mountains or one at the seashore? Only the third kind of question is a matter of sheer opinion. The second kind is a matter of reasoned judgment — we can rationally evaluate answers to the question (using universal intellectual standards such as clarity, depth, consistency and so forth).

When questions that require better or worse answers are treated as matters of opinion, pseudo critical thinking occurs. Students come, then, to uncritically assume that everyone’s “opinion” is of equal value. Their capacity to appreciate the importance of intellectual standards diminishes, and we can expect to hear questions such as these: What if I don’t like these standards? Why shouldn’t I use my own standards? Don’t I have a right to my own opinion? What if I’m just an emotional person? What if I like to follow my intuition? What if I don’t believe in being “rational?” They then fail to see the difference between offering legitimate reasons and evidence in support of a view and simply asserting the view as true. The failure to teach students to recognize, value, and respect good reasoning is one of the most significant failings of education today.

{"id":"266","title":"","author":"","content":"<p dir=\"ltr\"><em><span style=\"color: #333333;\"><span style=\"color: #666666;\"><strong>Introduction </strong></span><br /> </span></em><br /> <span style=\"font-family: Arial,Helvetica,sans-serif;\">Sometimes when people think poorly, they do so out of simple ignorance. They are making mistakes, they don&rsquo;t know they are making mistakes, but they would willingly correct their mistakes if they were pointed out to them. Often mistakes in thinking are quite humble. No one is apt to take them for models of how to think.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Such thinking may be quite uncritical, but is not pseudo critical thinking. Pseudo critical thinking is a form of intellectual arrogance masked in self-delusion or deception, in which thinking which is deeply flawed is not only presented as a model of excellence of thought, but is also, at the same time, sophisticated enough to take many people in. No one takes a rock to be a counterfeit diamond. It is simply other than diamond. But a zircon mimics a diamond and is easily taken for one and hence can be said to be a pseudo diamond. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">There is much &ldquo;sophisticated&rdquo; but deeply flawed thinking which is presented as a model for thought. This is nothing new in the history of thought and knowledge.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Medieval philosophy and theology, for example, was used as a sophisticated tool to resist, quite unknowingly of course, the advance of science. When deeply flawed thinking is embedded in teaching, then the development of thought and knowledge in the student is retarded or arrested. Teachers at every level of education, for example, tell students how to think. They point out thinking which they in effect encourage students to emulate. When what they point out as a model is deeply flawed, and yet sophisticated enough to take many in, it is a form of destructive pseudo critical thinking.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"> </span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; When deeply flawed thinking is embedded in teaching, then the development of thought and knowledge in the student is retarded or arrested.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"> </span></p>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Pseudo critical thinking is everywhere in the world, for everywhere there are people who take themselves to be models of good thinking and who are engaged in influencing others by their model. Sometimes they foster an approach to thinking quite explicitly &mdash; by, for example, designing a program that purports to foster critical thinking. But more often they simply implicitly propagandize for a form of flawed thinking, not aware of the thinking that they are modeling. In any case, it is a rare person, one who really does think critically, who recognizes fundamental flaws in his or her own thinking. Most people are victims of their bad thinking. They do not know how to analyze and assess thinking. Consequently, most believe that their thinking is instinctively and naturally of good quality. Most believe, in other words, that his or her own thinking is that of a fairminded person who judges persons and events in an impartial and accurate way. Often people, then, inadvertently buy into one or more kinds of pseudo critical thinking: in business, in politics, and, of course, in personal, emotional, and family life. The pseudo critical thinking that I propose to concentrate on in this chapter is pseudo critical thinking in the educational establishment.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">I will use as my major illustration, the California State Department of Education&rsquo;s new assessment tool for reading and writing. Its development and nature provide an illuminating example of how deeply-flawed thinking is generated and worked into the system, from the statewide to the classroom level. Of course, we must remember that there is local and statewide bureaucracy and that they exist in symbiosis, each feeding the other. And teachers themselves have learned to think the way they do in bureaucratic settings, so very often they are in effect asking for, from the system, what the system by its nature is ready to give them. It is therefore somewhat misleading to say that the flawed thinking at the statewide bureaucratic level is the cause &mdash; it is rather a cause &mdash; of flawed thinking in the classroom.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Before we proceed to our &ldquo;exemplar,&rdquo; I would like to set the stage for what we shall do by providing the reader in advance with one &mdash; hopefully intuitive &mdash; example of why it might be that flawed thinking is regularly generated in the educational establishment. It is important that the reader comes to see why the blunders and mistakes of the California reading and writing assessment, which I shall presently document, are not exceptions in a generally good record, but rather representative examples of a typically bad product in a system that, like many others, typically generates bad products.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Consider one way in which the educational environment invites flawed thinking. It is an environment in which many whose education may in fact have been quite narrow and flawed, (see &ldquo;Research Findings,&rdquo; p. 19) take themselves to be experts in one form of knowledge or another, <span style=\"font-family: Arial,Helvetica,sans-serif;\">and of course, not only in a form of knowledge per se but in the kind of thinking that has created or discovered the knowledge.</span></span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; . . . most people recognize that there is something incoherent about saying that one is well educated but thinks poorly.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">These experts &mdash; called teachers and administrators &mdash; are presumed to be qualified to tell the young not only what to think but how to think about mathematical, scientific, social, and literary questions, for example. It would be odd for someone to say, &ldquo;I&rsquo;m a qualified teacher but my thinking is deeply flawed.&rdquo; That is to say, most people recognize that there is something incoherent about saying that one is well educated but thinks poorly. Imagine someone saying, &ldquo;Jack is very well educated, but with just one minor exception; his thinking is unclear, imprecise, inaccurate, irrelevant, narrow, insignificant, and shallow. Other than that, he is well educated.&rdquo; Clearly this would be absurd. Hence to believe oneself an educator is pretty much tantamount to believing oneself a critical thinker, at least in some academic domain. Chemistry teachers take themselves to be experts in sound chemical thinking. Math teachers take themselves to be experts in sound mathematical thinking, and so on. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Yet many educators have been miseducated. Many are poor reasoners. Many confuse issues and questions, are easily diverted from the relevant to the irrelevant. Many lack a comprehensive educational philosophy. Many do virtually no serious reading. Many cannot speak knowledgeably <span style=\"font-family: Arial,Helvetica,sans-serif;\">outside a narrow field. And many are not even up-to-date in their own field. </span></span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; . . . many educators have been miseducated. Many are poor reasoners. Many confuse issues and questions, are easily diverted from the relevant to the irrelevant.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Furthermore, the educational environment dominant in the schools is not traditionally conducive to critical thinking or to the development of further learning on the part of teachers and administrators. Much of the in-service is episodic, intellectually unchallenging, and fragmented. At most schools there is very little discussion on or about serious educational issues, and when there is such discussion it is often simplistic. And that is not all. The kind of instruction that is prevalent at all levels is didactic instruction. The kind of testing that is prevalent is multiple-choice focused on recall. Most students pass their courses by relying on rote memorization. Most teachers, even college professors, passed most of their courses in the same way. (see &ldquo;Research Findings,&rdquo; p. 19) It is the thesis of this chapter that the models for thinking and the assessment of thinking presented in the schools are generally deeply flawed, and that the reason why this is so is systemic. I will also make recommendations at the end of this chapter as to the kind of action that is called for.</span></p>\r\n<h4><em><span style=\"color: #666666;\">The Bureaucracy Ignores Reasoning &amp; Intellectual Standards</span></em></h4>\r\n<p>&nbsp;<span style=\"font-family: Arial,Helvetica,sans-serif;\">Much of the pseudo critical thinking derives from the lack of a coherent understanding of the role of reasoning and intellectual standards in disciplined thought. What do I mean by this? Consider that as soon as we set our minds to the task of figuring anything out &mdash; a poem, a book, our bank account, a problem in our personal relationships, whatever &mdash; we are engaged in the task of reasoning, and reasoning can be done well or poorly. It can be assessed. And to assess it, we need intellectual standards. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The California Department of Education English Language Arts Assessment </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">(ELAA) committees are not clear about the role of reasoning in reading and writing, and therefore they are not clear about the role of intellectual standards in the assessment of reasoning in reading and writing. Unfortunately, when one is confused on a basic point such as this, the confusion inevitably spreads to other matters as well. And so we should not be surprised to find a variety of confusions in their work.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">I will enumerate for your convenience some of the major ones just below. In the next section, I list flaws characteristic of the educational establishment in general. Each item in this second list I analyze in detail, to provide a background set of understanding in preparation for an in-depth analysis of the California reading and writing assessment.</span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; The general point, running through-out, is that the ill-constructed California reading and writing assessment is not an anomaly.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The reason for this is simple. If one understands the general pattern of misunderstanding, then specific instances of the pattern are much easier to see. A third list of flaws follows the analysis of the test. This final list makes clear the significance and instructional implications of the flawed character of the test. The general point, running throughout, is that the ill-constructed California reading and writing assessment is not an anomaly. The mistakes it makes are painfully predictable, mistakes being made all over the country in any number of ill-designed tests, in any number of ill-conceived curricula, in any number of ill-thought-through assignments. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">You shall read, then, three lists of flaws. Remember that each has a somewhat different, but related, purpose. Now, the first list.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The California Department of Education English Language Arts Assessment materials, as we shall show below, contain all of the following flaws: </span></p>\r\n<ul>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Its treatment of intellectual standards is confused and erroneous.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">It confuses recall with knowledge.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">It confuses subjective preference with reasoned judgment.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">It confuses irrational with rational persuasion.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Its key terms are often vague.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Some key terms are dangerously ambiguous.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">It inadvertently encourages &ldquo;subjectivism.&rdquo; </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Its scoring is arbitrary. </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">It is both invalid and unreliable.</span> </li>\r\n</ul>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">But before we look at the detail of these manifestations of pseudo critical thinking in the California Department of Education&rsquo;s assessment materials, let&rsquo;s make clearer what some of the common confusions of pseudo critical thinking amount to in the domain of educational assessment and why they occur. With these understandings in hand, it will be easier to explain what precisely is wrong with California&rsquo;s reading and writing assessment.</span>&nbsp;</p>\r\n<h4><em><span style=\"color: #666666;\">What Does Pseudo Critical Thinking Look Like in Educational Assessment?</span></em></h4>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The advance of knowledge has been achieved not because the mind is capable of memorizing what teachers say but because it can be disciplined to ask probing questions and pursue them in a reasoned, self-critical way. Scholars pursuing knowledge submit their thinking to rigorous discipline, just as the discipline within which they think must itself submit to the broader discipline of more encompassing intellectual standards. Each academic discipline, in other words, develops special standards in virtue of its specialized concepts, procedures, and assumptions, but each also must submit to general standards that enable it to share its knowledge with all disciplines and enable all genuine <span style=\"font-family: Arial,Helvetica,sans-serif;\">knowledge to be integrated comprehensively and tested for coherence.</span></span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; Pseudo critical thinking is revealed in educational assessment when the assessment theory or practice &mdash; or the approaches to teaching, thinking, or knowledge that follow from it &mdash; fails to take into account fundamental conditions for the pursuit or justification of knowledge.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">All research must be put, therefore, into a form of reasoning taken seriously in a field and the reasoning must then submit to the reasoned critique of others, both within and ultimately without the field, who share not only its standards but the standards of good thinking generally. Every field must be intellectually accountable to every other field by demonstrating its commitment to clarity, precision, accuracy, relevance, consistency, depth, and coherence.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Pseudo critical thinking is revealed in educational assessment when the assessment theory or practice &mdash; or the approaches to teaching, thinking, or knowledge that follows from it &mdash; fails to take into account fundamental conditions for the pursuit or justification of knowledge. The result is the unwitting or unknowing encouragement of flawed thinking. What are some of the common ways, then, that the assessment of thinking or, indeed, any approach to the teaching of thinking might be flawed?</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Here are three. These are not by any means the only ones, but they are very common, very basic, and very important.</span></p>\r\n<h5><span style=\"font-size: small;\"><span style=\"color: #000099;\">First Basic Flaw<br /> </span><span style=\"color: #666666;\">The Lack or Misuse of Intellectual Standards</span></span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">This is one of the most common flaws. It derives from the fact that though all of us think, and think continually, we have not been educated to analyze our thinking and assess it. We don&rsquo;t have explicit standards already in mind to assess our thinking.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">We may then fall back on &ldquo;mental process words&rdquo; to talk about good thinking, words such as analyzing, <span style=\"font-family: Arial,Helvetica,sans-serif;\">identifying, classifying, and evaluating.</span></span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>. . . though all of us think, and think continually, we have not been educated to analyze our thinking and assess it.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">These are words that name some of what thinking does. We use our thinking to identify things, to classify them, to analyze them, to apply them, and to evaluate them. It is tempting, then, to think of critical thinking as merely thinking engaged in identification, classification, analysis, application, evaluation, and the like. But it is important to remember that responsible critical thinking requires intellectual standards. Hence, it is not enough to classify, one must do it well, that is, in accord with the appropriate standards and criteria. Misclassification, though a form of classification, is not an ability. The same goes for analysis, application, and evaluation.</span></p>\r\n<table border=\"1\" cellspacing=\"1\" cellpadding=\"1\" width=\"100%\">\r\n<tbody>\r\n<tr>\r\n<td>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">It might be helpful to remember that all critical thinking abilities have three parts: a process, an object, and a standard. Here are various critical thinking abilities which can serve as examples. As you read them see if you can identify the intellectual standard in each. </span></p>\r\n<ul>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the ability to evaluate information for its relevance</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the ability to accurately identify assumptions </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the ability to construct plausible inferences </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the ability to identify relevant points of view </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the ability to distinguish significant from insignificant information</span> </li>\r\n</ul>\r\n</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The standards used in these examples are &ldquo;relevance,&rdquo; &ldquo;accuracy,&rdquo; &ldquo;plausibility,&rdquo; and &ldquo;significance.&rdquo; Each of these standards would, needless to say, have to be contextualized. Nevertheless &mdash; and this is the key point &mdash; there can be no critical thinking without the use of intellectual standards.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Hence, if an approach to teaching or thinking focuses on the use of mental processes without a critical application of standards to that use, and persuades many to do the same, <span style=\"font-family: Arial,Helvetica,sans-serif;\">then, it is an example of pseudo critical thinking.</span></span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; There can be no critical thinking without the use of intellectual standards.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">There are in fact many such approaches in use in education today.</span></p>\r\n<h5><span style=\"font-size: small;\"><span style=\"color: #000099;\">Second Basic Flaw<br /> </span><span style=\"color: #666666;\">Misconceptions Built Into the System</span></span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Flaws occur when thinking or an approach to thinking embodies a misconception about the nature of thinking or about what makes for excellence in it. I will explain just two of the most common misconceptions. The first involves confusing reasoned judgment (which is one of the most important modes of thinking leading to the possibility of knowledge) with subjective preference (which is not a basis for attaining knowledge). The second misconception involves confusing recall (which is a lower order use of the mind) with knowledge (which requires higher order thinking). Here are the explanations in brief. See if you can follow the examples and relate them to your experience.<br /> <br /> </span><strong><em><span style=\"color: #000000;\">Reasoned Judgment Confused with Subjective Preference</span></em></strong><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> </span><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">Many pseudo critical thinking approaches present all judgments as falling into two exclusive and exhaustive categories: fact and opinion. Actually, the kind of judgment most important to educated people and the kind we most want to foster falls into a third, very important, and now almost totally ignored category, that of reasoned judgment. A judge in a court of law is expected to engage in reasoned judgment; that is, the judge is expected not only to render a judgment, but also to base that judgment on sound, relevant evidence and valid legal reasoning. A judge is not expected to base his judgments on his subjective preferences, on his personal opinions, as such. You might put it this way, judgment based on sound reasoning goes beyond, and is never to be equated with, fact alone or mere opinion alone.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Facts are typically used in reasoning, but good reasoning does more than state facts. Furthermore, a position that is well-reasoned is not to be described as simply &ldquo;opinion.&rdquo; Of course, we sometimes call the judge&rsquo;s verdict an &ldquo;opinion,&rdquo; but we not only expect, we demand that it be based on relevant and sound reasoning.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Here&rsquo;s a somewhat different way to put this same point. It is essential when thinking critically to clearly distinguish three different kinds of questions: 1) those with one right answer (factual questions fall into this category), 2) those with better or worse answers (well-reasoned or poorly reasoned answers), and 3) those with as many answers as there are different human preferences (a category in which mere opinion does rule).</span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><em><span>&nbsp;&nbsp; When questions that require better or worse answers are treated as matters of opinion, pseudo-critical thinking occurs.</span></em></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Here are examples of the three types: <strong>1)</strong> What is the boiling point of lead? <strong>2)</strong> How can we best address the most basic and significant economic problems of the nation today? and <strong>3)</strong> Which would you prefer, a vacation in the mountains or one at the seashore? Only the third kind of question is a matter of sheer opinion. The second kind is a matter of reasoned judgment &mdash; we can rationally evaluate answers to the question (using universal intellectual standards such as clarity, depth, consistency and so forth). </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">When questions that require better or worse answers are treated as matters of opinion, pseudo critical thinking occurs. Students come, then, to uncritically assume that everyone&rsquo;s &ldquo;opinion&rdquo; is of equal value. Their capacity to appreciate the importance of intellectual standards diminishes, and we can expect to hear questions such as these: What if I don&rsquo;t like these standards? Why shouldn&rsquo;t I use my own standards? Don&rsquo;t I have a right to my own opinion? What if I&rsquo;m just an emotional person? What if I like to follow my intuition? What if I don&rsquo;t believe in being &ldquo;rational?&rdquo; They then fail to see the difference between offering legitimate reasons and evidence in support of a view and simply asserting the view as true. </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">The failure to teach students to recognize, value, and respect good reasoning is one of the most significant failings of education today.</span></p>\r\n<p><br style=\"clear: both;\" /></p>","public_access":"1","public_downloads":"1","sku":"","files":[],"images":[]}

Recall Confused With Knowledge

A second common confusion which leads directly to pseudo critical thinking is recall confused with knowledge. As I suggested above this confusion is deeply embedded in the minds of many “educators.” It results from the fact that most instruction involves didactic lectures and most testing relies fundamentally on recall. Educators confuse students recalling what was said in the lecture with knowing the how and the why behind what was said. For example, a teacher might give you information, some of which is true and some of which is not, and you may not know which is which. Another way to see this point is to figure out why we don't think of parrots as gaining any knowledge when they learn to repeat words. Tape recorders get no credit for knowledge either. Do you see the point?

We tend to assume, to carry the point a bit further, that all information in a textbook is correct. Some, of course, is not. We attain genuine knowledge only when the information we possess is not only correct but, additionally, we know that it is and why it is. So, strictly speaking, I don’t know that something is true or correct if I have merely found it asserted to be so in a book. I need to have a greater understanding — for example, I need to know what supports it, what makes it true — to properly be said to know it.

So if someone tells me Jack has flown to Paris for the weekend, I don’t know if he actually did. I might believe that he had (because I trust the person who told me) and my belief might even be correct (through happen stance), but still I don’t yet know for sure that he did. I am operating on the basis of “hearsay.”

We attain genuine knowledge only when the information we possess is not only correct but, additionally, we know that it is and why it is.

The failure to appreciate the significance of this distinction causes a lot of problems in schooling because many who teach do not really know their own subjects well enough to explain clearly why this or that is so, and why this or that is not so. They know what the textbook says, certainly, but not why the textbook says what it says, or whether what it says is so or not. Having knowledge (for such confused persons) is nothing other than remembering what the textbook said.

Third Basic FlawThe Misuse of Intellect

“Skilled” thinking can easily be used to obfuscate rather than to clarify, to maintain a prejudice rather than to break it down, to aid in the defense of a narrow interest rather than to take into account the public good. If we teach students to think narrowly, without an adequate emphasis on the essential intellectual traits of mind (intellectual humility, intellectual honesty, fairmindedness, etc.) the result can then be the inadvertent cultivation of the manipulator, the propagandist, and the con artist. We unknowingly end up, then, undermining the basic values of education and public service, properly conceived.

It is extremely important to see that intelligence and intellect can be used for ends other than those of gaining “truth” or “insight” or “knowledge.” One can learn to be cunning rather than clever, smooth rather than clear, convincing rather than rationally persuasive, articulate rather than accurate. One can become judgmental rather than gain in judgment. One can confuse confidence with knowledge at the same time that one mistakes arrogance for self-confidence. In each of these cases a counterfeit of a highly desirable trait is developed in place of that trait.

One can learn to be cunning rather than clever, smooth rather than clear, convincing rather than rationally persuasive, articulate rather than accurate.

There are many people who have learned to be skilled in merely appearing to be rational and knowledgeable when, in fact, they are not. Some of these have learned to be smooth, articulate, confident, cunning, and arrogant. They lack rational judgment, but this does not dissuade them from issuing dogmatic judgments and directives. They impress and learn to control others, quite selfishly. Unless we carefully design schooling to serve the “higher” ends of education, it can easily, as it now often does, degenerate into merely serving “lower” ends. When this happens, schooling often does more harm than good. It spreads the influence and resultant harm of pseudo critical thinking.

With the above understandings in mind, we are prepared to examine the new California Assessment Program and its evaluation of reading and writing.

The California Assessment Program English Language Arts AssessmentIntroduction

California has developed the reputation of being a leader in educational reform. It was the first state to mandate critical thinking instruction at all educational levels.

However, it is now becoming apparent that at the K–12 level at least, the mandate is not on solid ground, for pseudo critical thinking approaches, and the misunderstandings that underlie them, are becoming rampant in the state. The jargon of reform is everywhere, but substance is virtually nowhere. Unfortunately, the California Department of Education is oblivious of the danger, in fact, is very much part of the problem. Not only is it failing to provide sound leadership in integrating critical thinking into instruction, it is developing an assessment program which is shot-through with pseudo critical thinking confusions.

It is now deeply involved in developing what it calls “authentic” assessment that focuses on student “performances” found in the student “construction of meaning” in language arts and social studies. Now all of these terms — “authentic” and “performance” and “construction of meaning” — are part of the buzz words of the day in educational circles. Of course, the theoretical insights that led to emphasis on these words are important, so let’s briefly review them.

Testing and assessment in this country has come under increasing fire, and for good reason. Much of what has traditionally been tested in the popular, machine scorable, multiple choice tests has contributed to little more than trivial pursuit, more and more emphasis in instruction on the lowest order of thinking: rote memorization.

Growing numbers of critics have pointed out that the items on which we have been testing students do not involve reasoning and have little relationship to the kinds of tasks that students will later be called upon to “perform.” The tests fail, in other words, to “authentically” test higher order “performances.” The reform of assessment has increasingly looked to an increased emphasis on “authentic” items that involve “performances” of a “higher order.”

Furthermore, research by cognitive psychologists and others have clearly established the fact that when humans deeply learn something — in contrast to, say, storing it temporarily in short-term memory — that learning involves the “construction of meaning.”

The reform of assessment has increasingly looked to an increased emphasis on "authentic" items that involve "performances" of a "higher order."

Here’s how you might look at it: In order to get about successfully in the world in which we live we have to continually “make sense” of things, to give a meaning to what is surrounding us. As we do this we develop networks, systems of meanings that enable us quickly to size up what’s in front of us. The result is we don’t see “meaningless” colors and shapes and sounds. We see trees, and people, and dogs, and speeding cars, and smog. We immediately construct “meaning” out of our experience. Our experience is made by our minds to “fit into” meanings we have already constructed, or, if we cannot do this, we set about constructing a new meaning out of the old ones.

Now, what puzzled educational researchers was what has come to be called the problem of “transfer.” Why don’t students take what they are studying at school and use it in their daily acts of “constructing meanings?” Why don’t they use scientific concepts when they make everyday predictions or form everyday theories about people and events? Why don’t they use concepts from their social studies textbooks when they go about interpreting social situations and trying to figure out solutions to their social problems? Their conclusion was that the students don’t use what they study in school in their everyday life because they are not engaged in the construction of meaning in class. In class, they are merely, or at least mainly, memorizing, not constructing meaning, not integrating school learning with everyday life.

Now we are ready to bring the three theoretical concepts together — “authentic,” “performance,” and the “construction of meaning.” In authentic performances students construct meaning. They do not simply memorize. So why not focus school instruction on just such matters? Why not give them tasks that are “authentic?” Why not help them, in “performing” those tasks, to actually “construct meaning,” in other words, to integrate what they are learning into the network of meanings they are already using to make sense of the world. This is the basic theoretical idea behind the ELAA materials, and, as far as it goes, there can be no objection. But as one wise person once said, “The important truths are in the details.” And the details of the ELAA materials are horrendous.

A close examination of the details of the California Language Arts Assessment reveal that it is flawed in all of the following ways:

The overall conception is not theoretically coherent. It is filled with vagueness and confusion. This is probably the result of the committee adopting the key buzz words without clearly understanding the theory underlying them. The buzz words are then used vaguely and the details are filled out with terms from the agendas of the various stake-holders.

It does not provide a realistic model of reasoning, of critical reading and writing. Indeed, it is clear that the developers of the assessment do not realize that both reading and writing intrinsically involve the use of reasoning and that reasoning can be done well or poorly.

The overall conception does not call attention to definite and clear intellectual standards. The criteria given are typically vague and applied inconsistently. Important intellectual standards are missing.

The test, in fact, leads the teachers in the direction of malpractice, that is, into the systematic misassessment of reading and writing, leading the students in turn to become inaccurate, imprecise, and undisciplined readers and writers.

There is no way that a teacher might grasp an organized and systematic approach to the role of reasoning in reading and writing by studying the materials being disseminated. The test, in fact, leads the teachers in the direction of malpractice, that is, into the systematic misassessment of reading and writing, leading the students in turn to become inaccurate, imprecise, and undisciplined readers and writers.

A Tangle of Confusions

Let’s now look at the details and shed light on some of the theoretical confusions that undermine the approach.

First of all, the ELAA commentators open by confusing knowledge with recall and “constructing meaning” with “reasonably constructing meaning.” Since these confusions are basic and lead to indiscriminate scoring, let’s look at how this occurs.

California’s New English-Language Arts Assessment: An Integrated Look, begins by announcing a paradigm shift. As the English Language Arts Assessment (ELAA) document explains, “At the heart of the framework is a paradigm shift in which ‘constructing meaning’ replaced ‘gaining knowledge’ as the primary goal.” Or, as it says later, “Since the construction of meaning is the essence of both reading and writing, the new assessment allows students to shape the outcome rather than to identify correct meanings that test makers have posited.”

It is clear that the writers of the assessment are either not clear about the difference between recall and knowledge, or they are wrongly assuming that the attainment of knowledge is not intrinsically connected to the construction of meaning, or both. Briefly, let’s make these relationships clear.

Rational and Irrational Constructions

“Constructing meaning” is a process that is common to all learning which becomes deep-seated in the mind of the learner. It applies, however, just as much to the formation of flawed, irrational meanings as it does to the formulation of defensible, rational meanings. Deep-seated irrational fears, for example, result as much from the personal construction of meaning as do insights and understandings.

Knowledge, on the other hand, though also the result of the construction of meaning, requires a clear-cut exercise of the rational faculties of the mind. For example, to appropriately judge a person accused of murder to be guilty or innocent, one’s thinking must be guided by a careful and rational use of evidence, legal criteria (the criteria for “murder”), and the canons of sound reasoning. When a jury appropriately attains the knowledge of guilt or innocence, that knowledge, expressed in their verdict, is a product of a rational, a reasoned, construction of meaning. Of course, a jury may not function as it ought. It may be irrational and prejudiced, and the judge may overturn its verdict precisely because it did not properly discharge its responsibility to be “rational.” There is nothing wrong, therefore, with focusing attention on the need of students to “construct meaning” but it must be underscored that the mere construction of meaning, as such, is not a significant achievement, since it is done as much by Archie Bunker as by Einstein.

But the authors of The California Student Assessment System are confused on this point, for they talk as if the construction of meaning is an end in itself. They forget that “prejudice,” “stereotypes,” “misconceptions,” “illusions,” “delusion,” “self-deceptions,” “false beliefs,” and all manner of other intellectually flawed creations of the mind, are just as much “constructions” and as “meaningful” constructions as ones more insightful and discerning.

We should rather be interested in fostering in children adherence to those intellectual standards that maximize their construction of genuine “knowledge,” for otherwise they are likely to engage in a great deal of “irrational” construction and they will not know they are doing so.

. . . it must be underscored that the mere construction of meaning, as such, is not a significant achievement, since it is done as much by Archie Bunker as by Einstein.

Education must discriminate between the quality of students’ constructions of meaning, both in their reading and in their writing. But this can only defensibly be done by judging them by means of those intellectual standards common to educated thought. A construct that is unclear is not to be confused with one that is clear. One that is inaccurate is not to be confused with one that is accurate. One that is relevant to an assigned task is not to be confused with one that is irrelevant. One that is superficial is not to be confused with one that is deep.

Hence we do not need to decide between emphasizing the construction of meaning and the goal of attaining knowledge. If we properly understand the “dual” character of “meaning construction,” we will immediately recognize the need to focus on the “reasoned” and “reasonable” construction of meaning, and not indiscriminately credit any construction of meaning.

To underscore the point, the human mind naturally and inevitably constructs meaning. The mere fact that students construct meanings tells us nothing about the quality of those constructs. For example, in the extended example on reading as a form of thinking which we cited (pp. 24–27), both readers, Stephen and Colleen, constructed meanings. But you will remember that the meanings constructed by Colleen were absurd. Both students reasoned about the text’s meaning, but there was a stark contrast in the quality of reasoning in the two cases.

The point should now be clear. We want to work with our students’ capacity to construct meaning from a text, but we want to do this while teaching them to discipline their reading, to learn how to fit their interpretations to the logic of the words of the text. We want them to develop definite intellectual standards for their reading and not feel free to treat a text as if it were “silly putty,” to be shaped into any “meaning” they choose.

In passing, in the workplace, there is no economic value in constructing irrational, fanciful, lively, and entertaining meaning if it is irrational and reflects a flawed understanding of the text. Employers are not looking for a flashy, individualized response to a piece of writing, but rather a solid grasp of the meaning intended by the author. This is a fundamental premise of written communication!

Employers are not looking for a flashy, individualized response to a piece of writing, but rather a solid grasp of the meaning intended by the author. This is a fundamental premise of written communication!

We should therefore continually underscore the pivotal role of intellectual standards, not only in assessment but in any form of intellectual work whatsoever, including of course, reading and writing.

A Pseudo Commitment to Intellectual Standards

Intellectual standards are essential to the appropriate assessment of reading and writing. At some level, the assessment authors are aware of this necessity. Their description of their own criteria imply both impartiality and commitment to intellectual standards. For example, the authors of the English-Language Arts Assessment often speak of their commitment to “encourage students to read widely and in depth.” (Depth implies criteria for distinguishing “deep” from “shallow” readings.) Secondly, they imply impartial assessment when they state the three-fold purpose of the new English-Language Arts Assessment (p. I-2):

1) To establish standards for evaluating students’ performance when they read diverse kinds of materials for different purposes.

2) To measure how well students are able to construct meaning.

3) To improve the instructional program by providing an assessment that reflects the Framework.

They use language that implies a concern with rational judgment: for example, the ELAA report says when speaking of “meaning-making,” that,

We want students to think critically as they explore interests, clarify values, solve problems, resolve conflicts, generate new ideas, synthesize/apply learnings, set goals, and make decisions in response to the literature they read. (p. I - 2) Now, the processes of thinking critically, clarifying values, solving problems, setting goals, and making decisions all presuppose the importance of rationality, of engaging in sound reasoning. To clarify values, for example, requires that we rationally analyze them.

It becomes clear, from here on, that the California testing experts’ glossy, global statements about critical thinking and meaning-making and standards are losing their luster.

Solving problems is not the product of an arbitrary construction of meaning, but requires, amongst other things, an objective and accurate analysis of the nature of the problems, of the information relevant to the problems, and such like. Effective goal setting requires that we accurately identify possible competing goals and reasonably assess which make most sense. And certainly, decisions can reasonably or unreasonably be arrived at. It becomes clear, from here on, that the California testing experts’ glossy, global statements about critical thinking and meaning-making and standards are losing their luster. Empty platitudes and vacuous ideals are a specialty of virtually all bureaucracies. So common are they now that they are hardly noticed anymore.

{"id":"267","title":"","author":"","content":"<p><span style=\"color: #000000; font-family: Arial,Helvetica,sans-serif;\"><strong><em>Recall Confused With Knowledge</em></strong></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">A second common confusion which leads directly to pseudo critical thinking is recall confused with knowledge. As I suggested above this confusion is deeply embedded in the minds of many &ldquo;educators.&rdquo; It results from the fact that most instruction involves didactic lectures and most testing relies fundamentally on recall. Educators confuse students recalling what was said in the lecture with knowing the how and the why behind what was said. For example, a teacher might give you information, some of which is true and some of which is not, and you may not know which is which. Another way to see this point is to figure out why we don't think of parrots as gaining any knowledge when they learn to repeat words. Tape recorders get no credit for knowledge either. Do you see the point?</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">We tend to assume, to carry the point a bit further, that all information in a textbook is correct. Some, of course, is not. We attain genuine knowledge only when the information we possess is not only correct but, additionally, we know that it is and why it is. So, strictly speaking, I don&rsquo;t know that something is true or correct if I have merely found it asserted to be so in a book. I need to have a greater understanding &mdash; for example, I need to know what supports it, what makes it true &mdash; to properly be said to know it. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">So if someone tells me Jack has flown to Paris for the weekend, I don&rsquo;t know if he actually did. I might believe that he had (because I trust the person who told me) and my belief might even be correct (through happen stance), but still I don&rsquo;t yet know for sure that he did. I am operating on the basis of &ldquo;hearsay.&rdquo; </span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; We attain genuine knowledge only when the information we possess is not only correct but, additionally, we know that it is and why it is.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The failure to appreciate the significance of this distinction causes a lot of problems in schooling because many who teach do not really know their own subjects well enough to explain clearly why this or that is so, and why this or that is not so. They know what the textbook says, certainly, but not why the textbook says what it says, or whether what it says is so or not. Having knowledge (for such confused persons) is nothing other than remembering what the textbook said.</span>&nbsp;</p>\r\n<h5><span style=\"font-size: small;\"><span style=\"color: #000099;\">Third Basic Flaw<br /> </span><span style=\"color: #666666;\">The Misuse of Intellect</span></span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">&ldquo;Skilled&rdquo; thinking can easily be used to obfuscate rather than to clarify, to maintain a prejudice rather than to break it down, to aid in the defense of a narrow interest rather than to take into account the public good. If we teach students to think narrowly, without an adequate emphasis on the essential intellectual traits of mind (intellectual humility, intellectual honesty, fairmindedness, etc.) the result can then be the inadvertent cultivation of the manipulator, the propagandist, and the con artist. </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">We unknowingly end up, then, undermining the basic values of education and public service, properly conceived.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">It is extremely important to see that intelligence and intellect can be used for ends other than those of gaining &ldquo;truth&rdquo; or &ldquo;insight&rdquo; or &ldquo;knowledge.&rdquo; One can learn to be cunning rather than clever, smooth rather than clear, convincing rather than rationally persuasive, articulate rather than accurate. One can become judgmental rather than gain in judgment. One can confuse confidence with knowledge at the same time that one mistakes arrogance for self-confidence. In each of these cases a counterfeit of a highly desirable trait is developed in place of that trait.</span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; One can learn to be cunning rather than clever, smooth rather than clear, convincing rather than rationally persuasive, articulate rather than accurate.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">There are many people who have learned to be skilled in merely appearing to be rational and knowledgeable when, in fact, they are not. Some of these have learned to be smooth, articulate, confident, cunning, and arrogant. They lack rational judgment, but this does not dissuade them from issuing dogmatic judgments and directives. They impress and learn to control others, quite selfishly. Unless we carefully design schooling to serve the &ldquo;higher&rdquo; ends of education, it can easily, as it now often does, degenerate into merely serving &ldquo;lower&rdquo; ends. When this happens, schooling often does more harm than good. It spreads the influence and resultant harm of pseudo critical thinking.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">With the above understandings in mind, we are prepared to examine the new California Assessment Program and its evaluation of reading and writing.</span><span style=\"font-family: Arial,Helvetica,sans-serif;\">&nbsp;</span></p>\r\n<h4><em><span style=\"color: #666666;\">The California Assessment Program<br /> English Language Arts Assessment</span></em><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> </span><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> </span><span style=\"color: #666666; font-family: Arial,Helvetica,sans-serif;\"><strong><em>Introduction</em></strong></span></h4>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">California has developed the reputation of being a leader in educational reform. It was the first state to mandate critical thinking instruction at all educational levels.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">However, it is now becoming apparent that at the K&ndash;12 level at least, the mandate is not on solid ground, for pseudo critical thinking approaches, and the misunderstandings that underlie them, are becoming rampant in the state. The jargon of reform is everywhere, but substance is virtually nowhere. Unfortunately, the California Department of Education is oblivious of the danger, in fact, is very much part of the problem. Not only is it failing to provide sound leadership in integrating critical thinking into instruction, it is developing an assessment program which is shot-through with pseudo critical thinking confusions. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">It is now deeply involved in developing what it calls &ldquo;authentic&rdquo; assessment that focuses on student &ldquo;performances&rdquo; found in the student &ldquo;construction of meaning&rdquo; in language arts and social studies. Now all of these terms &mdash; &ldquo;authentic&rdquo; and &ldquo;performance&rdquo; and &ldquo;construction of meaning&rdquo; &mdash; are part of the buzz words of the day in educational circles. Of course, the theoretical insights that led to emphasis on these words are important, so let&rsquo;s briefly review them. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Testing and assessment in this country has come under increasing fire, and for good reason. Much of what has traditionally been tested in the popular, machine scorable, multiple choice tests has contributed to little more than trivial pursuit, more and more emphasis in instruction on the lowest order of thinking: rote memorization.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Growing numbers of critics have pointed out that the items on which we have been testing students do not involve reasoning and have little relationship to the kinds of tasks that students will later be called upon to &ldquo;perform.&rdquo; The tests fail, in other words, to &ldquo;authentically&rdquo; test higher order &ldquo;performances.&rdquo; The reform of assessment has increasingly looked to an increased emphasis on &ldquo;authentic&rdquo; items that involve &ldquo;performances&rdquo; of a &ldquo;higher order.&rdquo;</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Furthermore, research by cognitive psychologists and others have clearly established the fact that when humans deeply learn something &mdash; in contrast to, say, storing it temporarily in short-term memory &mdash; that learning involves the &ldquo;construction of meaning.&rdquo; </span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><em><span>&nbsp;&nbsp; The reform of assessment has increasingly looked to an increased emphasis on \"authentic\" items that involve \"performances\" of a \"higher order.\"</span></em></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Here&rsquo;s how you might look at it:&nbsp; </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">In order to get about successfully in the world in which we live we have to continually &ldquo;make sense&rdquo; of things, to give a meaning to what is surrounding us. As we do this we develop networks, systems of meanings that enable us quickly to size up what&rsquo;s in front of us. The result is we don&rsquo;t see &ldquo;meaningless&rdquo; colors and shapes and sounds. We see trees, and people, and dogs, and speeding cars, and smog. We immediately construct &ldquo;meaning&rdquo; out of our experience. Our experience is made by our minds to &ldquo;fit into&rdquo; meanings we have already constructed, or, if we cannot do this, we set about constructing a new meaning out of the old ones.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Now, what puzzled educational researchers was what has come to be called the problem of &ldquo;transfer.&rdquo; Why don&rsquo;t students take what they are studying at school and use it in their daily acts of &ldquo;constructing meanings?&rdquo; Why don&rsquo;t they use scientific concepts when they make everyday predictions or form everyday theories about people and events? Why don&rsquo;t they use concepts from their social studies textbooks when they go about interpreting social situations and trying to figure out solutions to their social problems? Their conclusion was that the students don&rsquo;t use what they study in school in their everyday life because they are not engaged in the construction of meaning in class. In class, they are merely, or at least mainly, memorizing, not constructing meaning, not integrating school learning with everyday life. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Now we are ready to bring the three theoretical concepts together &mdash; &ldquo;authentic,&rdquo; &ldquo;performance,&rdquo; and the &ldquo;construction of meaning.&rdquo; In authentic performances students construct meaning. They do not simply memorize. So why not focus school instruction on just such matters? Why not give them tasks that are &ldquo;authentic?&rdquo; Why not help them, in &ldquo;performing&rdquo; those tasks, to actually &ldquo;construct meaning,&rdquo; in other words, to integrate what they are learning into the network of meanings they are already using to make sense of the world. This is the basic theoretical idea behind the ELAA materials, and, as far as it goes, there can be no objection. But as one wise person once said, &ldquo;The important truths are in the details.&rdquo; And the details of the ELAA materials are horrendous.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">A close examination of the details of the California Language Arts Assessment reveal that it is flawed in all of the following ways:</span></p>\r\n<ul>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">The overall conception is not theoretically coherent. It is filled with vagueness and confusion. This is probably the result of the committee adopting the key buzz words without clearly understanding the theory underlying them. The buzz words are then used vaguely and the details are filled out with terms from the agendas of the various stake-holders.<br /> </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">It does not provide a realistic model of reasoning, of critical reading and writing. Indeed, it is clear that the developers of the assessment do not realize that both reading and writing intrinsically involve the use of reasoning and that reasoning can be done well or poorly.<br /> </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">The overall conception does not call attention to definite and clear intellectual standards. The criteria given are typically vague and applied inconsistently. Important intellectual standards are missing.</span> </li>\r\n</ul>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; The test, in fact, leads the teachers in the direction of malpractice, that is, into the systematic misassessment of reading and writing, leading the students in turn to become inaccurate, imprecise, and undisciplined readers and writers.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<ul>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">There is no way that a teacher might grasp an organized and systematic approach to the role of reasoning in reading and writing by studying the materials being disseminated. The test, in fact, leads the teachers in the direction of malpractice, that is, into the systematic misassessment of reading and writing, leading the students in turn to become inaccurate, imprecise, and undisciplined readers and writers.</span> </li>\r\n</ul>\r\n<h5><span style=\"color: #000099; font-size: small;\">A Tangle of Confusions<br /> </span><br /><span style=\"font-size: small;\"> Let&rsquo;s now look at the details and shed light on some of the theoretical confusions that undermine the approach.</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">First of all, the ELAA commentators open by confusing knowledge with recall and &ldquo;constructing meaning&rdquo; with &ldquo;reasonably constructing meaning.&rdquo; Since these confusions are basic and lead to indiscriminate scoring, let&rsquo;s look at how this occurs.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">California&rsquo;s New English-Language Arts Assessment: An Integrated Look, begins by announcing a paradigm shift. As the English Language Arts Assessment (ELAA) document explains, &ldquo;At the heart of the framework is a paradigm shift in which &lsquo;constructing meaning&rsquo; replaced &lsquo;gaining knowledge&rsquo; as the primary goal.&rdquo; Or, as it says later, &ldquo;Since the construction of meaning is the essence of both reading and writing, the new assessment allows students to shape the outcome rather than to identify correct meanings that test makers have posited.&rdquo; </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">It is clear that the writers of the assessment are either not clear about the difference between recall and knowledge, or they are wrongly assuming that the attainment of knowledge is not intrinsically connected to the construction of meaning, or both. Briefly, let&rsquo;s make these relationships clear.</span><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> </span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">Rational and Irrational Constructions</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">&ldquo;Constructing meaning&rdquo; is a process that is common to all learning which becomes deep-seated in the mind of the learner. It applies, however, just as much to the formation of flawed, irrational meanings as it does to the formulation of defensible, rational meanings. Deep-seated irrational fears, for example, result as much from the personal construction of meaning as do insights and understandings.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Knowledge, on the other hand, though also the result of the construction of meaning, requires a clear-cut exercise of the rational faculties of the mind. For example, to appropriately judge a person accused of murder to be guilty or innocent, one&rsquo;s thinking must be guided by a careful and rational use of evidence, legal criteria (the criteria for &ldquo;murder&rdquo;), and the canons of sound reasoning. When a jury appropriately attains the knowledge of guilt or innocence, that knowledge, expressed in their verdict, is a product of a rational, a reasoned, construction of meaning. Of course, a jury may not function as it ought. It may be irrational and prejudiced, and the judge may overturn its verdict precisely because it did not properly discharge its responsibility to be &ldquo;rational.&rdquo; There is nothing wrong, therefore, with focusing attention on the need of students to &ldquo;construct meaning&rdquo; but it must be underscored that the mere construction of meaning, as such, is not a significant achievement, since it is done as much by Archie Bunker as by Einstein.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">But the authors of The California Student Assessment System are confused on this point, for they talk as if the construction of meaning is an <br /> end in itself. They forget that &ldquo;prejudice,&rdquo; &ldquo;stereotypes,&rdquo; &ldquo;misconceptions,&rdquo; &ldquo;illusions,&rdquo; &ldquo;delusion,&rdquo; &ldquo;self-deceptions,&rdquo; &ldquo;false beliefs,&rdquo; and all manner of other intellectually flawed creations of the mind, are just as much &ldquo;constructions&rdquo; and as &ldquo;meaningful&rdquo; constructions as ones more insightful and discerning.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">We should rather be interested in fostering in children adherence to those intellectual standards that maximize their construction of genuine &ldquo;knowledge,&rdquo; for otherwise they <span style=\"font-family: Arial,Helvetica,sans-serif;\">are likely to engage in a great deal of &ldquo;irrational&rdquo; construction and they will not know they are doing so.</span></span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>. . . it must be underscored that the mere construction of meaning, as such, is not a significant achievement, since it is done as much by Archie Bunker as by Einstein.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Education must discriminate between the quality of students&rsquo; constructions of meaning, both in their reading and in their writing. But this can only defensibly be done by judging them by means of those intellectual standards common to educated thought. A construct that is unclear is not to be confused with one that is clear. One that is inaccurate is not to be confused with one that is accurate. One that is relevant to an assigned task is not to be confused with one that is irrelevant. One that is superficial is not to be confused with one that is deep.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Hence we do not need to decide between emphasizing the construction of meaning and the goal of attaining knowledge. If we properly understand the &ldquo;dual&rdquo; character of &ldquo;meaning construction,&rdquo; we will immediately recognize the need to focus on the &ldquo;reasoned&rdquo; and &ldquo;reasonable&rdquo; construction of meaning, and not indiscriminately credit any construction of meaning.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">To underscore the point, the human mind naturally and inevitably constructs meaning. The mere fact that students construct meanings tells us nothing about the quality of those constructs. For example, in the extended example on reading as a form of thinking which we cited (pp. 24&ndash;27), both readers, Stephen and Colleen, constructed meanings. But you will remember that the meanings constructed by Colleen were absurd. Both students reasoned about the text&rsquo;s meaning, but there was a stark contrast in the quality of reasoning in the two cases.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The point should now be clear. We want to work with our students&rsquo; capacity to construct meaning from a text, but we want to do this while teaching them to discipline their reading, to learn how to fit their interpretations to the logic of the words of the text. We want them to develop definite intellectual standards for their reading and not feel free to treat a text as if it were &ldquo;silly putty,&rdquo; to be shaped into any &ldquo;meaning&rdquo; they choose.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">In passing, in the workplace, there is no economic value in constructing irrational, fanciful, lively, and entertaining meaning if it is irrational and reflects a flawed understanding of the text. Employers are not looking for a flashy, individualized response to a piece of writing, but rather a solid grasp of the meaning intended by the author. This is a fundamental premise of written communication!&nbsp; </span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; Employers are not looking for a flashy, individualized response to a piece of writing, but rather a solid grasp of the meaning intended by the author. This is a fundamental premise of written communication!</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">We should therefore continually underscore the pivotal role of intellectual standards, not only in assessment but in any form of intellectual work whatsoever, including of course, reading and writing.</span></p>\r\n<table border=\"1\" cellspacing=\"1\" cellpadding=\"1\" width=\"50%\">\r\n<tbody>\r\n<tr>\r\n<td>\r\n<p><span style=\"color: #000099; font-family: Arial,Helvetica,sans-serif;\"><strong>Intellectual Standards <br /> That Apply to Thinking in Every Subject</strong></span></p>\r\n</td>\r\n</tr>\r\n<tr>\r\n<td>\r\n<p><strong><em>Thinking that is: &lt;&gt;&nbsp;Thinking that is:</em></strong></p>\r\n</td>\r\n</tr>\r\n<tr>\r\n<td>\r\n<p>Clear vs Unclear<br /> Precise vs Imprecise<br /> Specific vs Vague<br /> Accurate vs Inaccurate<br /> Relevant vs Irrelevant<br /> Plausible vs Implausible<br /> Consistent vs Inconsistent<br /> Logical vs Illogical<br /> Deep vs Superficial<br /> Broad vs Narrow<br /> Complete vs Incomplete<br /> Significant vs Trivial<br /> Adequate (for purpose) vs Inadequate<br /> Fair vs Biased or One-Sided</p>\r\n</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<h5><span style=\"color: #000099; font-size: small;\">A Pseudo Commitment to Intellectual Standards</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Intellectual standards are essential to the appropriate assessment of reading and writing. At some level, the assessment authors are aware of this necessity. Their description of their own criteria imply both impartiality and commitment to intellectual standards. For example, the authors of the English-Language Arts Assessment often speak of their commitment to &ldquo;encourage students to read widely and in depth.&rdquo; (Depth implies criteria for distinguishing &ldquo;deep&rdquo; from &ldquo;shallow&rdquo; readings.) Secondly, they imply impartial assessment when they state the three-fold purpose of the new English-Language Arts Assessment (p. I-2):</span></p>\r\n<table border=\"0\" cellspacing=\"10\" cellpadding=\"0\" width=\"100%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>1) </strong>To establish standards for evaluating students&rsquo; performance when&nbsp;they read diverse kinds of materials for different purposes.<br /> <br /> <strong>2)</strong> To measure how well students are able to construct meaning.<br /> <br /> <strong>3)</strong> To improve the instructional program by providing an assessment that reflects the Framework.</span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\"><span style=\"font-family: Arial,Helvetica,sans-serif;\">They use language that implies a concern with rational judgment: for example, the ELAA report says when speaking of &ldquo;meaning-making,&rdquo; that,</span></p>\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\"><span style=\"font-family: Arial,Helvetica,sans-serif;\">We want students to think critically as they explore interests, clarify values, solve problems, resolve conflicts, generate new ideas, synthesize/apply learnings, set goals, and make decisions in response to the literature they read. (p. I - 2) <br /> Now, the processes of thinking critically, clarifying values, solving problems, setting goals, and making decisions all presuppose the importance of rationality, of engaging in sound reasoning. To clarify values, for example, requires that we rationally analyze them.</span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; It becomes clear, from here on, that the California testing experts&rsquo; glossy, global statements about critical thinking and meaning-making and standards are losing their luster.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n</blockquote>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Solving problems is not the product of an arbitrary construction of meaning, but requires, amongst other things, an objective and accurate analysis of the nature of the problems, of the information relevant to the problems, and such like. Effective goal setting requires that we accurately identify possible competing goals and reasonably assess which make most sense. And certainly, decisions can reasonably or unreasonably be arrived at. It becomes clear, from here on, that the California testing experts&rsquo; glossy, global statements about critical thinking and meaning-making and standards are losing their luster. Empty platitudes and vacuous ideals are a specialty of virtually all bureaucracies. So common are they now that they are hardly noticed anymore. </span></p>\r\n<p><br style=\"clear: both;\" /></p>","public_access":"1","public_downloads":"1","sku":"","files":[],"images":[]}

The Problem of Scoring

As you might expect, all of the confusions above come home to roost in the design for scoring student “performances” in reading and writing. For example, in explaining the design with respect to assessing elementary reading, the authors introduce us to a 15-point list under the head of the reading performances of effective readers.

Effective readers connect with, reflect on, and challenge the text. Readers do not need to show evidence of all the performances listed here. The discerning and insightful reader may display a broad spectrum of reading behaviors or may investigate a few selected behaviors in great depth. The exemplary reader may show variety, complexity, breadth, and/or depth. Through their writing and graphics, these readers show convincing evidence of their ability to construct meaning. They may:

Make connections between the text(s) and their own ideas, experiences, and knowledge.

Demonstrate emotional engagement with the text(s)

Retell, summarize, and/or paraphrase with purpose.

Reflect on the meaning(s) of the text(s), including larger or more universal significance; express a major understanding about or insight into a subject, an aspect of self, or of life in general.

The Escape Hatch

First off, it is clear that since each scorer can pick and chose from such a wide variety of criteria (a number of which as we shall see are extremely vague), the impartiality of application of the criteria is suspect. Also note that any and all of these activities can be done either defensibly or indefensibly. Hence, if these are to be assessed in order to be credited or discredited, criteria must be provided for each of the individual “performances” cited (over 50 are buried in the list). We need some explanation of how they expect someone assessing student reading to apply them. Consider each of the following. The assessor is left to her own intuitions in determining whether or not a student’s reading:

is insightful; discerning; perceptive;

is sensitive to linguistic, structural, cultural and psychological nuances and complexities;

Do the assessors really know how to impartially assess whether or not a student reader is being “sensitive to a psychological nuance” or to a “structural nuance” or a “cultural nuance?” Or can they impartially determine whether or not the student reader is entertaining a “challenging” idea? Or whether a student perception is “acute” or not? Isn’t it highly probable that different assessors are going to have somewhat different conceptions of each of these matters, for example, one thinking a given idea is “quite challenging” and another thinking it is not? Surely this much is clear! No criteria, however, are provided, and this invalidates any attempt to use the results to assess one student’s performances year after year, as well as to assess all California students collectively, year after year. Instead, having given us an array of vague descriptors, the authors now largely set them aside and focus instead on a six-point “Scoring Guide” that is to be used in distinguishing student reading into the following categories:

Exemplary Reading Performance (Six Points)

Discerning Reading Performance (Five Points)

Thoughtful Reading Performance (Four Points)

Literal Reading Performance (Three Points)

Limited Reading Performance (Two Points)

Minimal Reading Performance (One Point)

These general descriptors are of very little use. For example, consider the words “discerning” and ”thoughtful.” It is not obvious that one is better off being “discerning” than being “thoughtful.” It is also not obvious why “literal” is above “limited.” It is certainly not clear why the lowest score is “minimal” reading. What ever happened to just plain “poor” reading? Has it disappeared or is it one of many forms of “minimal” reading. (See Colleen’s reading on pp. 24–27.) Is it “minimal” or just plain “poor?”)

But that is not all. Each of the terms listed in each of the six point scoring guide create further problems for the conscientious scorer. Consider the terms in the first category alone, that of “exemplary reading performance.” A person who takes seriously the characterizations of this first category should be prepared to notice and assess whether or not the student is:

drawing on evidence objecting to text features considering the authority of the author considering the quality of the author’s sources suggesting ways of rewriting the text embracing the ideological position of a text resisting the ideological position of a text revising their understanding as they read carrying on an internal dialogue

The State Department Criteria for an Exemplary Reading Performance1) An exemplary reading performance is insightful, discerning and perceptive as the reader constructs and reflects on meaning in a text. Readers at this level are sensitive to linguistic, structural, cultural, and psychological nuances and complexities. They fill in gaps in a text, making plausible assumptions about unstated causes or motivations, or drawing meaning from subtle cues. They differentiate between literal and figurative meanings. They recognize real or seeming contradictions, exploring possibilities for their resolution or tolerating ambiguities. They demonstrate their understanding of the whole work as well as an awareness of how the parts work together to create the whole.

Readers achieving score point six develop connections with and among texts. They connect their understanding of the text not only to their own ideas, experience, and knowledge, but to their history as participants in a culture or larger community, often making connections to other texts or other works of art. Exceptional readers draw on evidence from the text to generate, validate, expand, and reflect on their own ideas.

These readers take risks. They entertain challenging ideas and explore multiple possibilities of meaning as they read, grounding these meanings in their acute perceptions of textual and cultural complexities. They often revise their understanding of a text as they re-read and as additional information or insight becomes available to them. They sometimes articulate a newly developed level of understanding.

Readers demonstrating a score point six performance challenge the text. They carry on an internal dialogue with the writer, raising questions, taking exception, agreeing, disagreeing, appreciating or objecting to text features. They may test the validity of the author's ideas, information, and/or logic by considering the authority of the author and the nature and quality of the author's source(s). They frequently suggest ways of rewriting the text, speculating about the ideology or cultural or historical biases that seem to inform a text, sometimes recognizing and embracing and sometimes resisting the ideological position that a text seems to construct for its reader.

Must the Student Just Do It or Do It Well?

Even more problematic than the likely disagreement among assessors as to the application of the vague standards provided is, as I suggested above, the problem of the assessors being given many criteria that name processes that can, in principle, be done well or poorly. The directions do not explain whether, to be credited, the student is obliged to use the cited processes well or simply use them in any way whatsoever. That is, there is no indication as to whether the assessor is to evaluate the “quality” of the way the student is

Unfortunately, once we examine the actual student writing examples along with the commentary provided, it becomes painfully clear that the assessors were simply looking to see if the student in any sense used the process and did not have, or did not use, criteria to assess how well the students used the processes credited.

doing these things or simply certify the fact of doing these things — however poorly. Remember, a student who is drawing an absurd meaning is still drawing a meaning. A student who is making a trivial connection is still making a connection. A student who is drawing on irrelevant evidence is still drawing on evidence. A student who is raising a silly or superficial question is still raising a question. And so forth, and so on.

Unfortunately, once we examine the actual student writing examples along with the commentary provided, it becomes painfully clear that the assessors were simply looking to see if the student in any sense used the process and did not have, or did not use, criteria to assess how well the students used the processes credited. The California State Department of Education falls directly into the trap of failing to discriminate between these crucial differences. The assessors are left to their own devices. They can draw these distinctions or fail to draw them. It is clear that most failed to draw them. This is a fatal flaw in the assessment. It renders the results of the assessment virtually useless.

The Misassessment of Elementary Reading

The only elementary reading passage which is given with scored examples is from a story by John Gardiner called Stone Fox. It is an emotionally explosive story, one chosen perhaps to ensure an emotional response. The children are simply asked to give some of their “thoughts, feelings, and questions” about what they are reading. No other kind of writing is given as an example.

Insufficient Directions Are Given

Now the first remarkable feature of the Stone Fox reading prompt is that the student readers are given no indication whatsoever of the purpose for which they are reading the story. This is ironic in the light of the fact that the CAP materials emphasize the fact that one can read and write “for different purposes” (p. I-3).

And yet, here, the students are asked to read for no particular purpose. Are they to read in a casual fashion, simply for amusement? Or are they to do a “close reading” for detail? Are they to be analytical and reflective, or not? The students are given no indication of how they will be assessed. Hence, they are asked to engage, at best, in an ill-defined “performance.” Nothing is given in the way of directions to the students except “Read to see what happens” and a place to the right in which the student may write notes under a column head titled, “My thoughts, feelings, and questions about what I am reading.” The students were apparently not even told that they should try to write out as much of their thoughts, feelings, and questions as they could. Some readers might presumably have thoughts, feelings, and questions they would not bother to express.

Consider that you are a student reading the story. You see a column at the right which says, “My thoughts, feelings, and questions about what I am reading.” How should you understand it? Wouldn’t you wonder which thoughts ... which feelings ... based on what? What am I to think about? Why am I to think about it?

In any case, since the student reader is not told that she is going to be evaluated on what she writes and is told none of the criteria, why should she be motivated to fully express her thoughts and feelings? Consider, on the other hand, what the directions might have said.

Possible Directions

When we read a story we have to try to understand and follow its meaning. We try to figure out what it is saying and we try to connect it with our own life in some way. We want you to read this story and see what it means to you. Why do you think it was written? Do you think that it was written well? Do you think it is true to life? Does it illustrate anything that you believe is important? Please write in as many of your thoughts on these questions as possible as you read. Help us to understand what is going on in your mind as you are reading and trying to relate this story to your life.

Can One Evaluate Purely Subjective Responses to Stories? When the student reads that she is directed to express her “feelings,” with no explanation given why she is so directed, then how can we legitimately go on to judge those feelings and “score” them one through six? Suppose the student quite sincerely said, “In my view this is a sentimental story that insults my intelligence. I feel disgusted when I read it and bored silly.” How should the assessor evaluate that “feeling” response? Is there any way to discredit it according to the directions? Certainly not. It is as good a response to a request for “feelings” as any other. To put the point succinctly, either we help students understand the difference between a request for a purely subjective response, (How do you feel when you read this?) and a request for a more reasoned response (What feeling do you judge the author wants you to feel and why? In your judgment is the author successful? Tell us the reasons why you think so.) or we must not indulge in any assessment of the students “feelings.” We have no legitimate grounds for doing so. The student can legitimately take the request to be one that asks for a subjective response and a purely subjective response cannot be impartially assessed or scored.

The Misassessment of Writing at the Elementary Level

Different kinds of writing will be assessed at different levels: Elementary: persuasive writing ... Middle School: problem solution, evaluation and speculation about cause and effect ... High School: evaluation, speculation about cause and effect, interpretation and controversial issues plus a reflective essay. (p. I–4)

The grade four writing assessment is designed to reflect a variety of purposes for which children write:

Expressive writing (“This is what I see, think, and feel . . .”)

Persuasive writing (“This is what I believe and why I think you should believe it . . .")

Narrative writing (“This is what happened . . .")

Informational writing (“This is what I know and how I know it . . .")

Criteria For Persuasive Writing

Persuasive writing is explained in the following terms (p.lll — 12)

Persuasive writing requires students to choose positions, to make judgments, to offer proposals, and to argue convincingly for their beliefs and ideas. However, some students may choose to explore both sides of an issue and then offer a compromise, . . . Effective writers use evidence such as examples or anecdotes to support their arguments. Convincing arguments may appeal to logic, emotions, and/or philosophical beliefs.

Persuasive writers establish themselves as informed, knowledgeable individuals. They orient readers . . . More than any other kind of writing, argument requires writers to consider their audience . . . The best persuasive writers systematically develop arguments with a strong sense of coherence and movement throughout the piece.

California’s Standards for Exceptional Writing (Six Points)

An exceptional score (6 points) must meet the following standards:

Focus/Coherence

Position. Writers of six-point papers usually assert and maintain a clear position throughout the piece; they present evidence and explanations in a purposeful way.

Occasionally these writers will effectively evaluate both sides of an issue and offer a reasonable compromise; or they may conclude that neither position is preferable, or they may suggest a third position.

Coherence. Writers provide overall links or transitions; they present arguments, evidence, and reasons logically so that the overall effect is one of coherence.

ElaborationDepth/Density of Arguments. Writers thoroughly develop and elaborate their reasons, examples, information and/or anecdotes. Some writers may develop only one reason or example, but they do so in depth; others may choose to develop several appropriate reasons, examples, and so forth.

Relevance of Arguments. Writers choose and present appropriate reasons, examples, information, and so forth to support their argument(s). They show their arguments are valid based on prior knowledge, personal experience and reflection.

Audience Awareness. Writers choose and present arguments with a clear awareness of reader needs. They often show credibility and a sense of authority by revealing source(s) of information. They may anticipate possible reader response by including some counter-arguments.

Style

Word Choice. These writers use lively, interesting concrete language that carries precise meanings and emotions. Word choice is appropriate to the writer's purpose.

Once again there are a host of problems with the criteria, the most serious being that, just as in the criteria for reading, there is no indication of whether the students are expected to do any of the above well, or simply do them in any form whatsoever. For example, consider the claim that “convincing arguments may appeal to logic, emotions, and/or philosophical beliefs.” Suppose a student uses “convincing” but “fallacious” logic — is the student to be credited? Or suppose the student appeals to the emotions of the reader by engaging in name-calling (“This stupid communist idea...!”) — is the student to be credited? The formulators of the criteria are seemingly oblivious of the problem.

A Grade Four Essay Judged to Illustrate High-Range Achievement

The CAP Commentary

From the opening . . . this writer exudes confidence, focuses on an audience and takes a firm stand on the issue. These attributes of persuasive writing are maintained throughout the piece.

Support for the writer’s credible arguments in favor of a park versus a mall are drawn from personal experiences. She uses examples: “We already have over five stores in our plaza,” and notes that the park is a “perfect place to play, take your dog for a walk.” She includes reflection, “Do you honestly think that children at the age of two or three are going to enjoy a mall ...?” Her arguments are appropriate and appeal to reason (safety) and emotions (“I didn’t come here to watch those C.A.T.s tare up that park!”)

This writer is consistently aware of and appeals to her audience with appropriate tone and lively language.

The Problem of Subjectivity, Once AgainThe problem of subjectivity that was so apparent in the elementary reading assessment reappears again in the writing assessment. It is clear in this, and in other examples, that the commentators are using the criteria for good persuasive writing literally and not making any real judgments of reasonability. Hence, if the student says anything which can be construed as falling under one of the criteria then that is credited. For example, if she uses an emotional appeal then credit the emotion criterion (however irrelevant or inappropriate it might be); if she says something that can be described as “reflection,” then credit the reflection criterion (however irrelevant or inappropriate the “reflection” might be). Any kind of a move that would work with an audience, in the view of the commentator, is credited as good (regardless of how irrational it might be).

To suggest that this is good persuasive writing is to teach children exactly the wrong lesson. It fails to show them the vital distinction between reason and its counterfeit.

But the significant question for anyone concerned with the traditional values of education is whether a reasonable audience should be persuaded or “moved.” The important distinction for students to grasp is that between what might be called “low level rhetorical appeals” and those appeals which would convince or “move” a reasonable audience. No such distinction is recognized by ELAA. The criteria are used crudely, without any important intellectual distinctions in evidence.

The implication of this is that if a student wrote a very rational appeal, which recognized the weaknesses in her position and the need to qualify her claims, she would be downgraded because she would not be “exuding confidence,” “taking a firm stand on the issue,” etc. Indeed, though her presentation might appeal to a “jury of reasonable persons,” the graders of the ELAA would not be impressed.

In this particular essay, for example, though it is good for children to display confidence, etc., the issue of park versus mall should be decided on rational grounds. Yes, it is important that the park is a favorite place for children and that they are safe there, but this is really a decision about options, and alternatives, and relative costs and benefits. This issue should be decided by looking at these factors and weighing them as rationally as possible, not on the basis of emotional appeals, like that of the child. To suggest that this is good persuasive writing is to teach children exactly the wrong lesson. It fails to show them the vital distinction between reason and its counterfeit.

Suppose the child has to present a case before an audience she knows to be racist; if one reads and applies the criteria presented for good persuasive writing in ELAA, the student who plays on those racist sentiments will score higher than one who opposes them.

What apparently matters in the mind of the ELAA assessors is that one successfully persuades, not that one argues reasonably. But it is harder to imagine a distinction which it is more vital for the educated person to grasp.

What apparently matters in the mind of the ELAA assessors is that one successfully persuades, not that one argues reasonably. But it is harder to imagine a distinction which it is more vital for the educated person to grasp. Indeed, one might almost regard it as a criterion of being educated that one sees the difference between fairminded, reasonable argument, and self-seeking, low-level, persuasive rhetoric. The demagogues may often win the day, but do we want to use public monies to generate armies of demagogues, all having mastered the art of demagogery at the public expense?

In another example, which contains some very good reasoning about why another child should feel good about himself, the commentators say, “She arranges her reasons and evidence in a sophisticated pattern.” There is no mention of whether the reasons and evidence are relevant or irrelevant, true or false, good or bad.

It is not that the reasons are arranged in a sophisticated pattern, but that they are good reasons!

Yet, again, isn't this what really matters? It is not that the reasons are arranged in a sophisticated pattern, but that they are good reasons!

In another, mid-range, example, about how to solve problems between children and parents, the commentators say (among other things), “Audience awareness is evident throughout, although appeals lack the vigor and exactness of higher score point papers. The use of bullets to summarize the writer’s arguments is an effective tool and adds to her sense of conviction.” Why is the writer criticized for a lack of vigor and praised for conveying a sense of conviction? Neither is a virtue in itself. What matters is that one exhibits the appropriate degree of vigor and conviction, depending on the strength of one’s case. Vigorously arguing a weak case and displaying conviction despite poor supporting reasons ought to be marked down, not praised.

The Young Hitler Scores High on the CAP Test

We can now make our point dramatically by considering how the following piece of persuasive, but highly irrational, writing should be graded according to the ELAA criteria.

. . the greatest revolution Germany has undergone was that of the purification of the Volk [people] and thus of the races, which was launched systematically in this country for the first time ever. [From the opening the writer exudes confidence, focuses on an audience and takes a firm stand on the issue.]

The consequences of this German racial policy will be more significant for the future of our Volk than the effects of all the other laws put together. For they are what is creating the new man. They will preserve our Volk from doing as so many historically tragic past prototypes of other races have done: lose their earthly existence forever because of their ignorance as regards a single question. [The writer arranges reasons, examples and information in a discernible and effective pattern resulting in overall persuasive effect.]

For what is the sense of all our work and all our efforts if they do not serve the purpose of preserving the German being? And what good is any effort on behalf of this being if we omit the most important thing to preserve it pure and unadulterated in its blood? [The writer asserts and maintains a clear position throughout. He also chooses and presents arguments with a clear awareness of reader needs.]

Any other mistake can be rectified, any other error can be corrected, but what one fails to do in this area can often never be amended. Whether our work in this area of purifying our race and thus our Volk has been fruitful is something you can best judge for yourselves here during these few days. For what you are encountering in this city is the German being. Come and see for yourselves whether he has become worse under National Socialist leadership or whether he has not indeed become better. Do not gauge only the increasing number of children being born — gauge above all the appearance of our youth. [The writer presents evidence and explanations in a purposeful way.]

How lovely are our girls and our boys, how bright is their gaze, how healthy and fresh their posture, how splendid are the bodies of the hundreds of thousands and millions who have been trained and cared for by our organizations! [The writer shows his arguments are valid based on personal knowledge and reflection.] Where are better men to be found today than those who can be seen here? It is truly the rebirth of a nation, brought about by the deliberate breeding of a new being [The overall effect is one of coherence.]

— (Hitler Speech 1937)

This piece asserts and maintains a clear position throughout and presents evidence and explanations in a purposeful way; the writer arranges reasons, examples, information and/or personal anecdotes in a discernible and effective pattern resulting in overall persuasive effect; the overall effect is one of coherence; the writer shows his arguments are valid based on prior knowledge, personal experience and reflection; the writer chooses and presents arguments with a clear awareness of reader needs; the writer evidences confidence, conviction, belief, and sometimes enthusiasm.

Clearly, for all these reasons we have to give this piece of Hitler’s writing a Point 6 score! It meets the CLAS criteria, as does much of his writing. Hitler’s writing was widely recognized to meet the needs of his audience, to exude confidence, etc. Is that really what we want to praise? Is that the model of persuasive writing that we want to hold up to our children? If so, shame on the California Learning Assessment System!

The Misassessment of Writing at the High School Level

Introduction — Same Problem: Next Level

At the high school level the Writing Assessment assesses four types of writing, Autobiographical Incident, Interpretation, Reflective Essay, and Speculation About Causes and Effects, some of which are successors to the elementary level Persuasive writing.

The same general faults that were mentioned earlier in connection with the elementary level are to be found at this level too. For example, the scoring guide for interpretation makes it clear that only subjective reactions to and subjective interpretation of fiction are really being considered. But interpretation is important in many other contexts, e.g. history, and in history it is crucial to distinguish between subjective response and objective interpretation, between reasons that persuade irrational audiences and reasons which persuade rational and fairminded persons, between something which is rhetorically convincing and something which is true. None of these distinctions is recognized in the CLAS.

To be specific in our criticisms, consider the scoring guide for Speculation About Causes and Effects; once again writing of this kind is judged almost entirely by subjective standards, by standards appropriate to one’s response to fiction.

Here is what is said about writing of this kind which should score Point 6, for Exceptional Achievement:

A six-point essay engages the reader immediately. It seems purposeful. The writer seems aware of reader’s questions and needs throughout the essay. The essay seems to be not just written but written to particular readers. The writer convinces the readers of the plausibility of the speculation.

A six-point essay demonstrates qualities all readers admire: conviction, enthusiasm, freshness. These essays may use an unconventional rhetorical approach. A six-point essay may take chances and succeed.

Presenting the Situation. The six-point essay writer clearly defines, identifies, or describes the situation to be speculated about. Though it does not dominate the essay at the expense of speculation, the situation is nevertheless presented fully and precisely. The writer limits the occasion appropriately, focusing reader attention on just those aspects of the situation that the writer will speculate about.

Writers of six-point essays may describe or detail the situation that is established in the prompt, or they might create the situation by using narrative or anecdotal techniques. In either case, they will use concrete language, rich in sensory detail.

The writer of the six-point essay acknowledges readers’ concerns. For real world situations, the writer of the six-point essay acknowledges the reader’s experience or familiarity with a situation and, using narrative or descriptive strategies, builds on this awareness to focus reader attention on a comparable situation. ...

Whether the essay arises from a factual assessment of a real situation or from a fanciful guess about a fanciful situation, the writer consistently demonstrates broad knowledge and clear understanding of the situation. In this way the writer establishes authority.

Logic and Relevance of Causes and Effects. In the six-point essay, the proposed causes and effects are clearly related to the particular situation that the writer has defined. Writers use imaginative, inventive argument to convince the reader of the logic of their speculation. The best writers are clearly considering possibilities and are seeing multiple perspectives.... Because speculation is essentially a persuasive type of writing, the best writers will be continually aware of readers’ needs. They might refer to the readers directly, trying to enlist their support ...

Elaboration of Argument. The six-point essay provides substantial elaboration, convincing the reader that the writer’s conjectures are valid for the situation. These writers elaborate their speculated causes and effects with carefully chosen evidence that is logically and fully developed. Such evidence is chosen because it is relevant and convincing. It is developed fully with precise, explicit detail to convince the reader both of the logic and the authenticity of the proposed cause and effect.

Some strategies writers may use to develop their arguments are the following:

Cite facts, opinions, projections, and personal experiences or observations (anecdotes) to explain or validate a cause or an effect.

Elaborate on possibilities arising from proposed causes and effects, showing possible “domino effects” that might determine the direction of the developing situation.

Give specific examples of comparable causes and effects that have arisen in analogous situations. (p. III–64)

How may we best make the point that these criteria again fail to recognize the crucial distinctions of which we have been speaking — between subjective responses and good reasons, etc? Perhaps the simplest way is to look at another example of persuasive writing, this time Speculating About Causes and Effects, and consider how we should grade it according to the CLAS criteria.

Just as every people, as a basic tendency of all its earthly actions possesses a mania for self-preservation as its driving force, likewise is it exactly so with Jewry too. Only here, in accord with their basically different dispositions, the struggle for existence of Aryan peoples and Jewry is also different in its forms. The foundation of the Aryan struggle for existence is the soil, which he cultivates and which provides the general basis for an economy satisfying primarily its own needs within its own orbit through the productive forces of its own people.

Because of the lack of productive capacities of its own the Jewish people cannot carry out the construction of a state, viewed in a territorial sense, but as a support of its own existence it needs the work and creative activities of other nations. Thus the existence of the Jew himself becomes a parasitical one within the lives of other people. Hence the ultimate goal of the Jewish struggle for existence is the enslavement of productively active peoples. In order to achieve this goal, which in reality has represented Jewry’s struggle for existence at all times, the Jew makes use of all weapons that are in keeping with the whole complex of his character.

Therefore in domestic politics within the individual nations he fights first for equal rights and later for super-rights. The characteristics of cunning, intelligence, astuteness, knavery, dissimulation, etc, rooted in the character of his folkdom, serve him as weapons thereto.

They are as much stratagems in his war of survival as those of other peoples in combat.

In foreign policy he tries to bring nations into a state of unrest, to divert them from their true interests, and to plunge them into reciprocal wars and in this way gradually rise to mastery over them with the help of the power of money and propaganda.

His ultimate goal is the denationalization, the promiscuous bastardization of other peoples, the lowering of the racial level of the highest peoples as well as the domination of this racial mish-mash through the extirpation of the folkish intelligentsia and its replacement by the members of his own people.

The end of the Jewish world struggle therefore will always be a bloody Bolshevization. In truth this means the destruction of all the intellectual upper classes linked to their peoples so that he can rise to become the master of a mankind become leaderless. — (Hitler’s Secret Book, pp. 212-213)

Hitler’s Assessment Based on CAP Criteria

This piece of writing certainly engages the reader immediately;

it seems purposeful;

the writer seems aware of reader’s questions and needs throughout;

the piece seems not just to be written but written to particular readers;

the writer convinced those readers of the plausibility of the speculation;

the writing shows conviction and enthusiasm;

the writer clearly defines the situation to be speculated about, acknowledges the reader’s experience or familiarity with the situation, and consistently demonstrates broad knowledge and clear understanding of the situation;

the writer uses imaginative, inventive argument to convince the reader of the logic of his speculation;

the writer elaborates on possibilities arising from the proposed causes and effects, showing possible “domino effects” that might determine the direction of the developing situation; etc., etc.

Can there be any doubt that Hitler’s writing in the category merits a Point 6 grade?! If that is so then once again, shame on the California Learning Assessment System, which has again failed to see the difference between a proper and improper use of rhetoric and reason.

Summary Judgment on the California Assessment of Reading & Writing

An assessment of reading and writing should not only underscore the role of reasoning in both, but also firmly establish defensible intellectual standards, appropriately and specifically explained and consistently and appropriately applied.

No assessment of intellectual work, nor foundation for teaching, should be based on an approach in which intellectual standards are confused and erroneous, confusing recall with knowledge, subjective preference with reasoned judgment, irrational with rational persuasion.

No assessment of intellectual work, nor foundation for teaching, should be based on an approach in which intellectual standards are confused and erroneous, confusing recall with knowledge, subjective preference with reasoned judgment, irrational with rational persuasion. No assessment of intellectual work should use its key terms vaguely or oscillate between two significant uses of a term or score in an arbitrary manner. And, most important of all, no intellectual assessment should encourage irrational subjectivism.

For example, it is striking that in the context of reading and listening, there is little discussion of the need to create an accurate interpretation: there are many contexts in which it is not appropriate for the readers to “create their own meaning” and where accuracy is what is required. This may be equally true if the author is addressing a particular question or problem, or using particular basic concepts; very often the good response to what is said or written is good precisely because it is based on an accurate construal of the text. Nothing in this assessment mentions the virtue of accuracy.

Most importantly however, this approach is flawed again and again because what gets credited is anything that could be construed as fulfilling one of the criteria and the criteria are the wrong ones for the purpose. If the writer is using any emotion (however discreditable) then it is marked positively; if she is giving anything that could be called evidence — even if it is bad evidence — it is credited; any kind of a move that would work with an audience, in the view of the reader is credited as good (regardless of how irrational it might be).

Underneath all of this is a question of values. We are obliged to educate our students, not simply to shape them.

The end result is that if a student wrote a very rational appeal, one that would persuade a rational audience, it is going to be graded down because it wouldn’t appeal to an irrational judge (because it won’t be maintaining a strong line, it won’t necessarily be persuasive; it will be putting in qualifications; it will be speaking in terms of greys and greys don’t persuade; it will not be engaging in hyperbole and hyperbole is effective; it won’t be trying to negate everything about the other side, it will be recognizing reasonable objections; it will express the degree of confidence that is appropriate but no more). Given a list of what a rational person would do, you will be able to see that the criteria for success under CLAS are negating these rational qualities and therefore encouraging irrational beliefs about how you communicate to people, indeed encouraging people to become manipulators!

Underneath all of this is a question of values. We are obliged to educate our students, not simply to shape them.

The educated person is reasonable; the educated person isn’t simply concerned with winning. The educated person wants to win when winning is the appropriate thing. However, when the other side is more reasonable, then the educated person, who is more interested in getting at the truth than in winning, will want to make concessions.

If the goal of education is simply to enable people to get what they want, then we should teach them tools of manipulation, ways to win battles, ways to undermine positions whether those positions are rational or not, ways to just get what they want, irrespective of fairness, and of the evidence. Inadvertently and unknowingly, this is what CLAS is supporting. CLAS does not have in mind a clear difference between the educated, fairminded, and rational person, on the one hand, and the person who is simply good at manipulating, winning, and defeating others, on the other.

In effect what CLAS has said is that the name of the game is to persuade the audience by whatever methods work and we’ll credit anything that works. Shouldn’t they instead have said, “Since we are obliged and committed to educating children, and since this requires they learn to reason well as readers, writers, and thinkers, we will not credit flawed reasoning. We will only credit well-reasoned responses.”

To Summarize Some of Our Criticisms of the CLAS Approach

1) The overall conception fails to capture the practices of critical readers, writers, and thinkers the world over.

2) The overall conception does not call attention to definite and clear intellectual standards, and without them, it becomes impossible for both teacher and student to engage in “objective” assessment.

3) The overall conception does not provide an organized and systematic approach to posing, analyzing, and reasoning through problems embedded in everyday personal and professional reading and writing.

4) The teacher who takes this approach seriously will misteach reading and writing.

5) The student who learns through this approach will mislearn the art of reading and writing.

Given these failings, it is clear that the California Learning Assessment System falls into the category of pseudo critical thinking, and will not help students and teachers to develop their critical thinking abilities, but will hinder this process. Indeed, when classroom teachers receive copies, as they surely will, of test items, sample answers, commentary, and scores, they will use them as a guide for instruction. Thousands of school children will lose an opportunity to begin to become critical readers and writers. Thousands of school children will themselves learn to confuse recall with knowledge, subjective preference with reasoned judgment, irrational with rational persuasion. They will learn to use language vaguely and to think that their subjective pronouncements are not to be criticized. Their reasoning skills will remain abysmally low.

Is The California Assessment Fiasco a Fluke? The Educational Bureaucracy and Self-Deception

One of the most significant facts about the California language arts test fiasco is that it is not a fluke. But of course, neither is it a plot to undermine education. The situation is worse than either. As a fluke it could be corrected. As a plot the perpetrators could be severely dealt with when exposed. No, the pathetic side of the case is that there are systemic reasons why educational bureaucracies, framed as they are, will continue to generate just such fiascos regularly and predictably. And predictably, many will be taken in. Furthermore, because classroom teachers have emerged from a long-term training that reflects a similar background to that of the test designers, they also will fall easily into line with the flawed thinking passed down to them.

To deal with the problem at its roots, we must own the fact that there are significant problems in education due to its wide-spread and large-scale bureaucratization.

One of the most significant facts about the California language arts test fiasco is that it is not a fluke. But of course, neither is it a plot to undermine education. The situation is worse than either.

Large-scale bureaucratization entails, or at least makes highly probable, a high degree of narrow specialization — and specialization tends to bring fragmentation, narrowness of vision, politicization, and self-deception in its wake. The fragmentation and narrowness of vision makes it difficult to effect fundamental changes because the parts do not work together in a rational way and no one sees clearly that this is so, since each element in the structure becomes an end in itself, to itself.

It is almost impossible for the most pressing problems of education to become “issues” in educational bureaucracies because the focus is inevitably on the political, the narrow, the fragmented part or parts. With each part serving itself as an ultimate end — including those on the top — the whole is left to take care of itself. No one is left responsible for it.

The executive wing is also focused on itself and typically is satisfied with or driven to manufacture an illusion of serving the announced or official goals and ends. Meanwhile, the politicization and self-deception helps hide those realities most unpleasant to think about, and to have to face, and consequently those realities most in need of change.

This includes, of course, the most significant one today: the fact that modern American bureaucratic schooling is a system that preserves at its heart a mode of instruction that is a hold-over from the 19th Century and whose consistent effect is a superficial one. Most students in most classes most of the time are not actively engaged in learning what is worth learning. Most students are, on their side, not taking their education seriously. On the teaching side, they are not given challenging instruction. They are not engaged in genuine intellectual work. They are not developing intellectual standards or discipline. And, most assuredly, they are not learning to reason scientifically, mathematically, geographically, economically, sociologically, or morally. (See “Research Findings,” p. 19.)

On the shoddy foundation of didactic instruction and passive, lower order learning, the rhetoric of high goals and ideals, the propaganda of the schools, is overlaid.

Modern educational bureaucracy has developed multiple ways to appear to be, and to appear to be doing, what it is not. In the K–12 domain especially, the history of education in the 20th Century is a triumph of propaganda and self-deception. This is documented in story after story of wave after wave of pseudo reform following pseudo reform, of new buzz words and new jargon replacing old buzz words and old jargon — each set of new words serving as a new mask to obscure the one-and-the-same consistent lower order face. (Cf “Critical Thinking in Historical Perspective chapter.) Unfortunately, virtually everyone in the game has a stake in making their playing of it look more honorable, more lofty, more noble and effective than it really is.

You may remember that it is only some three years now since every state in the union, through the massaging and manipulation of statistics by its own state department of education, proudly announced that its students had scored above the national average!!! This is the kind of self-serving propaganda and trickery that is the daily fare of educational reality.

State departments of education, as I have suggested, are a particularly interesting manifestation of the workings of the educational establishment. Each consists of huge bureaucracies, interlaced with committees that are in turn tied into networks of teachers and administrators spread across their states. The microcosm we analyzed in this chapter (the California State Department of Education’s new English-Language Arts Assessment materials) is still officially in draft stage, but already is being highly touted as a refined, future-oriented, testing instrument. We’ve seen what the ELAA has done. Let’s see what CLAS and the Department of Education says it has done. The test is being represented to the public and to teachers within the state and the nation as having,

The goal ... to evaluate students’ capacities for insightful, productive thinking with tests that support the finest curriculum and instructional programs in the language arts ... [California’s Learning Assessment System, CDE Publication]

The most important single component of the new assessment system will be the statewide performance standards, and the most important outcome of the assessment process will be the internalization of those standards in the thinking and work of teachers, students, and parents. The performance standards will undergird all aspects of the educational enterprise; serving as the center of the seamless web of teaching, learning and assessment. [Some Principles and Beliefs about the Role of Assessment in California's School Reform Plan, February 15, 1993, CDE]

Let’s see what the legislature mandated, Senate Bill 662 (Hart)

Develop a system for producing valid, reliable individual scores and to develop and implement common statewide performance standards of student achievement as a basis for reporting all test results and setting targets for improvement.

It sounded simple enough but it provided us with a classic model of pseudo critical thinking in the educational establishment. The manner in which it is structured provides a textbook case. By mirrors, illusion, and standards self-deception, it creates the

The result is that the most fundamental problem in education today — that students are not learning to reason well — is not only ignored, it is intensified.

appearance of substantial change and reform. In fact, nothing is really being changed. The result is that the most fundamental problem in education today — that students are not learning to reason well — is not only ignored, it is intensified.

Fragmentation and Vested Interests

There are a number of reasons why it is unlikely that fundamental reforms will be effected by state departments of education or that this critique, by itself, will bring about fundamental change. In the first place, most of the positions within the state departments of education are for specialists, for example, positions for those specializing in nutrition, for those specializing in transportation, for those specializing in the laws regarding education, for those specializing in learning disorders, for those specializing in a and b and c and d and e and f and g. Virtually no one, however, has a responsibility directly connected to the fundamental goals of education (except possibly the director of instruction). Each specialist has his or her own special interest to focus on and a special group of stake-holders to represent.

When there is a need to develop an assessment instrument, like the one we examined of the California Department of Education, a large group of teachers and administrators from around the state are appointed. For example, there are 33 members of the CAP English-Language Arts committee. In addition to the main committee there is a supporting committee (“Reserve Team”) consisting of 46 additional members. According to the California State Department: “These development team teachers have been responsible for shaping the test format, developing prompts for the assessment, and constructing scoring rubrics.” (p. I-2)

Many political considerations go into the selection of the members of the development teams; most of the members, for example, are not scholars with publications that could be used as the basis of selection. There is an effort made to balance the committee by region, gender, race, and ethnicity. In addition, many members will have personal agendas to advance.

The test becomes, then, both invalid and unreliable. In fact, it becomes a hot-bed of pseudo critical thinking, with a variety of misconceptions and flaws emerging. But while its intellectual value is low, its political value is high.

There are usually three “Consultants/Advisors” selected from universities to bring in the over-arching theoretical framework. These consultants are usually the pipeline to the latest buzz words and to the theory behind them. The consultants concede to each other the right to get their favored terms into the language of the test materials.

The result of this process, as we have seen, is that the official “standards” embodied in the test become extraordinarily numerous. Many of them remain vague and ill-defined. Others take on a dangerous ambiguity. The diverse criteria and the open-ended nature of the directions combined with the ill-defined nature of the terms, opens the way to arbitrary and inconsistent grading of student responses. The test becomes, then, both invalid and unreliable. In fact, it becomes a hot-bed of pseudo critical thinking, with a variety of misconceptions and flaws emerging. But while its intellectual value is low, its political value is high. The various political interests around the state are served. The media has a simplistic event to cover. Parents can delight in the fact that the scores will go up. (How can they go down when anything can count as a good answer?) The politicians will gain because they can speak of their state as in the vanguard. And so it goes. A new pseudo reform is put in place and the educational bureaucracy grinds on until the next wave of public criticism requires it to generate a new and fresh illusion of change, a new catalogue of counterfeit, bogus, and superficial “reforms.”

So What Can We Do? Recommendations

There is a pressing need to develop networks of educators, parents, politicians, and business people who see the need for truly fundamental reform. That reform must be advanced simultaneously on many levels, for it is not going to result from action on one level alone. Because it must go to the roots of things, because it must be substantial, because it involves deep understandings, it must be incremental, evolutionary, and long-term. Everyone with the insight to see the problem comprehensively should act within the sphere of his or her greatest influence. There is a role for everyone concerned to exercise influence for the better: for parents, for public citizens, for business people, for civic leaders, for superintendents, for teachers, for college professors, and ... yes, even for those in state educational bureaucracies. Let us consider each briefly in turn.

What Can Parents Do?

Insightful parents can make the case for an emphasis on intellectual discipline and reasoning in the school curriculum. They can ask whether there is any long-term in-service in critical thinking and reasoning. They can ask what intellectual standards the students are being taught and how they are being taught them. They can make the case to other parents. They can write letters to the local papers. They can organize groups of parents who petition the school board. And most important they can develop a home environment in which the reasonability and intellectual discipline of their children is fostered, in which both they and their children routinely ask and give good reasons in support of their decisions and reason together about issues of importance not only to the family but to the broader society as well.

What Can Citizens Do?

Insightful public citizens can make the case for an emphasis on intellectual discipline and reasoning in the school curriculum in virtue of the need to develop voters who will help the country maintain a democratic form of government. They, too, can go to the local school board and ask whether there is any long-term in-service in critical thinking and reasoning. They, too, can make the case to parents and other citizens. They can contact civic groups. They can write letters to the local papers. They can organize groups of interested citizens to petition the school board.

What Can Business People Do?

Insightful business people can use the respect that their success commands to exercise influence, alone or in concert with others, over educational decisions about what to teach and how to teach it. Since their success will be increasingly dependent upon their bringing critical thinking into the inner workings of their own businesses, on workers learning how to continually relearn and improve in their performances and in the systems they use, they will have ready access to models and paradigms that can be used to illuminate what should be happening in the classroom. Increasingly, cutting-edge businesses are moving away from an emphasis on hierarchy to an emphasis on group problem solving. Since critical thinking is essential to effective group problem solving, progressive business people will be able to talk intelligibly with educators and other citizens about how problem solving structures function in business and how parallel classroom problem solving groups might be set up. And, certainly, there are any number of civic groups that business people with insight might address on the problem of educational reform, putting emphasis, of course, on the missing foundation: the failure of teachers to learn how to think critically themselves and to teach for that thinking in their instruction, the failure to focus education, in other words, on “carefully-reasoned” problem solving. Finally, insightful business people can form alliances with insightful educators, to create symbiotic, reflective, mutually useful dialogues on what each group can learn from the other and how each can profit by working together.

What Can Civic Leaders Do?

Insightful civic leaders can draw public attention to the need for intellectual discipline and reasoning in instruction. They can articulate publicly the key links to developing responsible citizens, moral persons, and workers on the cutting edge of development. They can use their access to a more public forum by focusing the discussion of educational reform on the historical problem of the educational bureaucracy and its tendency to generate pseudo reform. They can create a public awareness of the importance of reasoning, critical thinking, and problem solving. They can help organize civic groups. They can use their superior access to other persons of leadership and influence to facilitate significant pressure on the educational bureaucracies. They can make contact with insightful and responsible politicians who are in a position to facilitate appropriate legislation.

What Can Superintendents Do?

Insightful superintendents can make the case for an emphasis on intellectual discipline and reasoning in the school curriculum to the school board, administration members, teachers, and parents. They can ensure that there is long-term in-service in critical thinking and reasoning. They can ensure that students are being taught intellectual standards in depth. They can create incentives to teachers motivated to move in this direction. They can make the case to civic groups. Most importantly they can model reasonability and help create an atmosphere conducive to making the school a network of communities of inquiry.

What Can Teachers Do?

Insightful teachers can make the case for an emphasis on intellectual discipline and reasoning in the school curriculum. They can request and help design long-term in-service in critical thinking and reasoning. They can bring intellectual standards into the classroom. They can make the case to parents. They can work with other teachers to foster a school environment in which reasonability and intellectual discipline are accepted school norms. Most importantly, they can routinely ask for and give good reasons in the classroom. They can foster student reasoning in history, science, math, and so forth. They can ensure that students must regularly assess their own work using intellectual standards.

What Can College Professors Do?

Insightful college professors can make the case for an emphasis on intellectual discipline and reasoning in the college curriculum. They can request and help design long-term faculty development in critical thinking and reasoning. They can bring intellectual standards into the classroom. They can do research on the significance of critical thinking and reasoning in their discipline. They can work with schools and departments of education to ensure that those studying to become teachers take classes that require reasoning and disciplined thought. They can articulate the need for prospective teachers to learn how to design assignments that require reasoning and critical thinking. Most importantly, they can routinely foster reasoning in their own classrooms and ensure that their students must regularly assess their own work using intellectual standards.

What Can Those in State-wide Bureaucracies Do?

Insightful members of state-wide bureaucracies (who recognize the systemic ways that educational bureaucracies have fostered pseudo reforms and constructed ill-designed assessments) can play a number of significant roles. They can inform themselves and others they work with of the fundamental changes that are being made in businesses adopting structures contrary to those of traditional bureaucratic organization. They can foster movement toward problem-solving teams. They can raise broader and deeper issues. They can recommend hiring people with broader vision and more developed reasoning abilities. They can help to work against narrow specialization. At the same time, they can argue for more appropriate use of experts, so that those who lack expertise in a subject will not become, for example, principal designers in tests or assessment instruments in that subject. They can argue for the construction of assessment instruments that assess reasoning in every subject area and so help to integrate emphases across subject areas.

Caveat

Doubtless you noticed my emphasis on “insightful” in characterizing those who can make important contributions to reform. It is important to underscore the problem of pseudo reform, which emerges when well-meaning persons use their intelligence inadvertently to re-duplicate an old problem in new form, creating the illusion of change. Many persons today are unwilling to think through the implications of accelerating change and intensifying complexity. Many are subconsciously wedded to rigid ideas and a static way of thinking. Many are taken in by their own platitudes and high-sounding words. These facts guarantee that a long struggle will be required to work through the superficial and work into the substantial.

Final Conclusion

Pseudo critical thinking is more or less inevitable in the educational bureaucracies, given the way we have traditionally arranged and ordered things. This is illustrated, as we have seen, in the way the American educational establishment goes about designing assessment. Unfortunately, faulty assessment leads to faulty teaching, which leads to more faulty thinking in society, in business, in politics, and in everyday social life. The California Department of Education is a model case of American educational bureaucracy at work and the new California reading and writing assessment instrument is the typical resultant bad practice. Good thinking is now a fundamental human need. And though it will take generations to fully evolve from a society in which pseudo critical thinking is dominant to one in which sound, fairminded, ethically-informed reasoning is dominant, every step in that direction will reduce the amount of suffering and injustice that exists and increase, by degrees, human well being and quality of life. It is our intellectual and moral responsibility to make some contribution to this evolution. Though we are only at the beginnings of this evolution, the irresistible dynamic of accelerating change and intensifying complexity will eventually force it upon us. I hope we learn our lessons sooner rather than later, that the price of waste and unnecessary human misery may be as little as possible.

References

All of the references in this chapter (unless otherwise noted) are from the Samplers for English Language Arts Assessment, for Elementary and High School, disseminated statewide in the Spring, 1993, by the California Department of Education.

{"id":"268","title":"","author":"","content":"<h5><span style=\"color: #000099; font-size: small;\">The Problem of Scoring</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">As you might expect, all of the confusions above come home to roost in the design for scoring student &ldquo;performances&rdquo; in reading and writing. For example, in explaining the design with respect to assessing elementary reading, the authors introduce us to a 15-point list under the head of the reading performances of effective readers.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Effective readers connect with, reflect on, and challenge the text. Readers do not need to show evidence of all the performances listed here. The discerning and insightful reader may display a broad spectrum of reading behaviors or may investigate a few selected behaviors in great depth. The exemplary reader may show variety, complexity, breadth, and/or depth. Through their writing and graphics, these readers show convincing evidence of their ability to construct meaning. They may:</span></p>\r\n<ol>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Experiment with ideas; think divergently; take risks; express opinions (e.g., speculate, hypothesize, explore alternative scenarios; raise questions; make predictions; think metaphorically).</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Explore multiple possibilities of meaning; see cultural and/or psychological nuances and complexities in the text.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Fill in gaps; use clues and evidence in the passage to draw conclusions; make plausible interpretations of ideas, facts, concepts, and/or arguments.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Recognize and deal with ambiguities in the text.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Revise, reshape and/or deepen early interpretations.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Evaluate; examine the degree of fit between the author&rsquo;s ideas or information and the reader&rsquo;s prior knowledge or experience.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Challenge the text(s) by agreeing or disagreeing, arguing, endorsing, questioning, and or wondering.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Demonstrate understanding of the work as a whole.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Show sensitivity to the structure of the text(s): how the parts work together; how characters and/or other elements of the work(s) change.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Show aesthetic appreciation of the text(s); see linguistic and structural complexities. </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Allude to and/or retell specific passages(s) to validate and/or expand ideas.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Make connections between the text(s) and their own ideas, experiences, and knowledge.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Demonstrate emotional engagement with the text(s)</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Retell, summarize, and/or paraphrase with purpose.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Reflect on the meaning(s) of the text(s), including larger or more universal significance; express a major understanding about or insight into a subject, an aspect of self, or of life in general.</span> </li>\r\n</ol>\r\n<p><strong><em><span style=\"color: #333333;\">The Escape Hatch<br /> </span></em></strong><br /> <span style=\"font-family: Arial,Helvetica,sans-serif;\">First off, it is clear that since each scorer can pick and chose from such a wide variety of criteria (a number of which as we shall see are extremely vague), the impartiality of application of the criteria is suspect. Also note that any and all of these activities can be done either defensibly or indefensibly. Hence, if these are to be assessed in order to be credited or discredited, criteria must be provided for each of the individual &ldquo;performances&rdquo; cited (over 50 are buried in the list). We need some explanation of how they expect someone assessing student reading to apply them. Consider each of the following. The assessor is left to her own intuitions in determining whether or not a student&rsquo;s reading: </span></p>\r\n<ul>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">is insightful; discerning; perceptive; </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">is sensitive to linguistic, structural, cultural and psychological nuances and complexities; </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">entertains challenging ideas; grounds meaning in acute perceptions of textual and cultural complexities. </span></li>\r\n</ul>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Do the assessors really know how to impartially assess whether or not a student reader is being &ldquo;sensitive to a psychological nuance&rdquo; or to a &ldquo;structural nuance&rdquo; or a &ldquo;cultural nuance?&rdquo; Or can they impartially determine whether or not the student reader is entertaining a &ldquo;challenging&rdquo; idea? Or whether a student perception is &ldquo;acute&rdquo; or not? Isn&rsquo;t it highly probable that different assessors are going to have somewhat different conceptions of each of these matters, for example, one thinking a given idea is &ldquo;quite challenging&rdquo; and another thinking it is not? Surely this much is clear! No criteria, however, are provided, and this invalidates any attempt to use the results to assess one student&rsquo;s performances year after year, as well as to assess all California students collectively, year after year. Instead, having given us an array of vague descriptors, the authors now largely set them aside and focus instead on a six-point &ldquo;Scoring Guide&rdquo; that is to be used in distinguishing student reading into the following categories:</span></p>\r\n<ul>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Exemplary Reading Performance (Six Points)</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Discerning Reading Performance (Five Points)</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Thoughtful Reading Performance (Four Points)</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Literal Reading Performance (Three Points)</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Limited Reading Performance (Two Points)</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Minimal Reading Performance (One Point)</span> </li>\r\n</ul>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">These general descriptors are of very little use. For example, consider the words &ldquo;discerning&rdquo; and &rdquo;thoughtful.&rdquo; It is not obvious that one is better off being &ldquo;discerning&rdquo; than being &ldquo;thoughtful.&rdquo; It is also not obvious why &ldquo;literal&rdquo; is above &ldquo;limited.&rdquo; It is certainly not clear why the lowest score is &ldquo;minimal&rdquo; reading. What ever happened to just plain &ldquo;poor&rdquo; reading? Has it disappeared or is it one of many forms of &ldquo;minimal&rdquo; reading. (See Colleen&rsquo;s reading on pp. 24&ndash;27.) Is it &ldquo;minimal&rdquo; or just plain &ldquo;poor?&rdquo;)</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">But that is not all. Each of the terms listed in each of the six point scoring guide create further problems for the conscientious scorer. Consider the terms in the first category alone, that of &ldquo;exemplary reading performance.&rdquo; A person who takes seriously the characterizations of this first category should be prepared to notice and assess whether or not the student is:</span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"100%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"color: #ffffff;\">xxxxxxxxx</span></td>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\">filling in gaps<br /> drawing meaning <br /> entertaining ideas<br /> raising questions<br /> taking exception<br /> agreeing; disagreeing<br /> exploring possibilities <br /> developing connections<br /> making connections</span></td>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\">drawing on evidence<br /> objecting to text features<br /> considering the authority of the author<br /> considering the quality of the author&rsquo;s sources<br /> suggesting ways of rewriting the text<br /> embracing the ideological position of a text<br /> resisting the ideological position of a text<br /> revising their understanding as they read<br /> carrying on an internal dialogue</span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><strong><span style=\"color: #000000;\">The State Department Criteria for an Exemplary Reading Performance<br /> </span></strong><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> <strong>1)</strong> An exemplary reading performance is insightful, discerning and perceptive as the reader constructs and reflects on meaning in a text. Readers at this level are sensitive to linguistic, structural, cultural, and psychological nuances and complexities. They fill in gaps in a text, making plausible assumptions about unstated causes or motivations, or drawing meaning from subtle cues. They differentiate between literal and figurative meanings. They recognize real or seeming contradictions, exploring possibilities for their resolution or tolerating ambiguities. </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">They demonstrate their understanding of the whole work as well as an awareness of how the parts work together to create the whole. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Readers achieving score point six develop connections with and among texts. They connect their understanding of the text not only to their own ideas, experience, and knowledge, but to their history as participants in a culture or larger community, often making connections to other texts or other works of art. Exceptional readers draw on evidence from the text to generate, validate, expand, and reflect on their own ideas.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">These readers take risks. They entertain challenging ideas and explore multiple possibilities of meaning as they read, grounding these meanings in their acute perceptions of textual and cultural complexities. They often revise their understanding of a text as they re-read and as additional information or insight becomes available to them. They sometimes articulate a newly developed level of understanding.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Readers demonstrating a score point six performance challenge the text. They carry on an internal dialogue with the writer, raising questions, taking exception, agreeing, disagreeing, appreciating or objecting to text features. They may test the validity of the author's ideas, information, and/or logic by considering the authority of the author and the nature and quality of the author's source(s). They frequently suggest ways of rewriting the text, speculating about the ideology or cultural or historical biases that seem to inform a text, sometimes recognizing and embracing and sometimes resisting the ideological position that a text seems to construct for its reader.</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">Must the Student Just Do It or Do It Well? </span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Even more problematic than the likely disagreement among assessors as to the application of the vague standards provided is, as I suggested above, the problem of the assessors being given many criteria that name processes that can, in principle, be done well or poorly. The directions do not explain whether, to be credited, the student is obliged to use the cited processes well or simply use them in any way whatsoever. That is, there is no indication as to whether the assessor is to evaluate the &ldquo;quality&rdquo; of the way the student is </span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; Unfortunately, once we examine the actual student writing examples along with the commentary provided, it becomes painfully clear that the assessors were simply looking to see if the student in any sense used the process and did not have, or did not use, criteria to assess how well the students used the processes credited.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">doing these things or simply certify the fact of doing these things &mdash; however poorly. Remember, a student who is drawing an absurd meaning is still drawing a meaning. A student who is making a trivial connection is still making a connection. A student who is drawing on irrelevant evidence is still drawing on evidence. A student who is raising a silly or superficial question is still raising a question. And so forth, and so on. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Unfortunately, once we examine the actual student writing examples along with the commentary provided, it becomes painfully clear that the assessors were simply looking to see if the student in any sense used the process and did not have, or did not use, criteria to assess how well the students used the processes credited. The California State Department of Education falls directly into the trap of failing to discriminate between these crucial differences. The assessors are left to their own devices. They can draw these distinctions or fail to draw them. It is clear that most failed to draw them. This is a fatal flaw in the assessment. It renders the results of the assessment virtually useless.<br /> <br /> </span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">The Misassessment of Elementary Reading</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The only elementary reading passage which is given with scored examples is from a story by John Gardiner called Stone Fox. It is an emotionally explosive story, one chosen perhaps to ensure an emotional response. The children are simply asked to give some of their &ldquo;thoughts, feelings, and questions&rdquo; about what they are reading. No other kind of writing is given as an example.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em><span style=\"color: #333333;\"><strong>Insufficient Directions Are Given</strong><br /> </span></em><br /> </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">Now the first remarkable feature of the Stone Fox reading prompt is that the student readers are given no indication whatsoever of the purpose for which they are reading the story. This is ironic in the light of the fact that the CAP materials emphasize the fact that one can read and write &ldquo;for different purposes&rdquo; (p. I-3).</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">And yet, here, the students are asked to read for no particular purpose. Are they to read in a casual fashion, simply for amusement? Or are they to do a &ldquo;close reading&rdquo; for detail? Are they to be analytical and reflective, or not? The students are given no indication of how they will be assessed. Hence, they are asked to engage, at best, in an ill-defined &ldquo;performance.&rdquo; Nothing is given in the way of directions to the students except &ldquo;Read to see what happens&rdquo; and a place to the right in which the student may write notes under a column head titled, &ldquo;My thoughts, feelings, and questions about what I am reading.&rdquo; The students were apparently not even told that they should try to write out as much of their thoughts, feelings, and questions as they could. Some readers might presumably have thoughts, feelings, and questions they would not bother to express.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Consider that you are a student reading the story. You see a column at the right which says, &ldquo;My thoughts, feelings, and questions about what I am reading.&rdquo; How should you understand it? Wouldn&rsquo;t you wonder which thoughts ... which feelings ... based on what? What am I to think about? Why am I to think about it? </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">In any case, since the student reader is not told that she is going to be evaluated on what she writes and is told none of the criteria, why should she be motivated to fully express her thoughts and feelings? Consider, on the other hand, what the directions might have said.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><span style=\"color: #0044aa;\"><em><strong><span style=\"color: #333333;\">Possible Directions</span></strong><br /> </em></span><br /> </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">When we read a story we have to try to understand and follow its meaning. We try to figure out what it is saying and we try to connect it with our own life in some way. We want you to read this story and see what it means to you. Why do you think it was written? Do you think that it was written well? Do you think it is true to life? Does it illustrate anything that you believe is important? Please write in as many of your thoughts on these questions as possible as you read. Help us to understand what is going on in your mind as you are reading and trying to relate this story to your life.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em><span style=\"color: #333333;\"><strong>Can One Evaluate Purely Subjective Responses to Stories?</strong><br /> </span></em></span><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> When the student reads that she is directed to express her &ldquo;feelings,&rdquo; with no explanation given why she is so directed, then how can we legitimately go on to judge those feelings and &ldquo;score&rdquo; them one through six? Suppose the student quite sincerely said, &ldquo;In my view this is a sentimental story that insults my intelligence. I feel disgusted when I read it and bored silly.&rdquo; How should the assessor evaluate that &ldquo;feeling&rdquo; response? Is there any way to discredit it according to the directions? Certainly not. It is as good a response to a request for &ldquo;feelings&rdquo; as any other. To put the point succinctly, either we help students understand the difference between a request for a purely subjective response, (How do you feel when you read this?) and a request for a more reasoned response (What feeling do you judge the author wants you to feel and why? In your judgment is the author successful? Tell us the reasons why you think so.) or we must not indulge in any assessment of the students &ldquo;feelings.&rdquo; We have no legitimate grounds for doing so. The student can legitimately take the request to be one that asks for a subjective response and a purely subjective response cannot be impartially assessed or scored. <br /> <br /> </span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">The Misassessment of Writing at the Elementary Level</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Different kinds of writing will be assessed at different levels: Elementary: persuasive writing ... Middle School: problem solution, evaluation and speculation about cause and effect ... High School: evaluation, speculation about cause and effect, interpretation and controversial issues plus a reflective essay. (p. I&ndash;4)</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The grade four writing assessment is designed to reflect a variety of purposes for which children write: </span></p>\r\n<ol>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>Expressive</strong> writing<br /> (&ldquo;This is what I see, think, and feel . . .&rdquo;) <br /> </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>Persuasive</strong> writing<br /> (&ldquo;This is what I believe and why I think you should believe it . . .\")<br /> </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>Narrative</strong> writing <br /> (&ldquo;This is what happened . . .\")<br /> </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>Informational </strong>writing <br /> (&ldquo;This is what I know and how I know it . . .\")</span>&nbsp; </li>\r\n</ol>\r\n<h5><em><span style=\"color: #333333;\">Criteria For Persuasive Writing</span></em></h5>\r\n<p>Persuasive writing is explained in the following terms (p.lll &mdash; 12)</p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Persuasive writing requires students to choose positions, to make judgments, to offer proposals, and to argue convincingly for their beliefs and ideas. However, some students may choose to explore both sides of an issue and then offer a compromise, . . . Effective writers use evidence such as examples or anecdotes to support their arguments. Convincing arguments may appeal to logic, emotions, and/or philosophical beliefs.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Persuasive writers establish themselves as informed, knowledgeable individuals.</span><span style=\"font-family: Arial,Helvetica,sans-serif;\"> They orient readers . . . More than any other kind of writing, argument requires writers to consider their audience . . . The best persuasive writers systematically develop arguments with a strong sense of coherence and movement throughout the piece.</span></p>\r\n<h5><em><span style=\"color: #333333;\">California&rsquo;s Standards for Exceptional Writing (Six Points)</span></em></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">An exceptional score (6 points) must meet the following standards:<br /> <br /> <strong>Focus/Coherence</strong></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>Position</em>. Writers of six-point papers usually assert and maintain a clear position throughout the piece; they present evidence and explanations in a purposeful way.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Occasionally these writers will effectively evaluate both sides of an issue and offer a reasonable compromise; or they may conclude that neither position is preferable, or they may suggest a third position.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>Organization.</em> Writers arrange reasons, examples, information and/or personal anecdotes in a discernible and effective pattern resulting in an overall persuasive effect.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>Coherence.</em> Writers provide overall links or transitions; they present arguments, evidence, and reasons logically so that the overall effect is one of coherence.<br /> <br /> </span></p>\r\n<p><span style=\"color: #0044aa; font-family: Arial,Helvetica,sans-serif;\"><strong><span style=\"color: #000000;\">Elaboration</span><br /> </strong></span><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> <em><span style=\"color: #000000;\">Depth/Density of Arguments.</span></em> Writers thoroughly develop and elaborate their reasons, examples, information and/or anecdotes. Some writers may develop only one reason or example, but they do so in depth; others may choose to develop several appropriate reasons, examples, and so forth.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>Relevance of Arguments.</em> Writers choose and present appropriate reasons, examples, information, and so forth to support their argument(s). They show their arguments are valid based on prior knowledge, personal experience and reflection.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>Audience Awareness.</em> Writers choose and present arguments with a clear awareness of reader needs. They often show credibility and a sense of authority by revealing source(s) of information. They may anticipate possible reader response by including some counter-arguments.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><span style=\"color: #0044aa;\"><span style=\"color: #000000;\"><strong>Style</strong></span><br /> </span><br /> <em>Word Choice.</em> These writers use lively, interesting concrete language that carries precise meanings and emotions. Word choice is appropriate to the writer's purpose.<br /> <br /> <em>Sentence Variety.</em> Writers vary sentence length and type, making the writing interesting and readable.<br /> <br /> <em>Voice.</em> Writers evidence confidence, conviction, belief and sometimes enthusiasm.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><span style=\"color: #0044aa;\"><strong><em><span style=\"color: #000000;\">Problems With the Criteria In Assessing Writing</span></em></strong><br /> </span><br /> </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">Once again there are a host of problems with the criteria, the most serious being that, just as in the criteria for reading, there is no indication of whether the students are expected to do any of the above well, or simply do them in any form whatsoever. For example, consider the claim that &ldquo;convincing arguments may appeal to logic, emotions, and/or philosophical beliefs.&rdquo; Suppose a student uses &ldquo;convincing&rdquo; but &ldquo;fallacious&rdquo; logic &mdash; is the student to be credited? Or suppose the student appeals to the emotions of the reader by engaging in name-calling (&ldquo;This stupid communist idea...!&rdquo;) &mdash; is the student to be credited? The formulators of the criteria are seemingly oblivious of the problem.</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">A Grade Four Essay Judged to Illustrate High-Range Achievement</span></h5>\r\n<p><span style=\"color: #666666; font-family: Arial,Helvetica,sans-serif;\"><strong><em>The CAP Commentary</em></strong></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">From the opening . . . this writer exudes confidence, focuses on an audience and takes a firm stand on the issue. These attributes of persuasive writing are maintained throughout the piece.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Support for the writer&rsquo;s credible arguments in favor of a park versus a mall are drawn from personal experiences. She uses examples: &ldquo;We already have over five stores in our plaza,&rdquo; and notes that the park is a &ldquo;perfect place to play, take your dog for a walk.&rdquo; She includes reflection, &ldquo;Do you honestly think that children at the age of two or three are going to enjoy a mall ...?&rdquo; Her arguments are appropriate and appeal to reason (safety) and emotions (&ldquo;I didn&rsquo;t come here to watch those C.A.T.s tare up that park!&rdquo;)</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">This writer is consistently aware of and appeals to her audience with appropriate tone and lively language.</span></p>\r\n<p><span style=\"color: #0044aa; font-family: Arial,Helvetica,sans-serif;\"><em><span style=\"color: #000000;\"><strong>The Problem of Subjectivity, Once Again</strong></span></em><br /> </span><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">The problem of subjectivity that was so apparent in the elementary reading assessment reappears again in the writing assessment. It is clear in this, and in other examples, that the commentators are using the criteria for good persuasive writing literally and not making any real judgments of reasonability. Hence, if the student says anything which can be construed as falling under one of the criteria then that is credited. For example, if she uses an emotional appeal then credit the emotion criterion (however irrelevant or inappropriate it might be); if she says something that can be described as &ldquo;reflection,&rdquo; then credit the reflection criterion (however irrelevant or inappropriate the &ldquo;reflection&rdquo; might be). Any kind of a move that would work with an audience, in the view of the commentator, is credited as good (regardless of how irrational it might be).</span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; To suggest that this is good persuasive writing is to teach children exactly the wrong lesson. It fails to show them the vital distinction between reason and its counterfeit.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">But the significant question for anyone concerned with the traditional values of education is whether a reasonable audience should be persuaded or &ldquo;moved.&rdquo; The important distinction for students to grasp is that between what might be called &ldquo;low level rhetorical appeals&rdquo; and those appeals which would convince or &ldquo;move&rdquo; a reasonable audience. No such distinction is recognized by ELAA. The criteria are used crudely, without any important intellectual distinctions in evidence. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The implication of this is that if a student wrote a very rational appeal, which recognized the weaknesses in her position and the need to qualify her claims, she would be downgraded because she would not be &ldquo;exuding confidence,&rdquo; &ldquo;taking a firm stand on the issue,&rdquo; etc. Indeed, though her presentation might appeal to a &ldquo;jury of reasonable persons,&rdquo; the graders of the ELAA would not be impressed.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">In this particular essay, for example, though it is good for children to display confidence, etc., the issue of park versus mall should be decided on rational grounds. Yes, it is important that the park is a favorite place for children and that they are safe there, but this is really a decision about options, and alternatives, and relative costs and benefits. This issue should be decided by looking at these factors and weighing them as rationally as possible, not on the basis of emotional appeals, like that of the child. To suggest that this is good persuasive writing is to teach children exactly the wrong lesson. It fails to show them the vital distinction between reason and its counterfeit.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Suppose the child has to present a case before an audience she knows to be racist; if one reads and applies the criteria presented for good persuasive writing in ELAA, the student who plays on those racist sentiments will score higher than one who opposes them. </span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; What apparently matters in the mind of the ELAA assessors is that one successfully persuades, not that one argues reasonably. But it is harder to imagine a distinction which it is more vital for the educated person to grasp.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">What apparently matters in the mind of the ELAA assessors is that one successfully persuades, not that one argues reasonably. But it is harder to imagine a distinction which it is more vital for the educated person to grasp. Indeed, one might almost regard it as a criterion of being educated that one sees the difference between fairminded, reasonable argument, and self-seeking, low-level, persuasive rhetoric. The demagogues may often win the day, but do we want to use public monies to generate armies of demagogues, all having mastered the art of demagogery at the public expense?</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">In another example, which contains some very good reasoning about why another child should feel good about himself, the commentators say, &ldquo;She arranges her reasons and evidence in a sophisticated pattern.&rdquo; There is no mention of whether the reasons and evidence are relevant or irrelevant, true or false, good or bad.</span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; It is not that the reasons are arranged in a sophisticated pattern, but that they are good reasons!</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Yet, again, isn't this what really matters? It is not that the reasons are arranged in a sophisticated pattern, but that they are good reasons!</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">In another, mid-range, example, about how to solve problems between children and parents, the commentators say (among other things), &ldquo;Audience awareness is evident throughout, although appeals lack the vigor and exactness of higher score point papers. The use of bullets to summarize the writer&rsquo;s arguments is an effective tool and adds to her sense of conviction.&rdquo; Why is the writer criticized for a lack of vigor and praised for conveying a sense of conviction? Neither is a virtue in itself. What matters is that one exhibits the appropriate degree of vigor and conviction, depending on the strength of one&rsquo;s case. Vigorously arguing a weak case and displaying conviction despite poor supporting reasons ought to be marked down, not praised. </span></p>\r\n<h5 style=\"margin-right: 0px;\" dir=\"ltr\"><em><span style=\"color: #333333;\">The Young Hitler Scores High on the CAP Test</span></em></h5>\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\"><span style=\"font-family: Arial,Helvetica,sans-serif;\"><span style=\"font-family: Arial,Helvetica,sans-serif;\">We can now make our point dramatically by considering how the following piece of persuasive, but highly irrational, writing should be graded according to the ELAA criteria.</span></span></p>\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\"><span style=\"font-family: Arial,Helvetica,sans-serif;\"><span style=\"font-family: Arial,Helvetica,sans-serif;\"> </span></span></p>\r\n<table border=\"1\" cellspacing=\"1\" cellpadding=\"1\" width=\"80%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\">\r\n<p><br /> &nbsp;. . &nbsp;the greatest revolution Germany has undergone was that of the purification of the Volk [people] and thus of the races, which was launched systematically in this country for the first time ever. <strong><em>[From the opening the writer exudes confidence, focuses on an audience and takes a firm stand on the issue.] </em></strong></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The consequences of this German racial policy will be more significant for the future of our Volk than the effects of all the other laws put together. For they are what is creating the new man. They will preserve our Volk from doing as so many historically tragic past prototypes of other races have done: lose their earthly existence forever because of their ignorance as regards a single question. <strong><em>[The writer arranges reasons, examples and information in a discernible and effective pattern resulting in overall persuasive effect.]</em></strong></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">For what is the sense of all our work and all our efforts if they do not serve the purpose of preserving the German being? And what good is any effort on behalf of this being if we omit the most important thing to preserve it pure and unadulterated in its blood? <em><strong>[The writer asserts and maintains a clear position throughout. He also chooses and presents arguments with a clear awareness of reader needs.]</strong></em></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Any other mistake can be rectified, any other error can be corrected, but what one fails to do in this area can often never be amended. Whether our work in this area of purifying our race and thus our Volk has been fruitful is something you can best judge for yourselves here during these few days. For what you are encountering in this city is the German being. Come and see for yourselves whether he has become worse under National Socialist leadership or whether he has not indeed become better. Do not gauge only the increasing number of children being born &mdash; gauge above all the appearance of our youth. <strong><em>[The writer presents evidence and explanations in a purposeful way.]</em></strong></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">How lovely are our girls and our boys, how bright is their gaze, how healthy and fresh their posture, how splendid are the bodies of the hundreds of thousands and millions who have been trained and cared for by our organizations! [<strong><em>The writer shows his arguments are valid based on personal knowledge and reflection.]<br /> </em></strong></span><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> Where are better men to be found today than those who can be seen here? It is truly the rebirth of a nation, brought about by the deliberate breeding of a new being <strong><em>[The overall effect is one of coherence.]</em></strong> </span></p>\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong><em>&mdash; (Hitler Speech 1937)</em></strong></span></p>\r\n</blockquote>\r\n</blockquote>\r\n</blockquote>\r\n</blockquote>\r\n</blockquote>\r\n</blockquote>\r\n</blockquote>\r\n</blockquote>\r\n</blockquote>\r\n</blockquote>\r\n</blockquote>\r\n</span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\"><span style=\"font-family: Arial,Helvetica,sans-serif;\"><span style=\"font-family: Arial,Helvetica,sans-serif;\"> <br /> </span></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">This piece asserts and maintains a clear position throughout and presents evidence and explanations in a purposeful way; the writer arranges reasons, examples, information and/or personal anecdotes in a discernible and effective pattern resulting in overall persuasive effect; the overall effect is one of coherence; the writer shows his arguments are valid based on prior knowledge, personal experience and reflection; the writer chooses and presents arguments with a clear awareness of reader needs; the writer evidences confidence, conviction, belief, and sometimes enthusiasm. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Clearly, for all these reasons we have to give this piece of Hitler&rsquo;s writing a Point 6 score! It meets the CLAS criteria, as does much of his writing. Hitler&rsquo;s writing was widely recognized to meet the needs of his audience, to exude confidence, etc. Is that really what we want to praise? Is that the model of persuasive writing that we want to hold up to our children? If so, shame on the California Learning Assessment System!</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">The Misassessment of Writing at the High School Level</span></h5>\r\n<h5><span style=\"font-size: small;\"><em><span style=\"color: #333333;\">Introduction &mdash; Same Problem: Next Level</span></em></span></h5>\r\n<p>At the high school level the Writing Assessment assesses four types of writing, Autobiographical Incident, Interpretation, Reflective Essay, and Speculation About Causes and Effects, some of which are successors to the elementary level Persuasive writing.</p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The same general faults that were mentioned earlier in connection with the elementary level are to be found at this level too. For example, the scoring guide for interpretation makes it clear that only subjective reactions to and subjective interpretation of fiction are really being considered. But interpretation is important in many other contexts, e.g. history, and in history it is crucial to distinguish between subjective response and objective interpretation, between reasons that persuade irrational audiences and reasons which persuade rational and fairminded persons, between something which is rhetorically convincing and something which is true. None of these distinctions is recognized in the CLAS.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">To be specific in our criticisms, consider the scoring guide for Speculation About Causes and Effects; once again writing of this kind is judged almost entirely by subjective standards, by standards appropriate to one&rsquo;s response to fiction.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Here is what is said about writing of this kind which should score Point 6, for Exceptional Achievement:</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">A six-point essay <strong>engages the reader immediately</strong>. It seems <strong>purposeful</strong>. The writer <strong>seems aware of reader&rsquo;s questions and needs throughout</strong> the essay. The essay seems to be not just written but <strong>written to particular readers</strong>. The writer <strong>convinces the readers of the plausibility</strong> of the speculation.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">A six-point essay demonstrates qualities all readers admire: conviction, enthusiasm, freshness. These essays may use an unconventional rhetorical approach. A six-point essay may <strong>take chances and succeed</strong>.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em><strong>Presenting the Situation.</strong></em> The six-point essay writer clearly defines, identifies, or describes the situation to be speculated about. Though it does not dominate the essay at the expense of speculation, the situation is nevertheless presented fully and precisely. The writer limits the occasion appropriately, focusing reader attention on just those aspects of the situation that the writer will speculate about. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Writers of six-point essays may describe or detail the situation that is established in the prompt, or they might create the situation by using narrative or anecdotal techniques. In either case, they will use <strong>concrete language, rich in sensory detail.</strong></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The writer of the six-point essay acknowledges readers&rsquo; concerns. For real world situations, the writer of the six-point essay acknowledges the reader&rsquo;s experience or familiarity with a situation and, using narrative or descriptive strategies, builds on this awareness to focus reader attention on a comparable situation. ...</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Whether the essay arises from a factual assessment of a real situation or from a fanciful guess about a fanciful situation, the writer <strong>consistently demonstrates broad knowledge and clear understanding of the situation. In this way the writer establishes authority.</strong></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em><strong>Logic and Relevance of Causes and Effects.</strong></em> In the six-point essay, the proposed causes and effects are clearly related to the particular situation that the writer has defined. Writers use imaginative, inventive argument to convince the reader of the logic of their speculation. The best writers are clearly considering possibilities and are seeing multiple perspectives.... Because speculation is essentially a persuasive type of writing, the best writers will be continually aware of readers&rsquo; needs. They might refer to the readers directly, trying to enlist their support ...</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em><strong>Elaboration of Argument</strong>.</em> The six-point essay <strong>provides substantial elaboration,</strong> convincing the reader that the writer&rsquo;s conjectures are valid for the situation. These writers elaborate their speculated causes and effects <strong>with carefully chosen evidence that is logically and fully developed.</strong> Such <strong>evidence is chosen because it is relevant and convincing.</strong> It is developed fully with precise, explicit detail to convince the reader both of the logic and the authenticity of the proposed cause and effect. <br /> <br /> Some strategies writers may use to develop their arguments are the following:</span></p>\r\n<ul>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Cite facts, opinions, projections, and personal experiences or observations (anecdotes) to explain or validate a cause or an effect.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Elaborate on possibilities arising from proposed causes and effects, showing possible &ldquo;domino effects&rdquo; that might determine the direction of the developing situation.</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">Give specific examples of comparable causes and effects that have arisen in analogous situations. (p. III&ndash;64)</span> </li>\r\n</ul>\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\"><span style=\"font-family: Arial,Helvetica,sans-serif;\">How may we best make the point that these criteria again fail to recognize the crucial distinctions of which we have been speaking &mdash; between subjective responses and good reasons, etc? Perhaps the simplest way is to look at another example of persuasive writing, this time Speculating About Causes and Effects, and consider how we should grade it according to the CLAS criteria.</span></p>\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<table border=\"2\" cellspacing=\"5\" cellpadding=\"1\" width=\"85%\">\r\n<tbody>\r\n<tr>\r\n<td>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><br /> Just as every people, as a basic tendency of all its earthly actions possesses a mania for self-preservation as its driving force, likewise is it exactly so with Jewry too. Only here, in accord with their basically different dispositions, the struggle for existence of Aryan peoples and Jewry is also different in its forms. The foundation of the Aryan struggle for existence is the soil, which he cultivates and which provides the general basis for an economy satisfying primarily its own needs within its own orbit through the productive forces of its own people.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Because of the lack of productive capacities of its own the Jewish people cannot carry out the construction of a state, viewed in a territorial sense, but as a support of its own existence it needs the work and creative activities of other nations. Thus the existence of the Jew himself becomes a parasitical one within the lives of other people. Hence the ultimate goal of the Jewish struggle for existence is the enslavement of productively active peoples. In order to achieve this goal, which in reality has represented Jewry&rsquo;s struggle for existence at all times, the Jew makes use of all weapons that are in keeping with the whole complex of his character.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Therefore in domestic politics within the individual nations he fights first for equal rights and later for super-rights. The characteristics of cunning, intelligence, astuteness, knavery, dissimulation, etc, rooted in the character of his folkdom, serve him as weapons thereto.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">They are as much stratagems in his war of survival as those of other peoples in combat.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">In foreign policy he tries to bring nations into a state of unrest, to divert them from their true interests, and to plunge them into reciprocal wars and in this way gradually rise to mastery over them with the help of the power of money and propaganda.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">His ultimate goal is the denationalization, the promiscuous bastardization of other peoples, the lowering of the racial level of the highest peoples as well as the domination of this racial mish-mash through the extirpation of the folkish intelligentsia and its replacement by the members of his own people.</span></p>\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\"><span style=\"font-family: Arial,Helvetica,sans-serif;\">The end of the Jewish world struggle therefore will always be a bloody Bolshevization. In truth this means the destruction of all the intellectual upper classes linked to their peoples so that he can rise to become the master of a mankind become leaderless. &mdash; (Hitler&rsquo;s Secret Book, pp. 212-213)<br /> </span></p>\r\n</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n</blockquote>\r\n<h5><em><span style=\"color: #333333;\">Hitler&rsquo;s Assessment Based on CAP Criteria</span></em></h5>\r\n<ul>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">This piece of writing certainly engages the reader immediately;</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">it seems purposeful; </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the writer seems aware of reader&rsquo;s questions and needs throughout; </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the piece seems not just to be written but written to particular <br /> readers; </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the writer convinced those readers of the plausibility of the <br /> speculation; </span></li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the writing shows conviction and enthusiasm;</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the writer clearly defines the situation to be speculated about, acknowledges the reader&rsquo;s experience or familiarity with the situation, and consistently demonstrates broad knowledge and clear understanding of the situation;</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the writer uses imaginative, inventive argument to convince the reader of the logic of his speculation;</span> </li>\r\n<li><span style=\"font-family: Arial,Helvetica,sans-serif;\">the writer elaborates on possibilities arising from the proposed causes and effects, showing possible &ldquo;domino effects&rdquo; that might determine the direction of the developing situation; etc., etc.</span> </li>\r\n</ul>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Can there be any doubt that Hitler&rsquo;s writing in the category merits a Point 6 grade?! If that is so then once again, shame on the California Learning Assessment System, which has again failed to see the difference between a proper and improper use of rhetoric and reason.</span></p>\r\n<h5><em><span style=\"color: #333333;\">Summary Judgment on the California Assessment of Reading &amp; Writing</span></em></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">An assessment of reading and writing should not only underscore the role of reasoning in both, but also firmly establish defensible intellectual standards, appropriately and specifically explained and consistently and appropriately applied.</span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; No assessment of intellectual work, nor foundation for teaching, should be based on an approach in which intellectual standards are confused and erroneous, confusing recall with knowledge, subjective preference with reasoned judgment, irrational with rational persuasion.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">No assessment of intellectual work, nor foundation for teaching, should be based on an approach in which intellectual standards are confused and erroneous, confusing recall with knowledge, subjective preference with reasoned judgment, irrational with rational persuasion. No assessment of intellectual work should use its key terms vaguely or oscillate between two significant uses of a term or score in an arbitrary manner. And, most important of all, no intellectual assessment should encourage irrational subjectivism.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">For example, it is striking that in the context of reading and listening, there is little discussion of the need to create an accurate interpretation: there are many contexts in which it is not appropriate for the readers to &ldquo;create their own meaning&rdquo; and where accuracy is what is required. This may be equally true if the author is addressing a particular question or problem, or using particular basic concepts; very often the good response to what is said or written is good precisely because it is based on an accurate construal of the text. Nothing in this assessment mentions the virtue of accuracy.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Most importantly however, this approach is flawed again and again because what gets credited is anything that could be construed as fulfilling one of the criteria and the criteria are the wrong ones for the purpose. If the writer is using any emotion (however discreditable) then it is marked positively; if she is giving anything that could be called evidence &mdash;&nbsp;<span style=\"font-family: Arial,Helvetica,sans-serif;\">even if it is bad evidence &mdash;&nbsp;it is credited; any kind of a move that would work with an audience, in the view of the reader is credited as good (regardless of how irrational it might be). </span></span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; Underneath all of this is a question of values. We are obliged to educate our students, not simply to shape them.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The end result is that if a student wrote a very rational appeal, one that would persuade a rational audience, it is going to be graded down because it wouldn&rsquo;t appeal to an irrational judge (because it won&rsquo;t be maintaining a strong line, it won&rsquo;t necessarily be persuasive; it will be putting in qualifications; it will be speaking in terms of greys and greys don&rsquo;t persuade; it will not be engaging in hyperbole and hyperbole is effective; it won&rsquo;t be trying to negate everything about the other side, it will be recognizing reasonable objections; it will express the degree of confidence that is appropriate but no more). Given a list of what a rational person would do, you will be able to see that the criteria for success under CLAS are negating these rational qualities and therefore encouraging irrational beliefs about how you communicate to people, indeed encouraging people to become manipulators!</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><span style=\"color: #000000;\">Underneath all of this is a question of values. We are obliged to educate our students, not simply to shape them.</span> </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The educated person is reasonable; the educated person isn&rsquo;t simply concerned with winning. The educated person wants to win when winning is the appropriate thing. However, when the other side is more reasonable, then the educated person, who is more interested in getting at the truth than in winning, will want to make concessions. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">If the goal of education is simply to enable people to get what they want, then we should teach them tools of manipulation, ways to win battles, ways to undermine positions whether those positions are rational or not, ways to just get what they want, irrespective of fairness, and of the evidence. Inadvertently and unknowingly, this is what CLAS is supporting. CLAS does not have in mind a clear difference between the educated, fairminded, and rational person, on the one hand, and the person who is simply good at manipulating, winning, and defeating others, on the other.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">In effect what CLAS has said is that the name of the game is to persuade the audience by whatever methods work and we&rsquo;ll credit anything that works. Shouldn&rsquo;t they instead have said, &ldquo;Since we are obliged and committed to educating children, and since this requires they learn to reason well as readers, writers, and thinkers, we will not credit flawed reasoning. We will only credit well-reasoned responses.&rdquo;</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">To Summarize Some of Our Criticisms of the CLAS Approach</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>1)</strong> The overall conception fails to capture the practices of critical readers, writers, and thinkers the world over. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>2) </strong>The overall conception does not call attention to definite and clear intellectual standards, and without them, it becomes impossible for both teacher and student to engage in &ldquo;objective&rdquo; assessment.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>3)</strong> The overall conception does not provide an organized and systematic approach to posing, analyzing, and reasoning through problems embedded in everyday personal and professional reading and writing.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>4)</strong> The teacher who takes this approach seriously will misteach reading and writing.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><strong>5)</strong> The student who learns through this approach will mislearn the art of reading and writing. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Given these failings, it is clear that the California Learning Assessment System falls into the category of pseudo critical thinking, and will not help students and teachers to develop their critical thinking abilities, but will hinder this process. Indeed, when classroom teachers receive copies, as they surely will, of test items, sample answers, commentary, and scores, they will use them as a guide for instruction. Thousands of school children will lose an opportunity to begin to become critical readers and writers. Thousands of school children will themselves learn to confuse recall with knowledge, subjective preference with reasoned judgment, irrational with rational persuasion. They will learn to use language vaguely and to think that their subjective pronouncements are not to be criticized. Their reasoning skills will remain abysmally low.</span></p>\r\n<h4><span style=\"color: #333333;\"><em>Is The California Assessment Fiasco a Fluke? <br /> The Educational Bureaucracy and Self-Deception</em></span></h4>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">One of the most significant facts about the California language arts test fiasco is that it is not a fluke. But of course, neither is it a plot to undermine education. The situation is worse than either. As a fluke it could be corrected. As a plot the perpetrators could be severely dealt with when exposed. No, the pathetic side of the case is that there are systemic reasons why educational bureaucracies, framed as they are, will continue to generate just such fiascos regularly and predictably. And predictably, many will be taken in. Furthermore, because classroom teachers have emerged from a long-term training that reflects a similar background to that of the test designers, they also will fall easily into line with the flawed thinking passed down to them. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">To deal with the problem at its roots, we must own the fact that there are significant problems in education due to its wide-spread and large-scale bureaucratization. </span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; One of the most significant facts about the California language arts test fiasco is that it is not a fluke. But of course, neither is it a plot to undermine education. The situation is worse than either.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Large-scale bureaucratization entails, or at least makes highly probable, a high degree of narrow specialization &mdash; and specialization tends to bring fragmentation, narrowness of vision, politicization, and self-deception in its wake. The fragmentation and narrowness of vision makes it difficult to effect fundamental changes because the parts do not work together in a rational way and no one sees clearly that this is so, since each element in the structure becomes an end in itself, to itself. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">It is almost impossible for the most pressing problems of education to become &ldquo;issues&rdquo; in educational bureaucracies because the focus is inevitably on the political, the narrow, the fragmented part or parts. With each part serving itself as an ultimate end &mdash; including those on the top &mdash; the whole is left to take care of itself. No one is left responsible for it.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The executive wing is also focused on itself and typically is satisfied with or driven to manufacture an illusion of serving the announced or official goals and ends. Meanwhile, the politicization and self-deception helps hide those realities most unpleasant to think about, and to have to face, and consequently those realities most in need of change.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">This includes, of course, the most significant one today: the fact that modern American bureaucratic schooling is a system that preserves at its heart a mode of instruction that is a hold-over from the 19th Century and whose consistent effect is a superficial one. Most students in most classes most of the time are not actively engaged in learning what is worth learning. Most students are, on their side, not taking their education seriously. On the teaching side, they are not given challenging instruction. They are not engaged in genuine intellectual work. They are not developing intellectual standards or discipline. And, most assuredly, they are not learning to reason scientifically, mathematically, geographically, economically, sociologically, or morally. (See &ldquo;Research Findings,&rdquo; p. 19.)</span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; On the shoddy foundation of didactic instruction and passive, lower order learning, the rhetoric of high goals and ideals, the propaganda of the schools, is overlaid.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Modern educational bureaucracy has developed multiple ways to appear to be, and to appear to be doing, what it is not. In the K&ndash;12 domain especially, the history of education in the 20th Century is a triumph of propaganda and self-deception. This is documented in story after story of wave after wave of pseudo reform following pseudo reform, of new buzz words and new jargon replacing old buzz words and old jargon &mdash; each set of new words serving as a new mask to obscure the one-and-the-same consistent lower order face. (Cf &ldquo;Critical Thinking in Historical Perspective chapter.) Unfortunately, virtually everyone in the game has a stake in making their playing of it look more honorable, more lofty, more noble and effective than it really is. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">You may remember that it is only some three years now since every state in the union, through the massaging and manipulation of statistics by its own state department of education, proudly announced that its students had scored above the national average!!! This is the kind of self-serving propaganda and trickery that is the daily fare of educational reality.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">State departments of education, as I have suggested, are a particularly interesting manifestation of the workings of the educational establishment. Each consists of huge bureaucracies, interlaced with committees that are in turn tied into networks of teachers and administrators spread across their states. The microcosm we analyzed in this chapter (the California State Department of Education&rsquo;s new English-Language Arts Assessment materials) is still officially in draft stage, but already is being highly touted as a refined, future-oriented, testing instrument. We&rsquo;ve seen what the ELAA has done. Let&rsquo;s see what CLAS and the Department of Education says it has done. The test is being represented to the public and to teachers within the state and the nation as having,</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The goal ... to evaluate students&rsquo; capacities for insightful, productive thinking with tests that support the finest curriculum and instructional programs in the language arts ... [California&rsquo;s Learning Assessment System, CDE Publication]</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The most important single component of the new assessment system will be the statewide performance standards, and the most important outcome of the assessment process will be the internalization of those standards in the thinking and work of teachers, students, and parents. The performance standards will undergird all aspects of the educational enterprise; serving as the center of the seamless web of teaching, learning and assessment. [Some Principles and Beliefs about the Role of Assessment in California's School Reform Plan, February 15, 1993, CDE]</span></p>\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\"><span style=\"font-family: Arial,Helvetica,sans-serif;\"><span style=\"color: #0044aa;\"><span style=\"color: #000000;\">Let&rsquo;s see what the legislature mandated, Senate Bill 662 (Hart)</span></span></span></p>\r\n<blockquote style=\"margin-right: 0px;\" dir=\"ltr\">\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\">Develop a system for producing <strong>valid, reliable</strong> individual scores and to develop and implement common statewide performance standards of student achievement as a basis for reporting all test results and setting targets for improvement.</p>\r\n</blockquote>\r\n<p style=\"margin-right: 0px;\" dir=\"ltr\">It sounded simple enough but it provided us with a classic model of pseudo critical thinking in the educational establishment. The manner in which it is structured provides a textbook case. By mirrors, illusion, and standards self-deception, it creates the</p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; The result is that the most fundamental problem in education today &mdash; that students are not learning to reason well &mdash; is not only ignored, it is intensified.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">appearance of substantial change and reform. In fact, nothing is really being changed. The result is that the most fundamental problem in education today &mdash; that students are not learning to reason well &mdash; is not only <span style=\"font-size: small;\">ignored, it is intensified. </span></span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">Fragmentation and Vested Interests</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">There are a number of reasons why it is unlikely that fundamental reforms will be effected by state departments of education or that this critique, by itself, will bring about fundamental change. In the first place, most of the positions within the state departments of education are for specialists, for example, positions for those specializing in nutrition, for those specializing in transportation, for those specializing in the laws regarding education, for those specializing in learning disorders, for those specializing in a and b and c and d and e and f and g. Virtually no one, however, has a responsibility directly connected to the fundamental goals of education (except possibly the director of instruction). Each specialist has his or her own special interest to focus on and a special group of stake-holders to represent. </span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">When there is a need to develop an assessment instrument, like the one we examined of the California Department of Education, a large group of teachers and administrators from around the state are appointed. For example, there are 33 members of the CAP English-Language Arts committee. In addition to the main committee there is a supporting committee (&ldquo;Reserve Team&rdquo;) consisting of 46 additional members. According to the California State Department: &ldquo;These development team teachers have been responsible for shaping the test format, developing prompts for the assessment, and constructing scoring rubrics.&rdquo; (p. I-2)<br /> <br /> Many political considerations go into the selection of the members of the development teams; most of the members, for example, are not scholars with publications that could be used as the basis of selection. There is an effort made to balance the committee by region, gender, race, and ethnicity. In addition, many members will have personal agendas to advance. </span></p>\r\n<table border=\"0\" cellspacing=\"1\" cellpadding=\"1\" width=\"66%\">\r\n<tbody>\r\n<tr>\r\n<td><span style=\"font-family: Arial,Helvetica,sans-serif;\"><em>&nbsp;&nbsp; The test becomes, then, both invalid and unreliable. In fact, it becomes a hot-bed of pseudo critical thinking, with a variety of misconceptions and flaws emerging. But while its intellectual value is low, its political value is high.</em></span></td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">There are usually three &ldquo;Consultants/Advisors&rdquo; selected from universities to <span style=\"font-family: Arial,Helvetica,sans-serif;\">bring in the over-arching theoretical framework. These consultants are usually the pipeline to the latest buzz words and to the theory behind them. The consultants concede to each other the right to get their favored terms into the language of the test materials.</span></span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">The result of this process, as we have seen, is that the official &ldquo;standards&rdquo; embodied in the test become extraordinarily numerous. Many of them remain vague and ill-defined. Others take on a dangerous ambiguity. The diverse criteria and the open-ended nature of the directions combined with the ill-defined nature of the terms, opens the way to arbitrary and inconsistent grading of student responses. The test becomes, then, both invalid and unreliable. In fact, it becomes a hot-bed of pseudo critical thinking, with a variety of misconceptions and flaws emerging. But while its intellectual value is low, its political value is high. The various political interests around the state are served. The media has a simplistic event to cover. Parents can delight in the fact that the scores will go up. (How can they go down when anything can count as a good answer?) The politicians will gain because they can speak of their state as in the vanguard. And so it goes. A new pseudo reform is put in place and the educational bureaucracy grinds on until the next wave of public criticism requires it to generate a new and fresh illusion of change, a new catalogue of counterfeit, bogus, and superficial &ldquo;reforms.&rdquo; </span></p>\r\n<h4><em><span style=\"color: #666666;\">So What Can We Do? Recommendations</span></em></h4>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">There is a pressing need to develop networks of educators, parents, politicians, and business people who see the need for truly fundamental reform. That reform must be advanced simultaneously on many levels, for it is not going to result from action on one level alone. Because it must go to the roots of things, because it must be substantial, because it involves deep understandings, it must be incremental, evolutionary, and long-term. Everyone with the insight to see the problem comprehensively should act within the sphere of his or her greatest influence. There is a role for everyone concerned to exercise influence for the better: for parents, for public citizens, for business people, for civic leaders, for superintendents, for teachers, for college professors, and ... yes, even for those in state educational bureaucracies. Let us consider each briefly in turn.</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">What Can Parents Do?</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Insightful parents can make the case for an emphasis on intellectual discipline and reasoning in the school curriculum. They can ask whether there is any long-term in-service in critical thinking and reasoning. They can ask what intellectual standards the students are being taught and how they are being taught them. They can make the case to other parents. They can write letters to the local papers. They can organize groups of parents who petition the school board. And most important they can develop a home environment in which the reasonability and intellectual discipline of their children is fostered, in which both they and their children routinely ask and give good reasons in support of their decisions and reason together about issues of importance not only to the family but to the broader society as well.</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">What Can Citizens Do?</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Insightful public citizens can make the case for an emphasis on intellectual discipline and reasoning in the school curriculum in virtue of the need to develop voters who will help the country maintain a democratic form of government. They, too, can go to the local school board and ask whether there is any long-term in-service in critical thinking and reasoning. They, too, can make the case to parents and other citizens. They can contact civic groups. They can write letters to the local papers. They can organize groups of interested citizens to petition the school board.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"><span style=\"color: #0044aa;\"><span style=\"color: #000099; font-size: small;\"><strong>What Can Business People Do?</strong></span><br /> </span><br /> </span><span style=\"font-family: Arial,Helvetica,sans-serif;\">Insightful business people can use the respect that their success commands to exercise influence, alone or in concert with others, over educational decisions about what to teach and how to teach it. Since their success will be increasingly dependent upon their bringing critical thinking into the inner workings of their own businesses, on workers learning how to continually relearn and improve in their performances and in the systems they use, they will have ready access to models and paradigms that can be used to illuminate what should be happening in the classroom. Increasingly, cutting-edge businesses are moving away from an emphasis on hierarchy to an emphasis on group problem solving. Since critical thinking is essential to effective group problem solving, progressive business people will be able to talk intelligibly with educators and other citizens about how problem solving structures function in business and how parallel classroom problem solving groups might be set up. And, certainly, there are any number of civic groups that business people with insight might address on the problem of educational reform, putting emphasis, of course, on the missing foundation: the failure of teachers to learn how to think critically themselves and to teach for that thinking in their instruction, the failure to focus education, in other words, on &ldquo;carefully-reasoned&rdquo; problem solving. Finally, insightful business people can form alliances with insightful educators, to create symbiotic, reflective, mutually useful dialogues on what each group can learn from the other and how each can profit by working together.</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">What Can Civic Leaders Do?</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Insightful civic leaders can draw public attention to the need for intellectual discipline and reasoning in instruction. They can articulate publicly the key links to developing responsible citizens, moral persons, and workers on the cutting edge of development. They can use their access to a more public forum by focusing the discussion of educational reform on the historical problem of the educational bureaucracy and its tendency to generate pseudo reform. They can create a public awareness of the importance of reasoning, critical thinking, and problem solving. They can help organize civic groups. They can use their superior access to other persons of leadership and influence to facilitate significant pressure on the educational bureaucracies. They can make contact with insightful and responsible politicians who are in a position to facilitate appropriate legislation.</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">What Can Superintendents Do?</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Insightful superintendents can make the case for an emphasis on intellectual discipline and reasoning in the school curriculum to the school board, administration members, teachers, and parents. They can ensure that there is long-term in-service in critical thinking and reasoning. They can ensure that students are being taught intellectual standards in depth. They can create incentives to teachers motivated to move in this direction. They can make the case to civic groups. Most importantly they can model reasonability and help create an atmosphere conducive to making the school a network of communities of inquiry.</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">What Can Teachers Do?</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Insightful teachers can make the case for an emphasis on intellectual discipline and reasoning in the school curriculum. They can request and help design long-term in-service in critical thinking and reasoning. They can bring intellectual standards into the classroom. They can make the case to parents. They can work with other teachers to foster a school environment in which reasonability and intellectual discipline are accepted school norms. Most importantly, they can routinely ask for and give good reasons in the classroom. They can foster student reasoning in history, science, math, and so forth. They can ensure that students must regularly assess their own work using intellectual standards. </span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">What Can College Professors Do?</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Insightful college professors can make the case for an emphasis on intellectual discipline and reasoning in the college curriculum. They can request and help design long-term faculty development in critical thinking and reasoning. They can bring intellectual standards into the classroom. They can do research on the significance of critical thinking and reasoning in their discipline. They can work with schools and departments of education to ensure that those studying to become teachers take classes that require reasoning and disciplined thought. They can articulate the need for prospective teachers to learn how to design assignments that require reasoning and critical thinking. Most importantly, they can routinely foster reasoning in their own classrooms and ensure that their students must regularly assess their own work using intellectual standards. </span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">What Can Those in State-wide Bureaucracies Do?</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Insightful members of state-wide bureaucracies (who recognize the systemic ways that educational bureaucracies have fostered pseudo reforms and constructed ill-designed assessments) can play a number of significant roles. They can inform themselves and others they work with of the fundamental changes that are being made in businesses adopting structures contrary to those of traditional bureaucratic organization. They can foster movement toward problem-solving teams. They can raise broader and deeper issues. They can recommend hiring people with broader vision and more developed reasoning abilities. They can help to work against narrow specialization. At the same time, they can argue for more appropriate use of experts, so that those who lack expertise in a subject will not become, for example, principal designers in tests or assessment instruments in that subject. They can argue for the construction of assessment instruments that assess reasoning in every subject area and so help to integrate emphases across subject areas.</span></p>\r\n<h5><span style=\"color: #000099; font-size: small;\">Caveat</span></h5>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Doubtless you noticed my emphasis on &ldquo;insightful&rdquo; in characterizing those who can make important contributions to reform. It is important to underscore the problem of pseudo reform, which emerges when well-meaning persons use their intelligence inadvertently to re-duplicate an old problem in new form, creating the illusion of change. Many persons today are unwilling to think through the implications of accelerating change and intensifying complexity. Many are subconsciously wedded to rigid ideas and a static way of thinking. Many are taken in by their own platitudes and high-sounding words. These facts guarantee that a long struggle will be required to work through the superficial and work into the substantial.</span></p>\r\n<h4><em><span style=\"color: #333333;\">Final Conclusion</span></em></h4>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">Pseudo critical thinking is more or less inevitable in the educational bureaucracies, given the way we have traditionally arranged and ordered things. This is illustrated, as we have seen, in the way the American educational establishment goes about designing assessment. Unfortunately, faulty assessment leads to faulty teaching, which leads to more faulty thinking in society, in business, in politics, and in everyday social life. The California Department of Education is a model case of American educational bureaucracy at work and the new California reading and writing assessment instrument is the typical resultant bad practice. Good thinking is now a fundamental human need. And though it will take generations to fully evolve from a society in which pseudo critical thinking is dominant to one in which sound, fairminded, ethically-informed reasoning is dominant, every step in that direction will reduce the amount of suffering and injustice that exists and increase, by degrees, human well being and quality of life. It is our intellectual and moral responsibility to make some contribution to this evolution. Though we are only at the beginnings of this evolution, the irresistible dynamic of accelerating change and intensifying complexity will eventually force it upon us. I hope we learn our lessons sooner rather than later, that the price of waste and unnecessary human misery may be as little as possible.</span></p>\r\n<h4><em><span style=\"color: #333333;\">References</span></em></h4>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\">All of the references in this chapter (unless otherwise noted) are from the <em>Samplers for English Language Arts Assessment</em>, for Elementary and High School, disseminated statewide in the Spring, 1993, by the California Department of Education.</span></p>\r\n<p><span style=\"font-family: Arial,Helvetica,sans-serif;\"> </span></p>\r\n<p><br /> <a href=\"#top\"><strong>Back&nbsp;to top</strong></a></p>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>\r\n<p><br style=\"clear: both;\" /></p>","public_access":"1","public_downloads":"1","sku":"","files":[],"images":[]}