EXPERIMENTAL RESEARCH 2

If the thinking lessons can be shown to improve thinking skill, then it would be difficult to find an excuse for not using them.

No amount of philosophising about the desirability of teaching thinking can replace actual results.

We might think it is a good idea to teach thinking and might set out to do so ahead of results in the hope of developing an effective method - but that would be pioneering.

Not everyone is a pioneer, and most people prefer to look at the results first and then decide.

The experimental results do show that CoRT Thinking Lessons have an effect on thinking skill.

This is not altogether surprising, because if you set out to train a skill then that skill does generally improve.

What is surprising is that as few as four lessons (and in one case a single lesson) can have such a marked effect - but then children are very good at learning new rules for new games.

ABSTRACT TESTS

Testing thinking is extraordinarily difficult.

The natural tendency would be to use abstract tests like IQ test or tests of cognitive ability. These use abstract puzzle-type situations. The advantage of these abstract tests is that they are objective and can be standardised. They are in fact excellent tests. But tests of what?

It is perfectly possible to construct a test which gives consistent and useful discriminatory results and yet not know exactly what is being tested. The answer is that it does not matter so long as the test results correlate with some other results.

For instance, it does not matter what IQ tests measure if the results correlate well with college board results (although this may mean either that IQ tests are good or college boards are bad).

The danger, however, is that such correlation can only test the status quo. Most people involved in testing are aware of this danger. Let us take two examples to illustrate it.

A natural athlete without training may be good at running the hundred-metre dash and also at the long jump. you would probably find that if you tested performance over the hundred-metre dash this would be a good indicator of "natural" performance in the long jump.

But if you deliberately set out to train some athletes in the long jump then they might get very much better at this but their performance over the hundred-metre dash might not change at all.

Two things would happen.

The first thing is that your testing over the hundred-metre dash would no longer give a good indication of long jump ability in specially trained athletes.

The second thing is that it would be fairly useless to try to measure improvements in long jump ability by repeating the hundred-metre dash test.

In a population that had been taught no maths, an IQ test would probably correlate well with natural performance in solving some simple mathematical problems. But if one group were deliberately taught some elementary mathematics the IQ test would no longer be a useful measure of mathematical ability across the whole population. Nor would you try to measure the improvement in mathematical ability by repeating the same IQ test.

The conclusion is obvious. The correlations of abstract tests are less valid if some definite attempt has been made to develop a special skill.

Innate ability may not change even though a new skill may be acquired.

This is exactly the situation with the thinking lessons. It is very unlikely that innate IQ or so-called cognitive abilities will have been changed by the lessons even though skill in the practical performance of thinking may have been improved considerably.

OBSERVATION TESTS

If we cannot use abstract tests, then we have to use observation tests. They are less satisfactory because they are much more subjective. They consist of observations made while a student performs an actual task that involves the skill taught. All our exams are observation tests - we watch how someone performs a specific task.

It is true that if exactly the same training is given to a group then abstract tests may possibly correlate with observation tests, since the innate ability to understand and learn may affect the response to the training. Nevertheless, so many other factors come in to play that no one seems very happy about abandoning usual examinations in favour of IQ tests of ability.

People do distinguish between potential and performance - you can measure potential but you have to observe actual performance. In any case, this trainability argument is irrelevant when only a fraction of the population has been given the training (as in thinking lessons) because tests of innate ability would no longer correlate across the population with amount of skill learned.

The tests used in assessing the effect of the thinking lessons are observation tests - that is to say, observing how students use their thinking skills.

Paradoxically, thinking is easier to test in this way than many other subjects because control groups can be used.

Since geography is a knowledge subject it would hardly make sense to compare a group which had studied geography with a control group which had not.

But you can do this with thinking since thinking is part of everyone's life and is a skill rather than a knowledge subject.

BATTING AND PITCHING

Sometimes we train people to pitch and then try and measure the effect of our training by seeing how many runs they score when batting.

In other words we try to test an ability quite different from that which has been trained.

A rather difficult dilemma arises here. If you test exactly what has been trained, then you cannot tell whether that skill can be transferred to different situations. For instance, students can he trained to tackle the sort of item they might find in an IQ test, but this does not mean that their general intelligence has been improved.

On the other hand, if you test a skill that has not been trained you would not expect any improvement, and you are back in the batting and pitching situation.

In the thinking lessons, what is tested is the student's ability to think about subjects they have not met before in the lessons.

Clearly "thinking" is involved in the thinking lessons, so this is the same, but in the tests they have to transfer this skill to new situations.

GENERAL IMPROVEMENT

Experiments with actual figures are shown later on in this section. There are certain types of improvement - possible the more important types - that cannot be reduced to figures.

For instance, many teachers report that the students doing the thinking lessons show great improvement in their written essays.

This cannot easily be quantified, and it may also be subjective. Nevertheless, since this is one of the few open-thinking areas in the curriculum, an improvement here is not unexpected.

In their discussions (tape-recorded), groups who have done the CoRT Thinking Lessons tend to differ from those who have not in the following general ways, in addition to more specific differences:

• No long gaps or pauses

• No wandering off into irrelevancies when a remark becomes the starting point for a different subject of discussion.

• Less giggling and whispering

• More listening to other people and less talking over people

• More tolerance less abuse and less shouting ( or voting ) down of someone with a different point of view

• Less tendency to attack the question itself as being ridiculous or preposterous – a willingness to think about any subject

• Use of thinking as an exploration instead of just to support or defend a particular point of view

• Use of thinking modes other than the purely critical mode

• Less egocentricity

• Knowledge of what to do instead of just waiting for an idea to arrive from somewhere.

Sometimes the difference between CoRT groups and control groups is quite striking.

It has been said that this general improvement in ability to think about a situation and discuss it is similar to the effect of a good home environment: the students who have done the thinking lessons behave more like students from a good home environment (where discussion may be frequent).

This is no more than a suggestion, but if it is true, then it is remarkable that a few hours of thinking lessons can help to make up for years of an adverse home environment.

It would also be highly significant since few other subjects have been able to redress this disadvantage.

MORE ABLE STUDENTS

The general improvements indicated above are less obvious with the more able students because their thinking is more fluent and more articulate to begin with.

This does not mean it is better.

Closer examination shows that the thinking of more able students may be very restricted and egocentric although expressed in a fluent manner. In fact, they may be even more restricted than others to the critical mode. They also tend to use thinking not to explore a situation but to support a pre-formed viewpoint.

Able students often make a presence of considering the other side of a question, but this is not a genuine attempt to do so.

They toss in one or two points to give an appearance of fairness but make no attempt to do these points justice.

EXAMPLE; A tape-recorded discussion of a group of elementary school students who had done ten CoRT Thinking Lessons was compared with a discussion by a control group from the same school. The subject was: "Should school children be paid wages?"

The first group had not done any thinking lessons.

The second group had done ten CoRT Thinking Lessons.

All children were ages 9 and 10. It is necessary to listen to the actual tape recording to appreciate the difference between the groups - shouting and shouting down cannot be seen in any transcripts. (We are in the process of putting the transcript on-line.)

What was remarkable was the way the control group immediately argued that if teachers got paid so should students.

Later the session turned into an airing of grievances like wanting longer vacations, or being able to chew gum in school.

The CoRT group wondered where the money was coming from, the effect of increased taxation, etc.

The recording lasted for 10 minutes in each case.

The extracted comments do not cover the details but only the dominant themes and are, as far as possible, in the children's own words.

In the first group there was a great deal of repetition, which is why the edited comments are shorter.

An extract of the main points follows:

Group 1.

If teachers get paid why shouldn't we?

I don't see why teachers get as much vacation as we do.

We're the ones that do all the work.

I don't see what teachers are moaning about.

We should have three weeks vacation after every three weeks of school.

I am going to save up for the holidays.

I am going to spend money on sweets.

I think we should be allowed to chew gum in school.

We should have free school lunches.

Anyway they cost too much.

Say yes - lets all vote that we should get paid.

We want money ( shouting ).

Group 2.

Why should we get paid?

We are learning something that is going to help us get paid when we grow up.

Anyway, children don't appreciate money very well, they'll spend it on sweets.

Look, the teachers are teaching and we are getting paid while they are teaching.

Where are they going to get all the money from?

It can't keep coming out of taxpayers' pockets. And if the school needed to get a bus, the money could have gone to the bus, instead of being wasted on us.

Only a few schools would try it and everyone would crowd these schools and they couldn't teach properly.

The first, second, third and fourth grades would argue over who should get the most money.

If taxes go up or wages go down it wouldn't be fair to a person who doesn't have any children to be paid.

Only people who work should get paid.

Now, if you want to get paid, your parents, when you first come to school, should pay about 200 to 500 Pounds which will pay your wages.

There are some private schools where they pay and if we start paying it will bring stress upon the economy.

It's a bad idea right from the beginning, really.

EXPERIMENTS

The purpose of the first ten CoRT Thinking Lessons was to widen the field of thinking so that, instead of looking at things in a narrow way students would be able to consider more aspects of a situation.

For this reason, the unit of measurement in the following experiments is the number of different aspects of the situation considered.

Wildly irrelevant aspects which have nothing to do with the situation are not included.

In this type of measurement there is a subjective element in deciding whether various aspects should be treated as different or lumped together under one heading.

In this respect, it is perhaps significant that in experiment 2b the number of egocentric points considered is slightly in favour of the control group, which suggests that no consistent bias was operating in the research worker.

Control groups were from parallel classes in the same school and of the same age and ability.

In crossover experiments, one-half of the class tackled the problem before doing any lessons and the other half afterwards.

In no case had the problem assigned been used during the thinking lessons.

In the experiments reported here, the output was verbal and was recorded on a tape recorder for subsequent analysis.

In other experiments the output was in an essay form.

In experiment 2, group number 6 was impatient to get out into the playground to enjoy the rest of the break.

EXPERIMENT 2 (a).

COMMUNITY COLLEGE

Three groups from a mixed ability class (ages 12 and up) that had done 12 CoRT Thinking Lessons were compared with three groups from a comparable class that had not done any of the lessons.

Each group of four students was asked to discuss the following question:

“In order to make better use of scarce educational resources - money for education - the following suggestions have been made:

1. Schools in country areas should be closed or,

2. That all schools should have fewer teachers.What do you think?”

Details

EXPERIMENT 2 (b).

HIGH SCHOOL

Six groups of entry level students (six students in each group) were drawn from three different mixed ability classes.

Three of the groups tackled the question before any CoRT Thinking Lessons, and the other three groups tackled the same question after doing four CoRT Thinking Lessons.

The groups discussed the question:

“Should children be allowed to leave school as soon as they have learned to read and write?”

Details:

This xperiment is particularly significant because it shows the widening effect so clearly. Moreover it shows this effect after only four lessons.

In this experiment there was not much difference in the number of egocentric points considered. In fact there was a slight drop on the CoRT side since they had to fit in so many other points in their discussion.

But when it came to considering the general effects on society, the view of the CoRT group was very much wider than that of the control group (18:1).

It may be argued that the CoRT Thinking Lessons increase the ability to think in groups but not necessarily individual thinking ability.

EXPERIMENT 2 (c).

COMMUNITY COLLEGE

Twenty Students (aged 12+) were asked to answer the questions listed below in a direct, tape-recorded interview with the research worker.

Ten of the students had done ten CoRT Thinking Lessons and the others had not done any.

The students were drawn from mixed ability classes and chosen to represent different abilities.

The questions were as follows:

a. If your parents were thinking of moving away and asked what you thought, what do you think you would say?

b. Should children be allowed to do as they like at home?

c. Should television last only 2 or 3 hours per day as in Norway, where it doesn't usually start before 8 p.m.?

d. Do you think that children should be allowed to choose which subjects they study at school?

e. Do you think it is right to raise the minimum age for quitting school to 18?

f. If you were a principal, how would you choose a new teacher?

g. What do you think people would say if you reported a student for beating up another student?

h. If your class wanted to take a trip to the seashore and the school secretary didn't have time to arrange it, what would you do?

i. If you were offered 2 vacation jobs, one in a shop and quite well paid, the other delivering newspapers, and not so well paid, which would you choose?

j. A friend of yours has stolen something, but you are accused. What can you do?

The following table shows the number of relevant points in the students' answers to each question.

This experiment 2c shows an improvement in the thinking of individuals.

All but one from the CoRT group exceeded the average for the control group, but not one from the control group exceeded the average of the CoRT group.

This is especially interesting since students of different ability were chosen for each group.

COMMENTS ON EXPERIMENTS

Group results are always more striking because groups maximise the effects of training. A single person who has responded well to the training can improve the output from the whole group.

Finally, it may be argued that the results obtained are due to giving students a definite opportunity to think and not necessarily to the nature of the CoRT Thinking Lessons themselves.

This may indeed be so, but the fact remains that the lessons have provided this opportunity.

Water does not have to take on the taste or permanent shape of the bucket in which it is carried - but without the bucket you could not carry the water.

Similarly the CoRT Thinking Lessons are a vehicle for developing the skill of thinking. The results do not show how or why they work - simply that there does seem to be an improvement in thinking skill after their use.