"Inequality at birth is neither just nor unjust. What's just and unjust is the way institutions deal with it" - John Rawls

"I always had a certain dislike for general principles and abstract prescriptions. I think it's necessary to have an "empirical lantern" or a "visit with the patient" before being able to understand what is wrong with him. It is crucial to understand the peculiarity, the specificity, and also the unusual aspects of the case" - Albert O. Hirschman

Pages

Need for learning assessments and five aspects that need attention

In the last three posts (here, here and here), we discussed the negative effects of over paced curriculum, need to teach to the level of the kid. If teachers have to teach to the level of the kid, they should be first able to know the level of the kid and their difficulties. How do they know them? May be through an exam. The technical word used for this is called assessment.

This post discusses four aspects of assessments that need attention.

1) Granular Diagnosis and Customized Treatment:If you go to a doctor with fever, the doctor doesn't give same medicine for fever that (s)he has given someone else. The type of fever is first diagnosed and medicines are given accordingly. But if a student doesn't understand a concept, do we do granular diagnosis like a doctor and give a customized treatment? Probably no.

The following example illustrates the nuances.

Suppose a student answers that 4.73>4.8, what could be the possible reasons?

Student might have ignored the decimal and considered it as 473 and 48 and hence answered 473>48.

Student might have first considered the number to the left of the decimal, 4 in this case. Since it is same in this case, the student might have then considered 73 and 8. Since 73 > 8, overall 4.73 > 4.8

There could be many other possible reasons, some that we know some that we don't know. Though the answer could be incorrect for many students, each of them did so due to different reasons and each of them require a different remediation, customized treatment.

It's not just the issue with students who have answered it incorrectly. Even those, who answered correctly might have done due to an incorrect reason. In this case, student might have considered 4.73 and 4.8 as 4 + 1/73 and 4 + 1/8. Since 1/8 > 1/73, 4.8 > 4.73. Thus, it requires multiple questions and probing to find out the actual difficulties. These aren't the theoretical scenarios. Researcher Kaye Stacey has documented around 15 such difficulties observed in students while learning decimals.

If teachers have to understand students' problems and remediate, such body of knowledge for all concepts is necessary which isn't currently not available or not upto the mark.. It is a public good and needs government attention.

2) Intra-school assessments: Schools usually conduct exams once in a month or quarter, which are famously called unit tests or term exams or formative assessments. Often, questions from textbook are directly asked in these exams and hence students try to memorize them. These exams are to be designed better with questions that test the actual understanding of children.

3) Framing appropriate questions: Question making is also a skill. The following example illustrates its nuances (question taken from here).

The objective of these two questions is to test the concept of simple machines - load, effort and distance from fulcrum.

Which is a better framed questioned? The one on the right, because the question on the left can be answered even without knowing the concepts of simple machines. It can be answered by just knowing the concept of equality of numbers. Data from such questions will lead to misinterpretations about students' skills.

4) Designing question papers: There can be different purposes for an assessment or exam. One, to understand the difficulties being faced by students, similar to the questions on decimals discussed above. Two, to test the overall understanding of all concepts of a particular grade level. Three, a qualification test where the objective is to just test minimum competencies. Similar to exams where everyone above a particular mark are qualified. Four, to distinguish exceptional students from the rest.

Can we use same type of questions in all these exams? No. Question papers also have constraints of number of questions, length and time. If the objective of the exam is to test the overall understanding of concepts of a grade level and if we ask multiple questions on decimal like above, there won't be space for questions from other concepts. Similarly, if the objective of the exam is to distinguish exceptional students from the rest and if all the questions in the exam are easy then most people would be able to answer them. This defeats the purpose of the test.

What type of questions are to be asked? How many can be asked? What should be the composition? How should the marks be calculated?

5) Aggregation of marks: If an exam has 50 questions, one mark each, the usual practice is to count the number of correct questions and share them as marks scored. If two students scored 35 marks, where first student answered 30 easy questions and 5 difficult questions but the second student achieved same 35 marks but by answering 35 easy questions, they aren't distinguished. It's the same 35 marks for both. Item Response Theory (IRT) is used to adjust for this where marks are aggregated giving weightage to each question based on its difficulty. It is also used for other aspects in assessments which are beyond the scope of this post.

There is an existing literature on design of question papers and aggregation of marks, which has to evolve. We need to put in efforts both to enhance this knowledge base and also incorporate them in the design of assessments to make them rigorous.But are assessments always good? In the next post, we would be discussing the criticisms on assessments.Stay tuned and do subscribe to the blog. :)