How Do We, Could We and Should We Assess in Mathematics?

Pamela Docherty Here in the School of Mathematics, at the University of Edinburgh, we recently welcomed Adrian Simpson of Durham University to our Learning and Teaching Colloquium. His topic was one that resonated with many of us in Mathematics. Provocatively entitled “How Do We, Could We and Should We Assess in Mathematics?” his presentation was based on a long-term project with Paola Iannone of the University of East Anglia. As the title suggests, there were three main components to the work:

How do we assess in Mathematics, then? Perhaps unsurprisingly, their research found that the vast majority of assessment in Mathematics in England and Wales consists of closed-book examinations. Depending on the institution and course choice, a Mathematics student can expect between 50-90% of their final degree classification to depend on closed-book examinations. The remaining assessment may consist of continuous assessment (it is extremely common in Mathematics for continuous assessment to consist of short weekly assignments rather than a few large pieces of coursework over the Semester), project and presentations.

Simpson singled out one assessment practice in particular which seems to be entirely absent from Mathematics undergraduate assessment regimes in the UK, although it is common in other European countries: the oral exam. This was not always the case. The oral examination seems to have been standard practice in the UK until the beginning of the 20th century. According to Stray (Stray, 2001), it fell from favour due to accusations of bias and inefficiency compared with written exams.

As part of the project, Simpson and Iannone were interested in finding out whether an oral examination could reasonably replace a component of assessment in a first-year Maths course (Iannone & Simpson, 2012). In the literature, three types of oral assessment are distinguished – presentation, application (such as medical students being examined on their ability to take a patient’s blood pressure) and interrogation (which covers everything from a simple Q&A to the full-on viva). The case study was concerned with the latter. The study involved replacing one assignment (in a course which followed the continuous assessment structure of weekly problem sheets) with an oral examination. The students were asked to prepare four questions from a problem sheet as usual, but rather than write them up and hand them in, they would be asked to present two of them (one being the student’s choice and one picked at random by the researchers from the remaining three). It was emphasised to the students that the exercise was not intended to catch them out but to help them express what they understand. To relieve anxiety the oral examination was referred to as a ‘one-to-one tutorial’. Nevertheless, many students reported feeling nervous when preparing for the tutorial, although it would be interesting to compare this to the stress of a traditional exam, given that the tutorial was only ten minutes long.

Despite the reported anxiety, the students agreed that the oral examination had many more positive aspects than the weekly problem sheets. These included making them think about the material, encouraging them to understand things better and being less likely to forget the material afterwards. In addition to the perceived benefits from the students’ point of view, the work also discussed the barriers to widespread implementation of an oral exam in Mathematics – concerns about quality assurance and bias being prominent. Hearteningly, the other often-expressed concern – increased time commitment for staff – seemed to be largely unfounded, at least for their experiment. The total staff time spent on the exercise was 18 hours, compared with an average of 16 hours for marking and giving tutorials in the other weeks.

But how should we assess? Iannone and Simpson asked students for their opinion. This is sensible, as there is strong empirical evidence that student’s perceptions of the value of assessment affect the way in which they approach learning. (How many of us have witnessed students rote-learning in order to pass the exam?) The aim therefore was to investigate students’ perception of how much various assessment methods act as tests of two cognitive characteristics – memory and understanding. The assessment methods in question include closed-,open-book and oral examinations as well as multiple choice examinations, dissertations, weekly examples sheets and presentations. The full details can be found in their paper (Iannone & Simpson, 2013), but to summarise: students perceive that closed book examinations test memory best of all, then multiple choice and oral exams, with projects rated least of the scale. Oral exams and closed book exams were rated the highest as tests for understanding, whereas open book exams and multiple choice exams were rated the lowest as tests for understanding.

This was of particular interest to me as for the past three years, all pre-honours Mathematics courses in Edinburgh have been assessed by open-book examination. I was therefore a little disappointed that students (at least in this study) did not rate it highly as a test of understanding (although, in fairness, the students in question were first-year students who had not yet undertaken any formal examinations, open book or otherwise). For us, it is extremely important to write exam questions in such a way in order to test conceptual understanding, rather than surface learning (state this definition, prove this theorem…) which can easily be found in the textbook which accompanies them in the exam hall. As we tell our students, this is how real-life mathematics works – an employer would not ask you to solve a mathematical problem by sitting alone in a room with no access to materials! Anecdotally, our students do seem to share in our belief that open-book exams help them to understand the material better.

I was interested to hear that Simpson and Iannone found that students place a lot of value on the perceived ‘fairness’ of the assessment: they seem to prefer assessment practices that are, in their view, a good discriminator of mathematical ability. Unsurprisingly then, that students rate multiple choice examinations low, possibly because one can ‘get marks for free’ by guesswork.

During his presentation, Simpson emphasised his reluctance to use the word ‘innovative’ when discussing this topic, as in the literature many writers often equate this with ‘better’. In fact, this is far from clear cut. There has been much scholarly work in Mathematics Education advocating projects, presentations and other forms of assessment, but almost none exploring the advantages and disadvantages of the traditional closed-book examination. It is worth noting that there is extensive general education literature on this issue. Many writers praise ‘innovative’ practices, that is, something other than the traditional closed-book exam. However, it seems that in many studies, Mathematics students (and indeed STEM students) are under-represented in the samples.

So, the closed-book examination is still seen as the ‘gold-standard’ of assessment practices in Mathematics. Iannone and Simpson suggest that staff see it as the best compromise between quality assurance and efficiency. Students seem to rate it highly as a fair mode of assessment and one that is a good indicator of both memory and understanding.
I am looking forward to future work further investigating how we can best serve our students with regards to how we assess them.