Richard Muckart Q&A: Making the grade

What testing takes place to check exam papers before students sit them?The questions and solutions for each examination are drafted by an experienced team of examiners. The draft paper is then attempted by a recently qualified fellow and reviewed by two staff actuaries, one of whom is the specialist for that paper. They check for out-of-syllabus questions, badly phrased questions and for too much being required to be done in the time. A team meeting, which includes the chairman or deputy chairman of the Board of Examiners and the specialist staff actuary, is then held and amendments are made or new questions drafted. The paper is independently reviewed for a second time, any comments are taken on board and the final version is produced.

Can you explain how the exam pass mark is adjusted in any one session to be consistent between exam sessions and ensure that the right students pass?We operate a double-blind marking system using volunteer markers. This means that none of the markers know which candidate’s paper they are marking nor the mark awarded by the other marker. Marking standards do vary from marker to marker, but we have a software programme that scales and adjusts marks to allow for this.

All markers and the examiners attend a meeting to agree the marking standards, adding in additional points not in the original mark schedule. Prior to the meeting, everyone marks the same five scripts to ensure consistency of approach and to give the examiners a rough idea of where the pass mark is likely to be.

All borderline scripts are third-marked and may be reviewed further by the principal examiner as a final check. Cases where there is a significant variation in the marks awarded by the two markers are also third-marked. Finally, any mitigating circumstances or other issues are considered.

The Profession’s website states: “As a matter of principle we do not publish pass marks but look at the performance of students from session to session.” What is the principle on which pass marks are not published?This means that the pass mark is not a fixed mark but depends on the leniency or strictness of the markers on aggregate. It may also vary where the paper set proves more difficult than envisaged. Publishing the actual pass mark used is likely to be unhelpful.

As a guide, however, students should be aiming to attain a mark of at least 60% in CTs and 50- 55% in CA1, STs and SAs to ensure a pass. The practical examinations, CA2 and CA3, require a mark of around 70%, similar to that required to pass coursework assignments at university.

Please explain the fail grade changes and what the new grades mean.The fail grading is there to help students understand how much extra work they may need to do to achieve a pass. In the past, in cases with a low pass mark, the FA band contained a large number of candidates.

Evidence collected from exam counselling indicated that many students with FA grades were surprised at how much more they needed to do to pass. Accordingly, the Board of Examiners felt that a narrower FA band would be appropriate to give students better information.

Following consultation with employers and students’ representatives, the new band system was introduced in April 2010. This means that a candidate achieving an FA gained a mark of 95-100% of the pass mark, an FB gained a mark of 80-95% of the pass mark, an FC 60-80% of the pass mark and an FD got less than 60% of the pass mark.

Pass rates seem to vary from exam diet to exam diet and examination to examination. Is this reasonable?Although students find it hard to believe, the Board of Examiners does not have fail quotas to achieve. Accordingly pass rates are free to vary (and do). They are determined by the quality of the candidates sitting the examination and in particular how well prepared they are. Fitness to pass is the criterion, not whether you can achieve a mark in the top 40% of candidates sitting.

The examiners’ reports vary from subject to subject, with some containing additional guidance while others only provide indicative solutions. Why?This topic was aired at a recent Board of Examiners’ meeting and we are going to try to be more consistent in providing additional guidance to students about each paper. We do not want to write comments for the sake of them and so we will focus on questions where students had difficulty answering them. Students should understand that the reports provided for the ST and SA examinations contain more points than are required to obtain full marks.

Any final tips for students as chair of the Board of Examiners?Students should ensure that they have read and understood core reading and related teaching materials, and you should have studied past papers to ensure that you can apply the principles covered.

The examiners try to lead students through questions — for example, the verb (‘list’, ‘outline’, ‘describe’, ‘evaluate’, ‘explain’) at the start of each question indicates what sort of answer is being sought, and students should follow these instructions.

It is always a good idea to answer the question that has been set, not the one you would like to be set. Finally, don’t waffle — you will not pick up extra marks for saying the same thing in more than one way.