Examiners

Do not prepare any examination questions or answers on any part
of the UNIX network. The network is insecure.

Do not email any examination questions or answers. The network is insecure.

Academic staff are responsible for producing their own exams.
Course unit leaders are responsible for overseeing the process.

There are 3 sets of exams during the year: semester
1, semester 2 (including full year), and resits.

All examination papers are internally moderated. A preparation pack will be
provided for each course unit. The pack includes report forms for use by
the moderators and the exam setters. All examinations and PG coursework
are sent to the external examiners for their comments. Exam setters are
required to act upon external examiner feedback and provide a formal response
via
Student Support Office.

For each set of exams you need to:

Set exam(s) and marking scheme(s)

At the beginning of each semester you will receive an exam call
by email which includes the links to vital information (shown below).

The above can, if desired, include one compulsory question.
Papers may be sectioned to ensure separation of answer books to aid
marking by multiple examiners, but this should not compromise the
choice.

Papers (and students' answers) should normally be monochrome. If
you want to use colour in your questions or any accompanying pictures
or diagrams, please check that this is not going to cause problems for
colour-blind students by contacting the departmental Disability
Support Coordinator and/or the University Disability
Support Office, and establish how the pictures etc. will be
reproduced and distributed.

Question style

Please try to set your paper to avoid an excessively high
standard deviation (over about 15%) in the students' marks. This
generally results from a paper where hard-working but weaker students
can find nothing to answer. Please set questions where weaker students
can do at least part of the question. When doing this please try to
make your question coherent and progressive, rather than a sequence of
disjoint, unrelated parts.

Please ensure the questions are not all straight bookwork. Ideally
every question will have a problem element, however trivial.

If the course-unit is new, or significantly
changed from previous years, you are expected to publish a "mock"
exam paper so students can see realistic exam questions.
(You can of course use individual questions, or parts of questions,
from exam papers for the previous version of a course-unit that has
changed.)

It is up to you whether you want to go as far as giving
model solutions for old exam questions. Referring students
to model solutions, lecture notes, or other materials,
can aid exam performance.

Be available to the students before your exam
to answer their questions

You aren't normally expected to do anything
exceptional but a prompt response to email
and other enquiries is vital.
You are likely to get as many queries in
the weeks before your exam as in the whole of the
rest of the semester.

Be available during your exam to deal with any
queries

If you need to be away from the School you must have permission from
the Head of School, have some other member of staff assume
responsibility for your exam(s), and
inform the Student Support Office.
If a query comes up during the exam you (or your deputy) may need
to go to the exam room.

Given that members of staff write their own exam papers
it is vital that you proof-read your exam carefully.
One legitimate query will
often trigger a flood of less sensible queries
that you will also have to respond to, so it is
much easier to deal with problems when setting
the exam!

Be available during the next few days after
handing in your marked exam

There are often
queries arising from the checking. If you are not
in the School, please leave contact information with
the Student Support Office.

Come to the various examiners' boards

Be available during externals' visit

Set resit papers

At the same time that 1st-year and 2nd-year course-units exam papers
are set you must also set a resit paper. Postgraduate
resit papers are set after the July board when resit students
have been identified.

If you are considering a hybrid exam, composed of both online questions and
a paper-based section, please be aware of the following from the
Examinations Office:

The practice and guidance of the exams office is that we do not
endorse combination assessments within PC Clusters and/or Labs, for the
reason that they are unsuitable spaces for the administration of paper
examinations. We have encountered a variety of problems over the years
and received several complaints with regards to the limited space available
from students. Please note that the amount of space available within
clusters is mostly inconsistent; one cluster may have more space than
another, thus allowing some students to have better exam conditions than
others. This is not a route that we recommend, nor would it be sustainable
in the long-term (i.e. without incident) as it could inhibit a student's
exam performance. That said, there is a need to promote and encourage the
development of e-assessment wherever possible. Therefore, it would be our
recommendation that the online element be sat in a PC cluster, and the
written element be sat separately in a more appropriate University approved
exam room.

If you decide to run a hybrid examination, it is vital that you contact
Jennie Ball-Foster
immediately so that she can advise Exams Office and
liaise with IT Services to ensure that the additional logistical
arrangements are factored into the scheduling.

Questions involving numerical or other symbolic calculations frequently have short answers, and the
marking scheme may require candidates to justify those answers, for example by showing their working.
In that case, the examination paper should contain the explicit instruction "Show your working"
or some appropriate reformulation thereof. Of course, candidates are expected to understand that the
onus is generally on them to convince the examiner that they have mastered the material. However,
students who simply write correct answers---particularly to short or undemanding questions---may, in
the absence of such an instruction, legitimately complain when they are not awarded full marks.

When the exam timetable is known you will receive a second email
containing your personal marking schedule. Please adhere to
the deadlines in the schedule.

About half a day after your exam, the students'
answer books will be delivered to the
Student Support Office.
The scripts will be prepared and a marking pack provided;
an email will be sent to you when they are
ready for collection.

If any student writes on their answer book about
non-academic matters (e.g. illness or family or
personal problems), please inform the
Student Support Office
and the relevant Year Tutor or Programme Director as soon as
possible.

Your marking pack will include a blank marks grid ordered in ascending
student id number. The answer books will be in the same order.

If you have any answer books which do not appear on
the list, just add them at the end (and please put
the answer books in the same order).

Negative marking is the practice of subtracting marks for incorrectly answered
parts of questions.

It is often employed in conjunction with multiple-choice questions, where candidates
could expect to obtain a significant non-zero mark by guessing. The preferred
policy of the School is not to use negative marking, unless its absence would
result in serious distortions in the overall mark. Small numbers of
marks obtained by guessing the answers to a few questions do not constitute a
serious distortion, and anyway scarcely justify the extra effort of implementing
negative marking schemes. Recent experience has shown that negative marking
generates complaints by students.

If you do decide to use negative marking on your examination
- for example if there is a significant multiple-choice component -
then the scheme should be made clear to candidates on the examination
paper. If this cannot be done succinctly, then it almost certainly requires
simplification anyway.

The preferred policy of the School is not to use negative marking, unless
its absence would result in serious distortions in the overall mark. If the
setter(s) decide to use negative marking - for example if there is a
significant multiple-choice component - then the scheme must be made clear
to candidates on the examination paper.

Most of what you might want or need to tell your students is
in the Student FAQs.

After the exams you will need to discuss their
results with them (meaning of marks, what to do next,
etc.). In particular after the 1st semester results
are published you are expected to meet for about 10
minutes with each of your students to
review how they are getting on and discuss how they
can improve their chances of achieving their personal
targets.

The main purpose of this exercise is
to make students stop and think about how they are
approaching their studies, and take more
responsibility for them. It is not just an exercise
for those students who are struggling, but also an
opportunity for the better students to reflect on
their studies and devise strategies for getting more
out of them. It is important not just to identify
problem areas, but to try to agree actions that the
students might take to improve the situation.

Make sure you understand what
information can be released, and when it is OK to do
so - this is particularly important for the June
exams. If in doubt check with the
Student Support Office.

It is the policy of the School of Computer Science not to selectively remark examination papers or coursework on request.

The School will not remark work simply because students believe that they should have obtained a better mark than that shown on their transcript.

After results are published, the School receives numerous requests from students for us to check their answer books for irregularities. Checks of the answer books in question will be done. If the checks disclose any anomalies the Director of Studies will be advised.

Examination papers and coursework will only be remarked where the school officers (Director of Studies or Examinations Officers) have good cause to believe that there has been a marking irregularity.

It is current University practice that sections of examination papers featuring
MCQs (where printed on paper) be collected in after the examination, and not
published. Nevertheless, for obvious security reasons, MCQs must be changed
every year, just like any other questions, in order to prevent the leakage of
information likely to be useful to candidates. Basic changes should go without
saying: wording of the questions and responses, values of numerical parameters
and---most obviously of all---order of responses. In addition, however, exam
setters are expected to build up a bank of questions from which a
non-predictable selection may be made each time the exam is run. As recent
experience shows, these concerns are not purely theoretical: it has to be
assumed that information about multiple choice examinations will escape, even
when question papers are supposed to be collected in.

Examinations are an integral part of teaching and must always be set and marked
by the members of staff who are allocated to teach the course unit in the School's
duties allocation. If there is any reason why you will have difficulty setting or
marking an examination in a particular year then alternative arrangements must be
agreed in advance with the Director of the Undergraduate School or the Director
of the Postgraduate School as appropriate.

The committees consider all students with illness
or other factors affecting their performance, which
will not be considered in detail in the main examiners'
board. If you hold any relevant information about
individual students (your tutees, project students
etc.), a copy should be deposited with the
student support office for the Action file as soon as
possible.

There are three exam sessions (January, May/June and August) which have corresponding sets of examination boards in February, June/July and September.

The School's Examination Officers, Year Tutors, Programme Directors and Lab Managers are detailed in the
Staff Loads.

All marks are displayed in exam grids where students are identified by ID
number rather than by name. Students, who have been considered by the
Mitigating Circumstances Committee, are flagged with an "M". Staff must ensure
that all personal issues raised by students are brought to the attention of the
Mitigating Circumstances Committees via the Student Support Office.
See above for further information if your students have
serious problems. The Year Tutors/Programme Directors will report
the Committee's recommendations and particularly where this will help the
student. The Examinations Officers will have made calculations informally
to see what effect such recommendations would have on a student's result and
report this to the Board to aid their decision-making. There may be further
voting on whether to promote a student or not.

February

There are two Marks Review Meetings: one for UG and one for PGT. Staff are not expected to attend these meetings, which are attended by UG Year Tutors. PGT Programme Directors, Examination Officers, Project Managers and Lab Managers respectively.

The meetings start by looking at the mark averages and spreads for each course unit and overall to ensure that they are reasonable or else that we understand and accept any anomalies. It may be that the meetings recommend scaling of marks for some exams, but this is not be applied until the summer boards have considered the full array of marks in order to make a final decision.

The meetings consider the semester 1 provisional marks. The purpose of the meetings is to highlight struggling students in order to provide support and to identify any issues from the assessment process. The meetings agree to publish the provisional results and minute any recommendations or actions to be undertaken in time for the summer boards.

June/July

Third Year Undergraduate

The first board is the 3rd Year Internal Single Honours Examiners Board for final year students (except for Joint Honours Maths). The board makes provisional decisions about degree classifications and prizes.

Classifications for those students on borderlines/affected by mitigating circumstances are subject to in-depth academic discussion (no personal details about mitigating circumstances are revealed) and the board will vote on whether to promote a student or not. All the provisional decisions and recommendations of the board are referred to the External Examiners for consideration during their visit; particularly those students on borderlines or where the board found difficulty in making a decision.

The next board is the 3rd Year External Examiners Board for final year single honours students. The External Examiners are present at the board, having been in School for two days considering all issues including those raised by the internal board's deliberations. The Externals will look at the work of the students involved and will provide guidance and advice to the board about how to proceed. There are instances when the External Examiners will disagree with initial decisions and further discussions can result further voting and changes to those decisions.
See below for further information about what the external examiners do.

Finally, the board will agree on the award of prizes. Note is also taken of students whose results cannot be released due to debt.

The 3rd year joint-honours Maths board is held after the single honours boards have been held in both Schools. Any adjustments to marks decided at the earlier meetings (e.g. where final-degree class borderlines have been placed) and the degree classes for 3rd-years established are made known to the joint honours board for comparison and guidance. The external examiner from Maths will be present to give advice. The Computer Science External Examiners will be consulted separately if necessary. The board follows the same process as the single honours boards.

1st and 2nd Year Undergraduate

The 1st-year and 2nd-year single honours examiners board decides passes and fails for individual course-units. We explicitly use the marks obtained by each student for each individual course-unit and automatically apply compensation for up to 40 credits of course units as outlined in the crib sheet. The Year Tutors will advise about recommendations from the Mitigating Circumstances Committee. Such students are flagged on the exam grids with an "M". The Examinations Officer will advise about the effect of applying those recommendations. The Board will discuss the impact of those changes on the final outcome for each student and make final decisions about resit requirements and whether students should sit at 1st attempt or 2nd attempt for each course unit failed. This is particularly important for 2nd year students, as the marks can affect their final degree class. Notes for such students will be carried forward for their 3rd year.

The 1st-year and 2nd-year joint honours Maths examiners board sits separately after the single honours boards for both Schools and follows the same process as the single honours boards. Decisions from the earlier single honours boards are made known to the joint honours board for comparison and guidance.

4th Year MEng/Postgraduate

The 4th Year MEng/PGT Internal examiners board decides passes and fails for individual course-units and identifies students who are in the Distinction classification and looks at all individual marks for the Progress Project Report. The board explicitly use the marks obtained by each student for each individual taught course-unit and automatically apply compensation for up to 30 credits of course units as outlined in the crib sheet. The Programme Directors will advise about recommendations from the Mitigating Circumstances Committee. Such students are flagged on the exam grids with an "M". The Examinations Officer will advise about the effect of applying those recommendations. The Board will discuss the impact of those changes on the final outcome for each student and make final decisions about resit requirements and whether students should sit at 1st attempt or 2nd attempt for each course unit failed. Students will also be identified for resubmission of the Report.

Resit Boards

The UG single honours Resit Examinations Board sits in September and considers
all students who have been flagged to do resit assessments either by exam and/or
coursework or lab. Any students affected by mitigating circumstances are
flagged with an "M" on the examination grid. The Year Tutors report on the
recommendations of the Mitigating Circumstances Commtitee and the Examinations
Officer reports on the effect of those recommendations on the student's results. Compensations still available under the rules will be applied to individual course units and final decisions made about students' progression.

The UG joint honours Maths resit examinations board sits in September and follows the same process as the single honours board.

The PG Resit Examinations Board sits in September and considers all students who have been flagged to do resit assessments either by exam and/or coursework or lab. Also students who have resubmitted their Progress Project Report are considered. Any students affected by mitigating circumstances are flagged with an "M" on the examination grid. The Programme Directors report on the recommendations of the Mitigating Circumstances Commtitee and the Examinations Officer reports on the effect of those recommendations on the student's results. Compensations still available under the rules will be applied to individual course units and final decisions made about students' progression

Dissertation Board

The PG Dissertations Board sits in late October/early November. The External Examiners are in attendance and have considered any issues arising from the dissertation submission and marking process, including any students affected by Mitigating Circumstances.

Examination grids and Dissertation results spreadsheets are tabled at the board. Any students affected by mitigating circumstances are flagged with an "M". The Programme Directors report on the latest recommendations of the Mitigating Circumstances Commtitee and the Examinations Officer reports on the effect of those recommendations on the student's results so that the board members can make final decisions about the students classification for the December graduation. Students are identified who need to resubmit dissertations.

All staff who have tutored, lectured or examined
students are required to be present at the examiners'
boards that deal with those students.
(See what happens at the examiners' boards
for details of the meetings.) In exceptional circumstances, a member of
staff may not be able to be present. However, permission to be
absent from the board(s) must be sought from the appropriate Director
of Studies. There
is then an obligation on that member of staff to find a substitute who can
speak to the former's students, courses and examination results, and who
is willing to do so, and who will be present at all relevant examiners'
boards. The Director of Studies must also be informed of the substitute.

Even if you have not had contact with a particular group of our
students, you are still welcome to attend any of our examiners' boards
and express an opinion.

The University's definition of
external examiners duties
casts them as auditors for the academic quality of our students and thus also for the process by which we assess it. In addition, we have Chief External Examiners for each of the Undergraduate and Postgraduate Schools. They adjudicate in any dispute or impasse where a final decision cannot be made by the examination boards.

All examinations and PG coursework are sent to the external examiners to
check our exam papers
and provide feedback comments. Exam setters are required to act upon external examiner feedback and provide a formal response via
Student Support Office.

The External Examiners are asked to visit the School for the summer boards.

The Undergraduate Externals visit for 2 days after the
June internal examiners board.
They mainly concern themselves with considering issues and problems raised by the internal board.

The Postgraduate Externals visit for 1 day after the
July internal examiners board.
Their focus is to consider the Progress Project Reports and any issues and problems raised by the internal board.

When External Examiners first arrive, they are briefed by the appropriate Director of Studies and Year Tutors/Programme Directors on the results of the internal examination boards. They examine answer books and project/dissertation reports of borderline and other students, and also written reports by project/dissertation supervisors. They may want to discuss a student's work with the supervisor, personal tutor, year tutor/programme director and/or any other relevant member of staff.

During the course of their visit the External Examiners will have the opportunity to meet with students.

Only once the External Examiners are happy that they have been able to consider all issues raised, the External Examiners' Board is convened. If the advertised time is delayed to allow for this process, staff will be advised by email.

Final decisions are made based on the externals' recommendation and advice. During this meeting, they present a verbal report on the examination results, standards and procedures. Finally, the External Examiners, Director of Studies and Examinations Officer sign off the results for publication.

The external examiners submit a written report at the end of the academic year (UG after the summer boards; PG after the Dissertation Board in November) which goes to the University and Faculty. The report is forwarded to the School for consideration by the appropriate Undergraduate or Postgraduate Committee and a formal response provided.

Undergraduate

Make sure marking is complete (and mark-list
etc. up-to-date!) as soon as possible after end of
semester

3rd year projects (marks returned via project
organisers)

Read and mark on time - usually before/during
exams, but please try and do this BEFORE you have
to mark any exams.

Anything else e.g. seminars, reports, essays,
mid-term tests

If you use them, mark them and tell us the
marks ASAP!

Postgraduate

The course unit leaders are responsible for producing a list of overall
coursework marks and should ensure that these are returned to the
Student Support Office
by the deadlines advertised in the examination calls.