Question setting CIEA Insight is a key support for CIEA members. Through Insight you can tap into a range of materials in one place that can: help you.

Similar presentations

Presentation on theme: "Question setting CIEA Insight is a key support for CIEA members. Through Insight you can tap into a range of materials in one place that can: help you."— Presentation transcript:

1
Question setting CIEA Insight is a key support for CIEA members. Through Insight you can tap into a range of materials in one place that can: help you develop your own knowledge and skills in specific areas of assessment help you develop your team’s knowledge and skills give you access to information that we hope you will find stimulating and informative.

2
Question setting This short CIEA Insight will give you a glimpse of some key aspects of question setting More detailed information on the subject is available to CIEA members in the members area of This Insight will cover Question types Some general points on good practice Things to avoid

3
Question setting Whether you are setting questions for assessment in your own centre, class or tutor group or involved in setting questions for public examinations, there are still good and bad approaches to the process.

4
Question setting Compare two main characters from different works of literature from different centuries. Good question?

5
Question setting Compare two main characters from different works of literature from different centuries. “There is no evidence that Lady Macbeth liked fish; Tarka the Otter loved it.” “Whereas Tarka was despatched with a spade, it was a balcony wot done for Lady Macbeth.”

6
Question setting Poorly written questions have a negative impact on the learners who are faced with them Poorly written questions also create frustration amongst assessors

12
Multiple Choice Question Stems Stems can be written in various ways Among these are: Completing a statement The capital city of Spain is... Logic and reasoning Which of the following statements can be justified using the information in the passage alone? Odd one out Each of the following is a cause of drought except...

13
Negative Stems Using the negative in a Stem can be confusing There are three things wrong with this question Which of the items in the following list is not one of the major organs of the human body? ASkin BHeart CLeg DPancreas

14
Vague and Confusing Stems Which of the items in the following list is not one of the major organs of the human body? ASkin BHeart CLeg DPancreas Some learners will focus on the word “organ” and simply pick one out The word “major” has not been defined The answer “Leg” is too obvious

15
Vague and Confusing Stems This question stem is confusing Which of these is not a good reason for the world economy to slow down in the event of a major natural disaster?

16
Vague and Confusing Stems Which of these is not a good reason for the world economy to slow down in the event of a major natural disaster? It uses the negative The main thrust of the question is separated from the opening It is difficult to word a correct answer and even harder to write distractors

17
Carefully Worded Odd One Out Questions It is valid to test the ability to recognise which items in a list stand out in some way Using the format: Each of the following is... except can be a good way around the confusing negative stem problem

18
True / False Questions True / False questions are not true multiple choice questions e.g. A basic principle of physics is that energy can neither be created nor be destroyed True or False?

19
True / False Questions A better approach would be to ask: A basic principle is that energy Acan be created but not destroyed Bcan be destroyed but not created Ccan be destroyed and created Dcannot be destroyed or created

20
True / False Questions A basic principle is that energy Acan be created but not destroyed Bcan be destroyed but not created Ccan be destroyed and created Dcannot be destroyed or created Even here the correct answer is the only one to contain the word “cannot” so it might need further revision

21
Multiple Choice Questions Summary You have now seen some of the benefits of multiple choice questions. They can: be objective be answered quickly test a breadth of knowledge be marked quickly and consistently

22
Short Answer Questions SAQs can be marked By human markers - possibly subject specialists By human markers who are not specialists On screen On line Automatically - if the list of possible outcomes is finite and easily referenced

23
Short Answer Questions Be clear in your demands Avoid ambiguity e.g. Say how glaciers move SLOWLY And the old maths classic:

24
Short Answer Questions are useful for developing points slightly further than multiple questions would allow a good way to explore a learner’s knowledge quickly easily administered and marked - but not always automatically are not Multiple Choice Questions Essay titles If the point you wish to test is ideally suited to another method then use the other method

25
Extended Answer Questions Traditional essay style questions Used to test knowledge in depth Need to be carefully worded to test what is actually required

26
Writing Extended Answer Questions e.g. Give an account of “Macbeth” This pretends to be an essay but is nothing more than a request for a candidate to write out the plot. A low order skill would actually be tested

27
Writing Extended Answer Questions e.g. Macbeth is a good man who allows himself to be tricked by the forces of evil. Discuss the above statement with reference to at least three scenes from the play. This is clearer in its demands and gives an element of structure to the task which was not present in the previous question.

28
Structuring Extended Answer Questions Having said that... Don’t turn an Extended Answer Question into nothing more than a series of SAQs (Short Answer Questions) This can be the case if the question has too many parts e.g. Give two reasons why Britain sent troops to France in 1914 and three reasons why the USA entered the war and two outcomes of the treaty of Versailles.

29
Deriving Extended Answer Questions The question is being asked in order for a candidate to demonstrate knowledge and/or skills The question should come naturally from a programme of study and not be tacked on artificially

30
Extended Answer Questions can assess knowledge in depth probe into skills should not lead to repetitive low order tasks be unconnected to a programme of study

31
Showing Off It might be interesting to set questions for the local pub quiz which test esoteric knowledge BUT Question setters should resist the temptation to show the world how clever they are by setting candidates ridiculously difficult tasks If there is a clear form of words then use it

32
Remember what you are testing The question is being asked for a clear reason It is in clear language It asks what it seems to ask Deciphering the question should not be the test

33
Teaching to the tests or testing what is taught? Whilst the role of assessment remains very important in school, college and workplace education and training, we should all remember that assessment is not an end in itself. Keep a sensible balance between teaching / training and the various forms of assessment. The CIEA hopes to engage members in wide ranging discussion about the nature and the future of assessment. You can find out more about question setting in the members’ area of the CIEA website -