The Learning Objectives Questionnaire (LOQ)

Introduction

The Learning Objectives Questionnaire (LOQ) was designed by Norton, Horn and Thomas (1997) as an ‘alternative’ course evaluation instrument, which is flexible and can be adapted for any course. It has been written about in my book (Norton, 2009) but colleagues are invited here to use it and adapt it for their own purposes.

Rationale

The LOQ uses a combination of closed and open-ended response questions which takes account of the extent to which students have actually understood what lecturers are trying to teach them. In this way, it differs from more traditional course evaluation instruments which can be affected by factors such as students’ expectations, study approaches, preferences and learning styles (see for example, Entwistle & Tait, 1990; Carkenord & Stephens, 1994; Sprinkle, 2008).

The LOQ was designed to minimise these effects by actually telling students what the main learning objectives are for each element of the course and then asking them to rate how successfully they think these objectives have been achieved.

The advantages of this type of approach to course evaluation are threefold:

The lecturer’s aims are clearly specified, so regardless of students’ preferred approaches, they specifically evaluate the elements that lecturers have identified as important.

Constructing the LOQ encourages lecturers to think in a more focused way about precisely what they want students to learn in their courses – an integral part of course design.

Concentrating on lecturer specified objectives, makes it easier to determine what specific modifications might need to be made to improve the effectiveness of a course.

Using the LOQ for pedagogical research

This tool could be adapted to evaluate the effectiveness of a pedagogical intervention, where a teaching or assessment issue has been identified as problematic.

Constructing a Learning Objectives Questionnaire

Give some form of general instruction at the beginning of the LOQ to explain the rationale which shifts the focus from more traditional course evaluation to students’ perceptions of how successful lecturers have been in conveying their learning objectives.

Select those elements of your course that you want to evaluate and present your learning objectives in the form of simple statements. There is no limit to the number of objectives you could list but if you have a large number of course elements, it might be advisable to limit the objectives to no more than three.

Ask students to respond to each specified learning objective on a 5 point rating scale : Very successful, Successful, Unsure, Not successful, Very unsuccessful which are scored as 5,4,3,2,1, respectively. Thus a high score indicates a positive student evaluation and a low score indicates a negative evaluation suggesting the learning objective has not been perceived as successful.

Obtain scores for each individual learning objective and total to give an overall score for the course element.

After each course element and related learning objectives, ask the open-ended question:

‘What is the single most important thing you felt you learned from…’

This question is based on Angelo and Cross’ (1993) ‘minute paper’ where they suggest that after each taught session, students are asked to write brief answers in response to two questions about what they had learned and what they felt they had not yet learned. Angelo and Cross argue that the advantage of this method is that it can be used to indicate to the lecturer where misunderstandings have arisen so that they can be discussed in the next session and put right. This idea has been adapted for the LOQ by asking just the one question about what students had learned. In this way it is possible to capture some important learning experiences that the LOQ might otherwise miss. It also enables lecturers to use the responses to interpret the ratings and identify areas of potential misunderstandings between lecturers and students. An example of how this can work is reported in Norton, Horn & Thomas (1997) where it is argued that using the LOQ can be a useful part of course design and modification, particularly when it is used as an ongoing and interactive course evaluation tool.

Example LOQ items

To illustrate the procedure an extract from a Counselling Psychology LOQ is given below. These items are taken from the Counselling Psychology course reported in Norton, Horn & Thomas (1997) and are chosen from a much longer instrument to illustrate how the LOQ can be used to incorporate different course elements in the same format. The three course elements here represent a lecture, a workshop and an assessment task.

The lecture element

This questionnaire has been designed to find out how successful you think we have been in achieving specific aims on this course. What we have tried to do is identify examples of different elements of the course and, for each of them, state what we intended would be the learning objectives. What we really need to know is whether they have worked or not, so please be absolutely honest. For each objective will you please circle the one response that most accurately represents your view, where:

What is the single most important thing you felt you learned from this assignment?

Template to create your own learning objectives questionnaire

Instructions

This questionnaire has been designed to find out how successful you think we have been in achieving specific aims/learning objectives on this course. What we need to know is whether you think they have worked or not, so please be absolutely honest. For each objective will you please circle the one response that most accurately represents your view, where: