360 Questionnaire Design

August 1, 2017

This week’s vlog focuses on 360 questionnaire design intervention.

Let me start by clearing something up: the confusion between 360 and psychometric testing. The word psychometricis formed from the Greek words for mental and measurement. Psychometric tests attempt to objectively measure aspects of your mental ability or your personality. You are most likely to encounter psychometric testing as part of the recruitment or selection process and occupational psychometric tests are designed to provide employers with a reliable method of selecting the most suitable job applicants or candidates for promotion.

Psychometric tests aim to measure attributes like intelligence, aptitude and personality, providing a potential employer with an insight into how well you work with other people, how well you handle stress, and whether you’ll be able to cope with the intellectual demands of the job. As a result, they should be statistically examined and subsequently licensed for face validity and reliability. Does that sound like a 360? No.

360 degree feedback is simply a survey of people’s perceptions about an individual’s behaviour, including that individual’s self-perception. In fact, it is often the contrast between self-perception and other people’s perceptions that shows the individual where their development opportunities are.

By behaviour, we mean something the person does (or doesn’t do) and/or something they say (or don’t say). As such, it is something that can be observed.

So what makes for a good 360 questionnaire? #1: be clear on what you want to measure, and why. Be as specific as possible about what behaviours are involved and how you will know if they are improving as you continue to measure them. Typically, each of your 360 questions will be a specific observable behaviour which describes what “finished” looks like. In other words, the statement is how you would like the person to behave.

If you already have a well-defined behavioural framework, this can become your 360 questionnaire. If you are using an existing competency framework, or leadership standard, or set of values, the same applies BUT ONLY if the statements are described as specific observable behaviours AND they meet the other criteria which I’m coming to.

#2: be sparing with the number of behaviour or competency headings, and with the number of questions per group. Between 4 and 6 headings is ideal, and between 4 and 6 questions (or statements) per heading. This would give between 16 questions overall at one end of the spectrum, and 36 at the other.

Why this number? Because when people are asked to contribute time and effort to completing a 360 questionnaire, either on themselves or on a colleague, this has to be seen as extra to their day job. It’s a huge gift you’re asking of them. And most of us are already time-poor. To complete 16 questions even if it’s just a button click on a rating scale is likely to take at least a minute per question. And that’s without any written comments.

And that leads us to make an important point: We think there’s a popular misconception about the purpose of a 360 questionnaire – too many of you believe that the value is in the report that comes out of the back end of the process, so therefore the more questions the more information will be provided. However, in our experience the real value of a 360 is in the conversations that it generates around the workplace, and in particular the resultant personal development plan that ensues. I invite you to explore the ‘checking out’ and ‘checking-in’ processes described elsewhere on the click-360 website. So on that basis, when it comes to the number of questions, less is more.

#3: Keep the questions simple. Use an active verb, for example “Thinks before acting”. Keep the statements positive. Ask only one thing at a time. Remember the ‘rule of 7’: 7 words max in the statement, 7 letters max in each word. I know that’s a bit idealistic, but you get my drift! Above all, keep the questions unambiguous.

#4: Choose a straightforward rating scale. Also known as a likert scale. The best ones are based on frequency or agreement. So at one of the scale you have “Always” or “Strongly agree”, at the other end you have “Never” or “Strongly disagree”. How many points on the scale? Well popular opinion suggests that if you have an even number of points you force people onto one side or the other (positive or negative), whereas if you have an odd number of points too many people will sit in the neutral middle: “Neither agree nor disagree”.

Consider the advantages of removing the mid-point in an even-numbered scale: it forces people to choose, people may be more discriminating and more thoughtful, and it eliminates a possible misinterpretation of mid-point. By contrast, the disadvantages are: it forces people to choose!!, respondents could become frustrated and you may not be collecting accurate responses. There is no “right” way – it comes down to personal choice. The good news is that with our click-360 questionnaire screens, you can drag your cursor to anywhere on the likert scale…you have in effect a 100 point scale!

#5: The fifth and final point to make around 360 questionnaire design is: invite free text comments to clarify and explain the ratings. In our experience, greater depth and clarity is afforded by having comments at the end of each behavioural heading, or group. One comment box asks for examples of things the participant does well in respect of that behaviour or competency, the other asks for things the participant could do differently.

Most of our implementations also invite a final page of open text comments, typically the headings are “Things the participant might STOP doing”, “Things the participant might START doing” and finally “Things the participant might CONTINUE doing.” This stop, start, continue approach allows the respondent to either reinforce ratings (or comments) already covered and/or to add comments about areas that lie outside of the questionnaire.

If you need help writing a bespoke questionnaire, or you’d like us to check out something that you already have, please just email or call.

Please post a comment and tell me what you think of the vlog and feel free to ask any questions. I’ll be answering questions and reviewing comments personally.