Recruitment and Selection

Rating Scales Commonly Used in Interviewing

Structured and Unstructured Interviews

Employment interviews can be structured, or unstructured in format. Structured interviews are planned in advance and often include suggested interview questions (most popular are behavioral questions that probe past experience and approach to tasks and challenges) that the interviewer can use to guide the interview. Unstructured interviews tend to be “looser” in nature and generally do not include standardized questions and a minimum (if any) of pre-planning. Substantial research over the past 30 years has consistently supported the use of structured interviews as far superior to unstructured interviews (higher validity and reliability coefficients). This article examines the use of rating scales in structured interviews.

Use of Rating Scales in Interviews

The interviewing process often includes the use of rating scales that the interviewers (also called “raters”) use to evaluate the quality of the interview and lead to some kind of recommendation of the candidate for the job in question. Ratings generally evaluate one or more of the following:

Responses to individual questions (planned in advance)

Job-related competencies that may or may not be linked to individual questions posed during the interview (also planned in advance)

The overall interview (as a kind of global experience, usually a single rating).

Each of these three approaches will be examined in turn below.

Per Question Rating

Organizations that use the first approach – per question rating – tend to be in the public-sector or otherwise very structured in their human resources or selection procedures. Below is a typical 5-point scale used with this approach:

Well Qualified (5) Candidate provides a thorough response to the question. Candidate demonstrates a thorough understanding of the issues at hand that is more to substantially more than the job requires. Response is well thought-out and well presented. Overall, candidate’s response is complete, addresses all aspects of the question and does not require probing.

Qualified (4-2) Candidate provides an acceptable response to the question. Candidate’s understanding of the issues at hand is equal to or slightly less that what the job requires. The response may not be as complete or thorough as the well-qualified candidate’s response. Overall, candidate’s response is complete, addresses the question and any probing required is minimal.

Not Qualified (2-1) Candidate fails to provide an acceptable response to this question. Candidate’s response does not convey the level of experience/expertise required in this position. Candidate’s response may be vague or incomplete. Overall, candidate fails to provide experience/expertise demonstrative of the requirements of this position.

Though this scale is well thought-out and could be useful to a rater in evaluating questions, the “per question” rating process is flawed. The problematic aspect of using the per question rating approach is two-fold: first, it limits flexibility in the interview itself; raters must ask every single listed question to be able to complete the rating process. Secondly, it assumes that the answer to each question will tap directly into the competency that the question was designed for, which is ambitious at best, and unrealistic and untenable, at worst. Evaluating a job-related competency is better accomplished by considering the candidate’s discussion and behavioral examples throughout the interview, rather than one-at-a-time.

For example, a question like this:

“Tell me about a time when you worked hard to track down the source of a problem.”

Might be designed to measure problem-solving skills, however, a candidate who responds with an answer like this:

“Well, on my last job, I discovered a problem with the intake process. I decided to design a new process based on our customers’ needs. First I designed a questionnaire to collect the data we needed. Then …”

Is not really reporting directly on her problem-solving skills, but may be giving a great example of initiative and creativity, which could be other essential competencies for the job for which he or she is applying (or not). Instead of penalizing him or her with a low rating on problem-solving (which would be required using the per question rating approach), a more flexible per competency rating scheme would apply his or her response more directly to ratings for initiative and creativity, in which case the interviewer could then listen for evidence of problem-solving skills in other parts of the interview, or even ask a second question related to problem-solving to collect more data.

Overall Interview Rating (in isolation)

This approach, rating the interview in general without rating individual questions or competencies, is used by organizations that prefer a low degree of structure and allow interviewers free reign to ask questions as they see fit, but still want a way to quantify the candidate’s overall interview performance. Below is a typical rating scale used to rate overall interview performance:

In general, this approach tends to result in very little differentiation with scores clustering in the middle with little evidence to support ratings. Raters just seem to need more structure than making a global judgment. In our view, this approach is too loose to work well, as raters need more direction in terms of what to listen for.

Assigning overall ratings can be useful, however, when combined with the per competency rating process described below.

Per Competency Rating

For all of the above-mentioned reasons, we favor the rating competencies based on candidate responses throughout the interview approach, even though we may have behavioral questions at our disposal that are designed to elicit examples of behavior that relate directly to that competency. An overall measure at the end of the interview can then be arrived at by considering ratings of individual competencies.

A typical scale used to rate competencies evaluated in an employment interview is:

5 = Superior skills in this competency; could mentor or teach others in this
4 = Good skills in this competency; above-average ability is apparent
3 = Adequate skills in this competency; no additional training is needed at this time.
2 = Marginal skills in this competency; some training would be required to bring skills up to an acceptable standard
1 = Not competent in this area; competency needs substantial development

Rating Scales Types

In our review of current practice in recruitment and selection in organizations of all sizes and types (or at least those with more than 100 employees), we found that a 5-point scale is most commonly used, though other types of scales can certainly be used if the raters clearly understand the standards tied to each numerical value. Since interviewing is a highly subjective experience, trying to construct an elaborate rating process is generally not advisable.

Further, some organizations try to adapt and apply their performance appraisal rating scales to the selection process, which generally does not work very well, as these two human resources practices are quite different, both in intent and format. Performance appraisal is looking backward at actual experience over an extended period of time (generally a year); interviews are looking at behavioral examples from different jobs and experiences and projecting forward trying to predict future fit in a particular job.

Interview Generator-a possible solution for all recruiters

Interview Generator, at www.BuildAnInterview.com, is an online system developed by the author of this article and used widely by HR professionals and line managers in the employment selection process. The system allows users to customize competency-based interview guides that are used by interviewers to conduct job-related selection interviews.

Once an interview guide has been finalized, the report generator within the program presents the user with options for producing a written report. These options include:

An HTML version

PDF version

Word versions (Long and Short)

The rating scales used in the first two versions is a 5-point scale with standards set at 3 points within the scale: Low (1), Moderate (3) and High (5).

The scale used within the Word versions of the report include a different scaling system that allows for weighting of the different competencies, a 5-point scale, and no standards beyond the target behaviors that are matched to each interview question.

Each rating scale is detailed below.

A. On page one of the guide

Instructions

The Interview Guide includes one to four interview questions for each competency category (for example, Interpersonal Skills). Also supplied are target behaviors to look for in the individual’s responses for that competency. Interviewers are advised to be sure to ask at least one question in each competency category; it may not be necessary to ask more than one of the supplied questions for each category because the candidate may have given sufficient evidence of the existence of a particular competency elsewhere in the interview and interviewer judgment is required to make this determination. In this way the interviewer has flexibility to focus the interview appropriately to obtain high quality information in a short amount of time.

Comments/Rating

In the Comments section, the rater is asked to make brief notes to support his or her observations and judgments about the individual’s skills as related to that particular competency and to make a rating, from Low to High, for each competency, using these standards:

Rating

Rating standard

1

Low

Candidate’s response contained very few of the target behaviors. Either the behaviors he/she discussed were not at, or even close to, the level indicated in the target behaviors, or the person did not give you enough information for you to have confidence that he/she has that competency at the level needed for success.

2

Low-Moderate

3

Moderate

Candidate’s response covered some of the target behaviors, but not quite at the level that would be ideal for that competency. Still, the quality of the person’s answers leads you to believe that he/she would be successful with some additional exposure and/or training.

4

High-Moderate

5

High

Candidate’s response contained many, if not all, of the target behaviors. His/her responses indicate well-developed skills and aptitude for that competency, which would most likely lead to job success. The person’s responses are of superior quality for this job.

B. On each of the question pages of the guide, following the questions, the rating scale below is included:

1

2

3

4

5

Low

Low-Moderate

Moderate

High-Moderate

High

C. On the last page of the guide is an Overall Rating

Your overall rating should reflect your individual competency ratings. Record notes to support your overall rating. Rate the candidate’s skill level for this position overall, based on the interview.

1

2

3

4

5

Low

Low-Moderate

Moderate

High-Moderate

High

Using the system below, if there are multiple raters for the same role, those raters should determine the relative weights of each competency and apply the same values to each candidate. Where raters may vary is on the Candidate Assessment side as they may not interpret the candidate’s interview answers exactly the same way, even given the target behaviors supplied with each interview question.

Using this system, the rater simply multiplies the criteria weighting times the candidate assessment score to yield a total score in the last column and then adds these to arrive at a final score that can be compared to other candidates.

Related Products:

Share this article.

About Anne Sandberg

With a degree in Experimental Psychology and a masters in Industrial/Organizational Psychology, Anne Sandberg has 25+ years of experience in the human resources, training and management consulting arenas. Anne is President of ReadyToManage, Inc. and can be contacted at Anne.Sandberg@ReadyToManage.com

ReadyToManage is your one-stop shop for world class employee and personal development resources. Our mission is to assist individuals and companies in developing management, leadership, and business skills in themselves and their employees through effective and affordable development materials and courses.