Businesspeople are overloaded with information about
information overload. They don't need to read one more article
about how many articles they need to read. And not only is there a
lot of information to synthesize, but serious research results can
be dry, scholarly, and difficult to digest.

The problem is, if executives miss the results of crucial
organizational research, they lose the chance to learn about highly
profitable management approaches. But most managers and executives
attained their positions because they're businesspeople, not
statisticians. They require an assist from meta-analysts.

Meta-analysis is the science of studying the results of many
studies at once. Meta-analysts pry the meaning from vast piles of
statistics, providing executives with actionable information, not
just a swarm of numbers. This research is the backbone of the
metrics of productivity -- the methods that provide real
results.

Dr. Frank Schmidt is one of the world's foremost
meta-analysts. He is also a Gallup Senior Scientist -- one of a
group of leading scientists who lend their expertise to The Gallup
Organization's work. Dr. Schmidt is the author of hundreds of
articles on meta-analysis, methodology, selection, individual
differences, and a vast spectrum of other issues in industrial
psychology. He is the winner of dozens of eminent awards,
including, with Dr. Jim Harter, Gallup Chief Scientist -- Workplace
Management and Dr. Ted Hayes, Gallup SRI Research
Director, the 2002 Best Paper Award from the Organizational
Behavior Division of the Academy of Management. He has taught in
universities in several countries and has been an advisor to
businesses, humanitarian groups, the military, and the criminal
justice system.

How can Dr. Schmidt's vast and heady academic expertise help
companies improve profit margins? Well, his research gives
executives valuable insights -- such as how much to value a highly
experienced job candidate and how much, literally, good employees
are worth. In the following interview, Dr. Schmidt shows how the
thousands of studies he's studied boil down to keys every executive
can use.

GMJ: What does meta-analysis offer that
individual studies don't?

Dr. Schmidt: The ability to make sense of large
numbers of studies on the same subject, which are typically
conflicting. You don't get the same results in every study, and
meta-analysis makes it unnecessary for you to even try to read all
the individual studies; the meaning of the studies as a whole is
condensed into the meta-analysis results.

And those results typically show that there isn't nearly as much
conflict and disagreement as it appeared when you looked at each
study individually. You typically find that there's a clearer
meaning than you thought to the research that's been done. That's
very, very helpful -- you can see what it means, and you can apply
it.

GMJ: And it takes a lot less time.

Schmidt: It takes less time than reading all
the studies. But even if you did read all of them, you still
couldn't disentangle them; you still could not interpret them.

I was thinking the other day that you can compare meta-analysis
to a search engine. We're all swamped with information overload
today. There's an incredible amount of information on practically
everything. Search engines allow you to find particular pieces of
information instantly that you might otherwise never find in a year
of searching through the library.

Meta-analysis tells you what very large amounts of information
on a particular subject mean. It eliminates the variation from
sampling error and other statistical artifacts in studies and gives
you the bottom-line results of a whole lot of research. Search
engines and meta-analysis are both ways of managing the incredible
information overload. The human mind simply cannot deal with that
much information, so we need tools to make it possible.

GMJ: So meta-analysis allows you to study
the results of surveys with 10,000 people and find the two or three
real problems that hamper employee engagement?

Schmidt: Well, take the Gallup Q12
employee engagement survey for example. It's often administered to
thousands of people at a time. But the Q12 only has 12
items, and each one of those items is very specific -- all the
questions have to be actionable if you want results. If a business
unit has a low score on one of these, then it is clear enough what
the problem is so that you can take action, no matter how many
people take the Q12 survey.

Next, the overall company results from the survey should not be
distributed to the company as a whole. Instead, the feedback should
be broken down by business unit. Remember, if you're going to give
feedback to an individual manager, it has to apply to that
manager's unit. The company scores as a whole should be looked at
by higher level executives, and the scores could also be compared
to other companies -- to norm groups -- to help benchmark results.
But the real value is at the level of the individual manager and
his or her department. Distributing results at the workgroup level
ensures that a manager receives feedback that provides guidelines
for what should be worked on, what should be improved.

GMJ: There's often some skepticism in the
ranks when these surveys are administered. But by the second or
third time, I've seen pure managerial evangelism for engagement
studies.

Schmidt: Well, you know, I think the Gallup
Q12 has a generic effect in improving managerial
performance. I think it stimulates an overall improvement in the
quality of management. And if there is some problem that doesn't
exactly fit into any of the particular questions on the
Q12, that problem will probably be addressed by the
manager as the manager becomes better in general.

GMJ: Do you have to give the surveys over
and over again to measure change, or can you do it once and call it
good?

Schmidt: These surveys are rarely one-shot
applications, and they really never should be. Company surveys --
the Q12 or any other -- should be done periodically;
they're typically done every year in many companies. That way,
managers have the chance to see an improvement in the engagement
scores -- item-by-item and overall, or question-by-question and
overall -- that tells them the process is working. That's because
these survey questions not only have to be actionable, managers
must review the results periodically to determine how well the
actions they took worked.

GMJ: So a meta-analysis can show you two
serious problems for productivity. But what's the relationship
between workforce productivity and good employee
selection?

Schmidt: Well, that's a question related to
practical utility. Many, many studies have been done on that, and
they show that good selection greatly increases the average level
of output of employees on the job. And that's the bottom-line
nature of the utility metric. Going from a poor selection procedure
to a good selection procedure can easily increase the average
output per person on the job by 10% or maybe even 15%. To some
people, that doesn't sound like a lot, but it translates into very
large numbers.

GMJ: One of the more traditional methods of
employee selection is to confirm that the candidate has lots of
experience, then do interviews to see what the candidate is like.
What are your thoughts on that?

Schmidt: Well, it's not the best method. You
mentioned relevant experience. We have done a lot of research on
that. Job experience does have validity for predicting job
performance, but that validity fades out with time. After five
years on the job, it will be the more intelligent people with
better personalities who will be the better workers -- not those
who initially had more job experience.

GMJ: So if you have to make a choice
between a conscientious, intelligent person with three years of
experience and a real jerk who has ten years of experience, you're
better off with the first person?

Schmidt: Under those circumstances, I would
prefer the one with three versus ten years of experience. The
validity of experience for particular job performance fades over
time, but the personal characteristics validity does not fade.
Initial learning on the job is pretty steep during the first five
years. It makes a big difference whether you've been on the job one
year or three years.

Learning through experience, however, levels off with time --
and for many jobs, that's at about five years. So a person with ten
years of experience will not, on average, have better job
performance than a person with five years. It might be a longer
period for more complicated jobs, but for the typical mid-level
job, that's about the breaking point -- people have learned about
as much as they are going to learn about how to do that job after
the fifth year. The difference you can attribute to experience will
fade away and will no longer affect performance. What will become
important will be mental ability, personality, and
conscientiousness -- personality traits. These traits do not fade
away. That is, their predictive ability continues.

GMJ: You've done a lot of research into
"individual differences." What's that?

Schmidt: Maybe I should start by telling you
what the alternative is. There are a lot of people in psychology
who look for general laws -- the ways all people are alike. That's
the opposite of the individual-differences approach. You're looking
at differences in general intelligence, differences in interest,
differences in values -- these differences are just gigantic. It's
important if you're looking at the ways people respond, for
example, to different training techniques. And it really does call
into question whether it is possible to have a general law that
applies to everyone. I said this in class the other day, and my
students were shocked -- the range of ability in the top 1% of a
group is greater than the range of ability in the middle 98%.

GMJ: How can that be?

Schmidt: Because it's the shape of the normal
bell curve. The normal bell curve just keeps going further and
further out. You know, in the top 1%, a person who is right at the
cutoff can be different by four standard deviations from someone
who is further out on the curve. There aren't very many people way
out there, but there are some, and they are important.

GMJ: How can knowing the
individual-difference bell curve help businesses?

Schmidt: For example, we were studying the
practical value of selection procedures, and we found that
supervisors were much more concerned about low performance than
high performance in their employees. Their attention tended to be
focused on the people who were a problem. They thought it was more
important to increase the performance of the people at the bottom,
just bring the low end up by a certain amount. As a result, they
didn't appreciate their high performers as much as they should
have, and really great performers didn't get much attention because
they didn't cause problems.

GMJ: That's just disastrous to an
organization -- if managers spend all their time and energy on
people who barely produce, they won't find time to make their stars
into superstars. What can change that focus on the low
end?

Schmidt: High levels of competition in the
industry. It's a shame that it has to get to that point, but if
some other organization is breathing down your neck, you start
focusing on the high performers because these are the people who
can meet the challenge. As I mentioned, just a 10% improvement in
productivity translates into big numbers, so paying attention to
the top of the curve can be tremendously beneficial.

GMJ: As long as you don't hire
jerks.

Schmidt: Well, we don't use terms like that.
But yeah, the key is to hire the right people.