Journal Interview, May 2011

In a recent analysis ofEssential Science IndicatorsSMfromThomson
Reutersdata, the journal Applied
Psychological Measurement (APM) entered the rankings of highly
cited journals inthe field of Psychiatry
& Psychology. The journal's current record includes 320 papers
cited a total of 1,542 times between January 1, 2000 and December 31,
2010.

APM was founded in 1976 by Dr. David J. Weiss. It is now published
by SAGE Publications, Inc., and edited by Dr. Mark L. Davison.

Below, ScienceWatch.com talks with Weiss and
Davison about APM's history and citation
achievements.

Did you expect Applied Psychological
Measurement (APM) to become highly cited, or is this surprising to
you?

We were pleasantly surprised by this improvement in our citations. Like
many journals, APM gives some consideration to the likelihood of
citation in its editorial process, and we have been trying to improve our
citation index. But our evaluations of articles do not consider the
likelihood of citation explicitly. Our major criteria are "significance"
(to the field of applied psychological measurement), "appropriateness" (fit
to the journal's content mission), "clarity of expression," and" technical
adequacy." Likelihood of influencing practice plays a bigger role in our
decisions than does likelihood of citation.

This result is not completely surprising, however, as the measurement
methodologies published in APM have been slowly spreading from
applications in psychology and education to other fields, in particular
medical behavior and medical outcomes research.

How would you account for the high citation
rate of APM?

One reason for the increased citation rate may be the increased number of
subscribers. From 2006 to 2009, the number of subscribers has more than
tripled, rising from 1,369 to 5,128. The increased subscription base may
mean an increase in the number of potential authors who may cite the
journal.

"At the time the journal was created, there were journals dealing with
statistical and technical issues in educational measurement but no
journals devoted to similar issues in psychological measurement."

We try to publish articles that are readable and relevant to practice. Most
of our articles introduce new methods or provide a comparative evaluation
of several competing methods. These latter types of articles are designed
to help test developers and researchers decide how best to address a
problem in their work. We try to alert researchers and practicing
professionals to new developments in relevant software. We also try to
provide reviews of highly specialized books relevant to our field.

In a nutshell, our goal is to publish useful information.

Would you give us a brief history of the
journal?

The journal was founded by Dr. David J. Weiss in the Department of
Psychology at the University of Minnesota in 1976. He created a private,
nonprofit corporation (Applied Psychological Measurement, Inc.)
whose sole purpose was to publish the journal. He served as editor for 25
years. Dr. Mark Reckase of Michigan State University served as the second
editor, and I succeeded him.

When the founder, Dr. Weiss, decided to step down, he decided to sell the
journal to SAGE Publications, Inc. as a way to ensure the journal's
continuity. Funds from the sale were used to create a foundation (the
Applied Psychological Measurement Foundation) whose
income has been used to fund graduate student research at universities
around the world in the area of educational and psychological measurement
methodologies.

At the time the journal was created, there were journals dealing with
statistical and technical issues in educational measurement but no journals
devoted to similar issues in psychological measurement. The journal focus
is on methods applicable in psychological measurement or methods applicable
in both educational and psychological measurement.

At the time the journal was founded, item response theory (a leading-edge
technology for psychological measurement) and computerized testing were
emerging as important new developments. Throughout its history,
APM has had a special interest in computerized testing and the
item response theories on which it is based.

What historical factors have contributed to
the success of APM? Have there been specific developments in
the fields served by APM that may have contributed?

One of the most important was is the emergence of both item response theory
and computerized testing in higher education admissions, military testing,
and professional licensing.

Another is the continuing societal concern about test fairness to women and
minorities. The journal has published many articles on "differential item
functioning," which involves methods of studying test fairness.

While this may not be an historical trend, of late we have been receiving
and publishing articles on the detection and prevention of cheating.

The accountability movement in education has also been an influence.

What, in your view, is this journal's main
significance or contribution in the field of Psychiatry &
Psychology?

"We try to publish articles that are readable and relevant to
practice."

In the past, APM's main contributions have been its role in the
development of computerized testing and assessment. This has influenced all
sorts of assessments in education (including medical education), industrial
organizational psychology, counseling psychology, clinical psychology,
psychiatry, and some areas of health care (e.g., mental functioning of the
elderly in gerontology and the development of instruments for measuring
patient-based outcomes of medical treatments).

How do you see your field(s) evolving in the
next few years?

Computerized testing will continue to develop, but I think it will develop
in new directions. Currently, computerized tests typically employ
multiple-choice items, but in ways which allow examinations to (a) maintain
accuracy but shorten test length or (b) increase accuracy while holding
test length constant. It also allows us to report results far more quickly.
Rather than employing multiple-choice response formats, future tests will
use response formats that employ joysticks and touch screens to create
objectively scorable test items that are not multiple choice.

Also, item presentation formats will change and become much more dynamic
than is possible with a printed test booklet. Questions will become more
interactive so that the questions posed at each stage will depend on the
answers at earlier stages; in the future, the computer will "probe"
examinee answers. These new testing methodologies will require new
psychometric developments to handle the unique measurement problems that
they pose.

In many current tests, the total score is very useful but subscores are
less so. Future research will be designed to improve the information and
interpretation of the subscores. For instance, an evaluation of employees
may provide a useful "total" score evaluating the overall performance of
the employee, but the subscores describing information in specific areas
(leadership, team building, providing feedback) is often not nearly so
useful.

Future work will be designed to improve the reliability and validity of
subscores without markedly increasing the test length. By improving
subscores, the tests will be providing more information and will become
more useful for diagnosis, vocational placement, differentiation of
instruction, and other applications.

Farther into the future, research on brain scanning techniques (e.g., fMRI)
might lead to completely new methods for the measurement of such things as
emotions and mental processing. However, it will take some time before
these developments begin to affect applied psychological measurement.

What role do you see for your
journal?

The journal's focus is on more technical aspects of the testing process.
The journal will continue to publish new methods and evaluations of
alternative methods for addressing problems in the field, including the
problems described above. It will alert readers to software that can be
used to address the problems and to highly specialized books describing
possible solutions and research methods for evaluating those solutions. And
it will continue to publish leading-edge methodological developments that
address issues in all applications of psychological measurement
techniques.