Cognitive Sciences Stack Exchange is a question and answer site for practitioners, researchers, and students in cognitive science, psychology, neuroscience, and psychiatry. It's 100% free, no registration required.

There are different kinds of psychometric tests, I think the numerical and verbal ones are fair enough, I can see their usefulness. However, I cannot understand how the 'soft' psychometric tests work and how they apply to a particular career.

I just finished one which consisted of 90 'questions', except they were not questions, they were all "pick one of the two following sentences" 90 times. Some made sense, such as

Do you like to lead a group.

Do you like to fit in with a group.

But others I could not answer because I could not choose either option for example:

You are slow at planning.

You are slow at making decisions.

I have to answer one. In this case, I'd say that I am not slow in either.

How exactly (detail) do these types of psychometric tests work? A link to an open one would be what I am looking for. How do they create the answers? How do they pair them? Do they change based on your previous answers?

2 Answers
2

The basic idea is that you get a sample population. Let's say you have 100 people. You give each of those a creativity rating through some process.

Afterwards you give every person in the sample population a stack of questions. You run statistical tests to determine which of those questions correlate with someone being creative. You keep all the questions that strongly correlate with your original creativity rating and throw questions that don't correlate away.

When making a test for a job application you also throw questions that seem to have "correct" answer and who therefore allow for cheating away. Alternatively you can also use those questions to determine the amount to which the test subject tries to game the test.

There is a large general literature evaluating the degree to which personality tests predict job performance. In particular see for example the review by Barrick et al (2001). In general such reviews find that personality measures provide a small but meaningful prediction of job performance.

Ipsative testing

So in general, you are asking about how personality tests work. But more specifically, you are asking about ipsative personality tests:

Ipsative is a descriptor used in psychology to indicate a specific
type of measure in which respondents compare two or more desirable
options and pick the one that is most preferred (sometimes called a
"forced choice" scale).

Regarding your statement that you could not answer one of the questions, presumably the instructions asked you to indicate which statement applied most. Thus, it would be hoped that even if there were two statements that both applied to you a lot, or both applied a little, you would be able to indicate which applied more.

For general scientific use where you can rely on people to answer honestly, researchers typically do not use ipsative testing. Ipsative testing generally distorts the correlations between subscales.However, ipsative tests is popular in applied settings such as in selection and recruitment. The argument is that ipsative testing reduces faking.

Relevant references include:

Heggestad et al (2006) interpreted their results to indicate that ipsative testing was not a viable method for controlling for faking:

Recent research suggests multidimensional forced-choice (MFC) response
formats may provide resistance to purposeful response distortion on
personality assessments. It remains unclear, however, whether these
formats provide normative trait information required for selection
contexts. The current research evaluated score correspondences between
an MFC format measure and 2 Likert-type measures in honest and
instructed-faking conditions. In honest response conditions, scores
from the MFC measure appeared valid indicators of normative trait
standing. Under faking conditions, the MFC measure showed less score
inflation than the Likert measure at the group level of analysis. In
the individual-level analyses, however, the MFC measure was as
affected by faking as was the Likert measure. Results suggest the MFC
format is not a viable method to control faking.

However, Bowen et al reached a more positive conclusion about ipsative testing and the degree to which it is faking-resistant:

The current study extends previous research by comparing empirical
data on ipsative and normative versions of the 30-scale Occupational
Personality Questionnaire (OPQ). Participants were randomly assigned
to “honest” or “faking” groups. Results support the proposition that
when the number of scales is large, the measurement dependency problem
associated with the ipsative format is negligible. There was
satisfactory convergent validity between ipsative and normative forms
in both groups. On both forms, the distance between an ideal profile
and participants' ratings were narrower in the faking group than in
the honest group. The results suggest that though ipsative measures
were not completely free from faking, they were relatively more
effective in guarding against faking than more commonly used normative
measures.

Meade (2004) discuss ipsative measurement more generally.

Data are described as ipsative if a given set of responses always sum
to the same total. However, there are many properties of data
collection that can give rise to different types of ipsative data. In
this study, the most common type of ipsative data used in employee
selection (forced-choice ipsative data; FCID) is discussed as a
special case of other types of ipsative data. Although all ipsative
data contains constraints on covariance matrices (covariance-level
interdependence), FCID contains additional item-level
interdependencies as well. The psychological processes that give rise
to FCID and the resultant psychometric properties are discussed. In
addition, data from which both normative and ipsative responses were
provided by job applicants illustrate very different patterns of
correlations as well as very different selection decisions between
normative, FCID and ipsatized measures.

References

Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: what do we know and where do we go next?. International Journal of Selection and Assessment, 9(1‐2), 9-30. PDF