Idea of Construct Validity

Construct validity refers to the degree to which inferences can legitimately be made
from the operationalizations in your study to the theoretical constructs on which those
operationalizations were based. I find that it helps me to divide the issues into two
broad territories that I call the "land of theory" and the "land of
observation." The land of theory is what goes on inside your mind, and your attempt
to explain or articulate this to others. It is all of the ideas, theories, hunches and
hypotheses that you have about the world. In the land of theory you will find your idea of
the program or treatment as it should be. You will find the idea or construct of the
outcomes or measures that you believe you are trying to affect. The land of observation
consists of what you see happening in the world around you and the public manifestations
of that world. In the land of observation you will find your actual program or treatment,
and your actual measures or observational procedures. Presumably, you have constructed the
land of observation based on your theories. You developed the program to reflect the kind
of program you had in mind. You created the measures to get at what you wanted to get at.

Construct validity is an assessment of how well you translated your ideas or theories
into actual programs or measures. Why is this important? Because when you think about the
world or talk about it with others (land of theory) you are using words that represent
concepts. If you tell someone that a special type of math tutoring will help their child
do better in math, you are communicating at the level of concepts or constructs. You
aren't describing in operational detail the specific things that the tutor will do with
their child. You aren't describing the specific questions that will be on the math test
that their child will do better on. You are talking in general terms, using constructs. If
you based your recommendation on research that showed that the special type of tutoring
improved children' math scores, you would want to be sure that the type of tutoring you
are referring to is the same as what that study implemented and that the type of outcome
you're saying should occur was the type they measured in their study. Otherwise, you would
be mislabeling or misrepresenting the research. In this sense, construct validity can be
viewed as a "truth in labeling" kind of issue.

There really are two broad ways of looking at the idea of construct validity. I'll call
the first the "definitionalist" perspective because it essentially holds that
the way to assure construct validity is to define the construct so precisely that you can
operationalize it in a straightforward manner. In a definitionalist view, you have either
operationalized the construct correctly or you haven't -- it's an either/or type of
thinking. Either this program is a "Type A Tutoring Program" or it isn't. Either
you're measuring self esteem or you aren't.

The other perspective I'd call "relationalist." To a relationalist, things
are not either/or or black-and-white -- concepts are more or less related to each other.
The meaning of terms or constructs differs relatively, not absolutely. The program in your
study might be a "Type A Tutoring Program" in some ways, while in others it is
not. It might be more that type of program than another program. Your measure might be
capturing a lot of the construct of self esteem, but it may not capture all of it. There
may be another measure that is closer to the construct of self esteem than yours is.
Relationalism suggests that meaning changes gradually. It rejects the idea that we can
rely on operational definitions as the basis for construct definition.

To get a clearer idea of this
distinction, you might think about how the law approaches the construct of
"truth." Most of you have heard the standard oath that a witness in a U.S. court
is expected to swear. They are to tell "the truth, the whole truth and nothing but
the truth." What does this mean? If we only had them swear to tell the truth,
they might choose to interpret that as "make sure that what you say is true."
But that wouldn't guarantee that they would tell everything they knew to be true. They
might leave some important things out. They would still be telling the truth. They just
wouldn't be telling everything. On the other hand, they are asked to tell "nothing
but the truth." This suggests that we can say simply that Statement X is true and
Statement Y is not true.

Now, let's see how this oath translates into a measurement and construct validity
context. For instance, we might want our measure to reflect "the construct, the
whole construct, and nothing but the construct." What does this mean? Let's
assume that we have five distinct concepts that are all conceptually related to each other
-- self esteem, self worth, self disclosure, self confidence, and openness. Most people
would say that these concepts are similar, although they can be distinguished from each
other. If we were trying to develop a measure of self esteem, what would it mean to
measure "self esteem, all of self esteem, and nothing but self esteem?"
If the concept of self esteem overlaps with the others, how could we possibly measure all
of it (that would presumably include the part that overlaps with others) and nothing
but it? We couldn't! If you believe that meaning is relational in nature -- that some
concepts are "closer" in meaning than others -- then the legal model discussed
here does not work well as a model for construct validity.

In fact, we will see that most social research methodologists have (whether they've
thought about it or not!) rejected the definitionalist perspective in favor of a
relationalist one. In order to establish construct validity you have to meet the following
conditions:

You have to set the construct you want to operationalize (e.g., self esteem) within a semantic
net (or "net of meaning"). This means that you have to tell us what your
construct is more or less similar to in meaning.

You need to be able to provide direct evidence that you control the
operationalization of the construct -- that your operationalizations look like what they
should theoretically look like. If you are trying to measure self esteem, you have to be
able to explain why you operationalized the questions the way you did. If all of your
questions are addition problems, how can you argue that your measure reflects self esteem
and not adding ability?

You have to provide evidence that your data support your theoretical view of the
relations among constructs. If you believe that self esteem is closer in meaning to self
worth than it is to anxiety, you should be able to show that measures of self esteem are
more highly correlated with measures of self worth than with ones of anxiety.