Another excellent Edublogs.org weblog

Menu

Main menu

Daily Archives: August 1, 2009

A Live Science article, Obama: Key to Future Is Innovation by Robert Roy Britt, discusses Obama’s call for innovation through education and controlling health care costs, The problem is that the concepts are a little to vague and unfocussed. What we need most is innovation in value creation similar to that called for by Umair Haque who writes in the Edge Economy Blog about The Value Every Business Needs to Create Now. He advocates for thick value. Thin value is “built on hidden costs, surcharges, and monopoly power”, while thick value is “awesome stuff that makes people meaningfully better off”. As a country we either need people to shift to thick value or we need to start picking winners and losers with the loser coming from the ranks of the thin group. If we reduce the cost of health care, it has to come out of someone’s pocket unless that someone starts to create thick value.

Most of my ideas on validity are based on the ideas of Cronbach and Messick. Lee J. Cronbach and Paul E. Meehl wrote the 1955 classic paper: Construct Validity in Psychological Tests in which they were the first to present a fully formed conception of construct validity. Working out the details of this paper eventually led to a unitary concept centering on construct validity. Tests and assessments, to varing degrees, are measures of constructs, not things in themselves. Validity is the process of assembling various sorts of data to make inferences about the meaning and utility of the measurement of these constructs. Messick lists six type of validity data that I discussed here. Messick was instrumental in advocating for a unitary conception of validity, but I give him most credit for adding consequential evidence as a catagory of data for making inferences about validity. Validity is not just technical standards, it’s about making accomplishments using assessments as artifacts in processes to attempt to do things.

I admire these men not just because they coined specific approaches, but because they dealt seriously with the big philosophical issues of their time. Philosophies like positivism and objectivism were being criticized and challenged during the second half of the 20th Century. They did not ignore these critiques, but took them seriously and addresses them head-on through their development of validity.

Why is Validity so Seldom Addressed as a Basis for Evidentiary Considerations

This is also about the 20th Century paradigm wars. Most people in the 20th Century fell into the camps of either positivist or post-positivists. Neither of these camps had much interest in Messick’s and Cronbach’s construct validity. Positivists wanted simple empirical evidence of validity that could be expressed authoritatively in some sort of criterion coefficient. They had little interest in any manner of inferring that was convoluted by theory or consequences in a validity that had to be argued instead of proclaimed. Post-positivists on the other hand often rejected measurement and empirical methods for textual and critical methods. These textual and critical methods were needed, but they sort of threw out the baby with the bath water. By rejecting measurement approaches in general, they never saw that Cronbach and Messick came to share many of the same concerns and was working out a critique of positivism in their own way.

There were few people in the middle ground back then, but I believe that is changing today. There is increasing call for evidence-based approaches and a renewed interest in both empirical and rational methodologies. To avoid falling into the same problems of the positivists, we should seriously consider the type of post-positivist rational empirical method of validity advocated by Cronbach and Messick.