This course seeks to turn learners into informed consumers of social science research. It introduces concepts, standards, and principles of social science research to the interested non-expert. Learners who complete the course will be able to assess evidence and critically evaluate claims about important social phenomena. It reviews the origins and development of social science, describes the process of discovery in contemporary social science research, and explains how contemporary social science differs from apparently related fields. It describes the goals, basic paradigms, and methodologies of the major social science disciplines. It offers an overview of the major questions that are the focus of much contemporary social science research, overall and for China. Special emphasis is given to explaining the challenges that social scientists face in drawing conclusions about cause and effect from their studies, and offers an overview of the approaches that are used to overcome these challenges. Explanation is non-technical and does not involve mathematics. Statistics and quantitative methods are not covered.
Explore the big questions in social science and learn how you can be a critical, informed consumer of social science research.
Course Overview video: https://youtu.be/QuMOAlwhpvU
After you complete Part 1, enroll in Part 2 to learn how to be a PRODUCER of Social science research.
Part 2: https://www.coursera.org/learn/social-science-research-chinese-society

审阅

Filled StarFilled StarFilled StarFilled StarHalf Faded Star

4.5（17 个评分）

5 stars

13 ratings

3 stars

3 ratings

2 stars

1 ratings

从本节课中

Challenges

In Week 6, we will focus on Challenges. By the end of this week, you should be able to have a better understanding of key challenges to interpreting results from social science research, and be able to reflect on potential problems with study design.

教学方

Cameron Campbell

Professor of Social Science

脚本

[MUSIC] Hi, in this module, I'm going to talk about the issue of validity of measures. That is the issue of whether or not the things we go out and measure for our study really capture the concepts that we are interested in our theory. Translating concepts into indicators that we can measure out in the real world actually turns out to be quite difficult. When we go out and identify a measure that we want to go out and collect in our study. The two key issues that come up are what we referred to as the validity of he measure and its reliability. A measure is said to have high validity, in this case, we're talking about the most broad definition of validity, construct validity. If it is somehow capturing or strongly related to the perhaps abstract concept of interest that is the subject of our study. So for example, if we're conducting a study to look at the relationship between egalitarianism and economic growth. We would be concerned about whether what we are going out and measuring in countries or regions. And claiming to be a measure of egalitarianism really captures important aspects of that abstract concept. We say that a measure has low contrast validity if it turns out to be unrelated to whatever our interest is in the study. When we talk about the validity of measures through our apecific types of validity that we are especially concerned with. One is content validity. Does our measure capture all of the meaningful dimensions of interest for the concept that is part of our theory or just selected dimensions? I'll give a more concrete example in just a little bit. Another aspect of validity is what we refered to as face validity, these are measure something that we could explain to a lay person. And those that have some sort of common sense appeal where it's obvious that it indeed should represent the concept in which we're interested in. Finally, criterion validity. Is the measure that we are offering correlated with other measures that have already been validated in other studies. So for example, if in studying a particular concept, there have already been a number of studies that make use of different measures. That are widely accepted as measuring that concept, are there studies or can we prove that the measure we propose is correlated or related to those other measures. Let's talk about a concrete example. Blood pressure is a measure of health. You've probably had your blood pressure measured. In fact, I hope you had your blood pressure measured. And then studies of health, it's very common to collect information on blood pressure as a measure of health. So we can talk about how it fits with these different types of validity. Now there's content validity. Certainly, blood pressure is excellent with respect to content validity when it comes to cardiovascular health. Whether it's content validity for health overall is adequate is another question. Health has many dimensions. Emotional health, mental health, there are other organ systems, the liver and so forth. Where blood pressure may not really capture all of the dimensions of health in those other domains. Face validity. Well, most of us have had our blood pressure taken. Our doctors have told us that it's very important. So it seems commonsensical that blood pressure should be a measure of cardiovascular health. And finally, criterion validity have studies established that blood pressure is indeed related to other measures of cardiovascular health. Or to the rate of dying or the rate of developing heart disease. Or it turns out there have been studies going back over decades that have made the relationship of blood pressure to cardiovascular health very clear. Now another aspect of validity is what we refer to as internal and external validity. This typically refers to situations when we're talking about experiments but they have more broader application. Internal validity refers to the measure and the design of the study and the aspects that are helpful in figuring out whether indeed we have evidence of cause and effect. We say something has high internal validity if we're confident that the measures and our way of analyzing them really will give us insight into cause and effect. External validity refers to the ability of our measures to generalize to a larger population and the ability of our study to generate results to a larger population. In fact, it turns out there are trade-offs, where sometimes in the quest for internal validity to prove cause and effect. We may make use of data that's so unusual or so specialized in terms of the population it covers. That we become worried about external validity. Another thing that we're interested in when it comes to measurement is reliability. Reliability refers to the consistency of the results of a measure across repeated applications. Does the same measure yield the same result every time you carry it out. Or are the results highly dispersed? This is an important issue. The implications are several. One is that if individual measurements are prone to substantial random error, that can affect the analysis. If there's a lot of error, then that may obscure relationships between the groups being compared or relationships between variables of interest. Measurement error in particular may make relationships between variables appear weaker than they are actually are. It's a bit technical and we won't be able to get into the details to why that's the case just here. So how do we reduce measurement error? Researchers go to a lot of trouble to minimize measurement error and increase reliability. When people are designing questionnaires, they pay careful attention to the design of the questions to maximize their clarity. And then for outcomes of interest that vary naturally, researchers make multiple measurements and then they use perhaps their average in an analysis. And then they might also specify a protocol for the collection of the measure to try to again, isolate or screen out other sources of variation that create error. Lets think about some examples to help make this a little more concrete. One is blood pressure which I've been talking about. It turns out that blood pressure varies a lot for the same person across the course of the day. You've probably noticed this when you've been in the doctor's office. Or on other occasions when you've measured your blood pressure. The readings can vary greatly over the space of a few minutes. So a single measurement of blood pressure is not typically considered reliable. Again, because measurements vary a great deal. Doctors often warn diagnosed high blood pressure until they've made multiple measurements. So when people are collecting blood pressure in a survey as a part of study of health. They may make repeated measurements of blood pressure and then take the average as the one to be used in the analysis. And they may also specify a very specific criteria for when to collect blood pressure in under what circumstances to account for known variation across the day in levels of blood pressure. Another issue is income, we've talked about it previously, so I won't talk about it in detail here. But income turns out to be very hard to measure. Except in very special cases where people have fixed incomes and salaries. If you talk about somebody engaged in agriculture or a day laborer, their reported income may vary a lot from day to day depending on how business was. So people have to think carefully about to measure income. Whether it's over the year, the month or even the last week. Whether it should be averaged across multiple observations. Finally, there's the example of time allocation. It turns out that when we conduct studies where we ask people how they've been using their time. How often they've been engaging in certain types of activities, the responses can vary a lot, from week to week, from day to day. So people have to think about how to average responses across multiple surveys to get a more reliable measure. So fluidity and reliability are two issues that we're especially interested in when it comes to measurement. And these are things that social science researchers pay a great deal of attention to when they design other studies. because if you don't get the validity and the reliability right, the results of your study may not yield useful conclusions about the theory of interest.