1Quasi-Experimental Design 2Quasi-experiments provide a good model for the kind of detective work and creative problem solving that should be used in any kind of research. 3

One Group Post-Test Only Design

X O

The simplest and the weakest possible design

Lack of a pretest prevents assessment of change

Lack of a control group prevents threats from being ruled out.

4One Group Post-Test Only Design X O This design highlights the importance of contextual knowledge when interpreting results. Some research with weak designs can be convincing because of the background expectations against which the results are judged. Example Why was Milgrams obedience research so convincing? 5One Group Post-Test Only Design X O Without changing the basic nature of this design, it can be improved considerably by adding additional outcome measures O1 X O2 O3Compared to norms or expectations, only O2 should be unusual. 6Post-Test Only Design with Nonequivalent Groups X O O This design is an improvement, but the lack of a pretest means that any posttest differences could be due to selection. How do we know if the groups are equivalent? 7Post-Test Only Design with Nonequivalent Groups X1 OX2 OX3 OX4 O OWhy is this a stronger version of the same basic design? 8

One-Group Pretest Post-Test Design

O X O

This very common applied design is susceptible to all threats to within-groups comparisons

History

Maturation

Testing

Regression

Instrumentation

9

One-Group Pretest Post-Test Design

O X O

Threats in this design might be ruled out with logic or common sense (e.g., regression).

Some maturation threats might be ruled out by correlating the pretest with other available maturation variables (e.g., age).

Attrition after the pretest can create additional selection problems.

10One-Group Pretest Post-Test Design O X O One powerful modification is to add pretests O O O O O X O Maturation threats can now be examined and their influence separated from treatment effects. 11O O O O O O O O O X O 12Untreated Control Group Design with Pretest and Posttest O1 X O2 O1 O2 This design is stronger than the previous three because it provides both kinds of comparisons within-groups and between-groups. The major threat is selectionthe possibility that any outcome differences are due to pre-existing differences between the groups. 13Untreated Control Group Design with Pretest and Posttest O1 X O2 O1 O2 This design highlights the importance of outcome patterns. Some patterns of outcome are implausible as arising from some common threats. Others are quite plausible and require design modification to eliminate. 14Treatment OutcomeControlPretestPosttestThis pattern might be interpreted as evidence for the effectiveness of treatment, but some plausible threat-based explanations would first have to be ruled out. 15This pattern could arise from differential maturation in the two groups. Internal analyses might be able to rule that out. In an internal analysis, data from the study are used to diagnose problems that threaten the main purpose of the study. 16TreatmentOutcomeControlPretestPosttest 17The pattern might also arise from instrumentation problems that affect the two groups differently.OutcomeTreatmentConstructControl 18The pattern might also arise from differential statistical regression.ControlTreatment

The treatment group is selected from the extreme end of the distribution to make participants as similar as possible to the average control group participant. 19Treatment OutcomeControlPretestPosttestThis design and outcome might be typical when the purpose of research is to reduce some problematic behavior to the level of a comparison group. 20Two alternative explanations are plausible. This pattern might occur because of instrumentation problems (a floor effect that prevents the control group from decreasing) and differential maturation. Perhaps the treated group is simply composed of late bloomers who would shift their behavior on their own in the absence of treatment. How would either be ruled out? 21ControlOutcomeTreatment PretestPosttestCompensatory designs suffer from the same problems. These designs, with large pretest differences, are also susceptible to regression problems. How can those be ruled out? 22Treatment ControlOutcomePretestPosttestThis outcome pattern effectively rules out instrumentation problems. Why? 23ControlTreatmentOutcomeControlTreatmentConstruct 24This outcome pattern is unlikely to arise from regression, unless the treatment group was selected from the lower end of thatgroups distribution and the mean is higher than the distribution from which the control group was selected. ControlTreatment

25This pattern would not likely arise from a simple maturation process.TreatmentOutcomeControlAge 26Untreated Control Group Design with Proxy Pretest Measures OA1 X OB2 OA1 OB2 In this design, the pretest and posttest are different measures. This may be necessary if the posttest cannot be given as a pretest or if there is concern that it might sensitize participants if it was given as a pretest. 27Untreated Control Group Design with Proxy Pretest Measures OA1 X OB2 OA1 OB2 Example The study might investigate the impact of a new curriculum on learning of algebra. It might make no sense to give an algebra test before the introduction of the treatment because there might not be any variability. 28Untreated Control Group Design with Proxy Pretest Measures OA1 X OB2 OA1 OB2 Instead, a measure that correlates with the ability to learn algebra (e.g., math aptitude) could be given. The hope is that the proxy pretest will allow the two advantages of covariance analysis. 29Unadjusted differenceAdjusted differenceAlgebra AchievementError is also reducedControlTreatmentMath Aptitude 30The major problem with the proxy pretest design is that the proxy is an imperfect measure of the construct that should be controlled. Consequently, the covariance adjustment may not be adequate. 31

Inadequate covariance adjustment due to a faulty or unreliable pretest can

Produce a spurious treatment effect

Pretest PosttestPretest PosttestWithout adjustment or with poor adjustment of pretest differencesWith adjustment of pretest differences 32Inadequate covariance adjustment due to a faulty or unreliable pretest can (b) Fail to uncover a real treatment effectPretest PosttestPretest PosttestWithout adjustment or with poor adjustment of pretest differencesWith adjustment of pretest differences 33The Big Picture Threats can arise in any design in many and sometimes subtle ways. Controlling those threats requires using a variety of tactics to render threats less plausible, if they cannot be eliminated entirely. These tactics include considering the patterns that might indicate the presence of threats, design modifications, and statistical controls. 34

The ability to recognize and solve threats to internal validity is critical to conducting good research, for everyone

An intended experiment fails to accomplish one of the key defining features

An experiment cannot be conducted for practical or ethical reasons

A consumer of research needs to know what conclusions can be trusted

35(No Transcript)

About PowerShow.com

PowerShow.com is a leading presentation/slideshow sharing website. Whether your application is business, how-to, education, medicine, school, church, sales, marketing, online training or just for fun, PowerShow.com is a great resource. And, best of all, most of its cool features are free and easy to use.

You can use PowerShow.com to find and download example online PowerPoint ppt presentations on just about any topic you can imagine so you can learn how to improve your own slides and
presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!