Presentation Description

No description available.

Comments

Posting comment...

Premium member

Presentation Transcript

Reliability and Validity:

Reliability and Validity

THE GREAT TRIAD :

Reliability – Does the instrument measure accurately and consistently? Validity- Does the instrument measure what it says it measures? Objectivity- Is the instrument immune to the personal attitudes and opinions of the researcher? THE GREAT TRIAD

RELIABILITY :

RELIABILITY The degree to which test scores are free from errors of measurement. The degree of consistency with which an instrument measures what it is measuring.

RELIABILITY AND ERROR :

Error Variance Systematic Error – there is a system (methods are planned, orderly, and methodical) Unsystematic Error – there is a lack of a system (occurrences are presumed to be random) RELIABILITY AND ERROR

3 IMPORTANT MEASURES OF RELIABILITY :

1. Coefficient of Stability How stable is it over time? 2. Coefficient of Internal Consistency Is the test internally consistent? Coefficient of Equivalence Is the test the same/different than others? 3 IMPORTANT MEASURES OF RELIABILITY

Standard Error of Measurement:

AKA: SEM Every score has 2 parts: True Score + Error The SEM estimates the error and places a band around the given score. The true score will fall within the range. This is referred to as a confidence band. Standard Error of Measurement

SEM Practice:

If the SEM is 6 and the person’s score was 90, we would say that his true score would be between 84 and 96. 90 – 6 = 84, 90 + 6 = 96. If the score was 112 and the SEM was 5, what would the Confidence band be? 107 – 117. SEM Practice

Concerns:

High coefficients of stability does not mean that internal consistency is high. Reliability coefficients apply to data and not to instruments. Reliability estimates consistency Concerns

VALIDITY:

VALIDITY The degree to which all of the accumulated evidence supports the intended interpretation of test scores for the intended purpose The ability of research instruments to measure what they say they measure

Internal Invalidity:

Additional Problems:

Hawthorne Effect – If subjects know they are part of an experiment – or if they are given more attention because of the experiment – their performance sometimes improves. Halo Effect – The halo effect occurs when a trait that is not being evaluated influences a researcher’s rating on another trait. Rosenthal Effect – The experimenter’s beliefs about the individual may cause the individual to be treated in a special way so that the individual begins to fulfill the experimenter’s expectations. Reactive Effect – The very presence of the researcher can influence behavior. Additional Problems

External Invalidity:

Ask the question, “How confidently can I generalize my experimental findings to the world?” Concerned with the IV External Invalidity

Observation Obstacles:

Measurement Types:

Nominal Measurement Variables which are categorized into discrete groups. Ordinal Measurement Variables which are rank ordered. Interval Measurement In order to measure distance between data points, we need a scale of equal, fixed gradation. No meaningful zero! Ratio Measurement The same as interval measurements except... With a meaningful zero! Measurement Types