Know when we would expect to see a normal distribution (NRT, or certain CRTs), positive skewness, or negative skewness

A Norm-Referenced Test (NRT) can be expected to have a relatively normal distribution with skewness and kurtosis relatively close to zero and the measures of central tendency close together.

However, normal distribution is not expected in a Criterion-Referenced Test (CRT).

A pre-test will have strongly positive skewness (with few high scores)

while a post-test, given after students should have mastered the content, can be expected to have a negative skewness (with many high scores).

The only times we should expect to see a normal distribution in a CRT are in cases where students at many different levels are taking the same test (e.g. in placement and proficiency testing). (Carr p. 231).

POSITIVE = peak is on the left

NEGATIVE = right peak.

correlation

the degree of relationship between two sets of numbers.

correlation coefficient

the mathematical estimate of the strength of relationship between two sets of numbers.

categorical/nominal variable

variables that can be grouped into categories, but the categories cannot be put in any sort of meaningful order(e.g. gender, native language, country of origin, and favorite color). All we can do is to count the number of people in each category.

ordinal variable

variables where different levels, ordered categories essentially, can be observed, and put into

interval variable

variables which are arranged in order, as with ordinal data, but are equal distances between each point on the scale.

continuous variable

a variable that can have any value within a particular range, with the range typically being fairly large.

ratio variable

similar to interval variables, and that there is an equal distance between each pair of points on a scale, the ratio variables also include a true zero point. Somewhat rare and language learning research and testing.

correlation coefficient used in ordinal data, or for badly non-normal interval variables.

Know whether Pearson r or Spearman r would be more appropriate for correlating a particular set of variables (e.g., essay scores, scores on two long tests, test score rankings, scores on different item-based sections of a test, speaking test scores).

earson R is used for correlations that are more linear and concrete; thus, it would be good for discrete-point, itemized tests.

Since Spearman Rho is less linear focused (monotopic relationships), it would be best for subjective scores like essays and speaking test scores.

difference index

subtractive CRT discrimination index that requires giving the test twice, wants to Masters and wants to non Masters. Can be calculated as part of a differential group study or intervention study. As a rule of thumb, the D I should be. .40

item f

s a measure of association for two binary variables. Introduced by Pearson, this measure is similar to the Pearson correlation coefficient in its interpretation (Pearson R). In fact, a Pearson correlation coefficient estimated for two binary variables will return the phi coefficient.

differential groups study

approach to calculating the DI in which to existing groups are identified. One group is deemed to be at the mastery level in whatever ability is being assessed, and the other group is judged to still be non-masters

intervention study

essentially a pre-test/ post-test approach to calculating the DIi in which the test is administered to one group of students twice, before and after they have received instruction in whatever material is being assessed.