Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, grade 4 geography assessment: 2001

Block

Item

Range of response codes

Sample size

Percent exact agreement

Cohen's Kappa

Intraclass correlation

G3

G027201

1–2

700

100

0.99

†

G027301

1–2

700

99

0.99

†

G027401

1–2

700

95

0.93

†

G027501

1–2

700

98

0.97

†

G027601

1–2

700

97

0.96

†

G027701

1–2

700

94

0.91

†

G027801

1–2

700

96

0.94

†

G028201

1–4

700

91

†

0.97

G028401

1–3

700

89

†

0.93

G028501

1–3

700

94

†

0.95

G028701

1–4

700

91

†

0.95

G4

G008001

1–3

600

99

†

1.00

G008201

1–4

600

93

†

0.97

G008503

1–3

600

94

†

0.96

G008701

1–3

600

94

†

0.97

G009001

1–4

600

84

†

0.95

G5

G009201

1–3

600

95

†

0.97

G009402

1–3

600

96

†

0.95

G009403

1–3

600

95

†

0.93

G009601

1–4

600

94

†

0.98

G6

G029301

1–2

700

97

0.96

†

G029401

1–2

700

98

0.97

†

G029501

1–2

700

98

0.96

†

G029601

1–2

700

99

0.98

†

G029701

1–2

700

98

0.95

†

G030101

1–3

700

87

†

0.85

G030501

1–4

700

81

†

0.93

G030801

1–2

700

98

0.98

†

G030901

1–2

700

99

0.99

†

G031001

1–2

700

98

0.97

†

G031101

1–2

700

98

0.98

†

G031401

1–4

700

96

†

0.99

G7

G031801

1–3

600

96

†

0.97

G031901

1–3

600

95

†

0.97

G032401

1–3

600

100

†

1.00

G032601

1–2

600

98

0.97

†

G8

G012201

1–3

600

98

†

0.99

G012503

1–3

600

100

†

1.00

G012902

1–3

600

99

†

0.99

G013001

1–4

600

94

†

0.97

G013201

1–3

600

93

†

0.92

† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.