Range of response codes, percent exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 4 U.S. history assessment, by item and block: 2006

Block

Item

Range of response codes

Sample size

Percent exact agreement

Cohen's Kappa

Intraclass correlation

H3

H065701

1–4

500

77.47

†

0.72

H065801

1–4

500

97.98

†

0.99

H066001

1–3

400

77.23

†

0.75

H066501

1–4

400

97.05

†

0.98

H066701

1–3

400

93.98

†

0.90

H4

H054301

1–3

500

86.42

†

0.87

H054601

1–3

500

91.19

†

0.88

H054801

1–3

500

90.82

†

0.91

H054901

1–3

500

94.94

†

0.95

H055301

1–3

400

91.33

†

0.91

H5

H055901

1–3

600

90.23

†

0.91

H056101

1–3

400

93.43

†

0.93

H056401

1–3

500

95.84

†

0.96

H056601

1–3

500

95.26

†

0.95

H056801

1–3

500

97.40

†

0.95

H6

H067401

1–3

500

91.72

†

0.91

H067701

1–3

500

95.73

†

0.96

H068001

1–4

500

83.84

†

0.83

H068201

1–3

400

94.64

†

0.93

H068301

1–3

400

92.00

†

0.90

H068501

1–3

300

87.19

†

0.83

H7

H057501

1–3

500

97.35

†

0.96

H057701

1–3

500

87.87

†

0.83

H057801

1–3

400

88.50

†

0.84

H058601

1–3

400

84.44

†

0.85

H058701

1–3

400

82.92

†

0.80

H8

H034101

1–4

500

96.13

†

0.82

H034401

1–3

500

96.22

†

0.95

H034501

1–2

500

99.36

0.96

†

H034702

1–3

400

98.41

†

0.93

H035001

1–3

400

86.17

†

0.84

H035101

1–3

500

88.24

†

0.90

† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2006 U.S. History Assessment.