Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous years, rescored in 2005, by block and item, grade 8 mathematics combined national and state main assessment: 2005

Block

Item

Range of response codes

Sample size

Percent exact agreement

Cohen's Kappa

Intraclass correlation

M3

M075401

1–3

1,200

93

†

0.91

M075601

1–3

2,000

94

†

0.96

M019901

1–3

400

99

†

0.97

M047301

1–4

400

99

†

0.97

M020101

1–2

800

99

0.98

†

M076001

1–5

2,000

82

†

0.88

M4

M046001

1–5

400

99

†

0.97

M046901

1–5

600

99

†

0.98

M046601

1–4

1,000

96

†

0.92

M046801

1–5

800

99

†

0.99

M066601

1–3

800

92

†

0.94

M067201

1–3

1,000

73

†

0.76

M068003

1–3

400

99

†

1.00

M068005

1–3

600

93

†

0.88

M068008

1–3

1,400

89

†

0.84

M068006

1–3

2,400

91

†

0.92

M093601

1–3

1,000

97

†

0.96

M053001

1–2

600

89

0.72

†

M086301

1–5

1,400

92

†

0.96

M5

M085701

1–3

600

96

†

0.96

M085901

1–3

1,800

95

†

0.95

M019701

1–2

400

99

0.98

†

M020001

1–2

200

97

0.94

†

M066501

1–3

1,000

94

†

0.96

M066301

1–3

1,400

96

†

0.89

M067901

1–3

600

97

†

0.96

M047901

1–3

400

99

†

0.98

M053101

1–5

1,800

86

†

0.94

M6

M141301

1–4

1,100

923

†

0.97

M141601

1–2

3,300

85

0.68

†

M141901

1–5

2,400

90

†

0.96

M7

M144201

1–2

400

99

0.99

†

M144601

1–3

1,100

93

†

0.94

M144901

1–4

2,500

90

†

0.96

M145101

1–4

1,300

96

†

0.94

M8

M021001

1–2

400

100

1.00

†

M020901

1–2

600

89

0.77

†

M092401

1–3

800

95

†

0.91

M092601

1–3

800

96

†

0.94

M073601

1–5

2,200

81

†

0.94

M9

M013131

1–2

400

98

0.96

†

M073501

1–3

900

97

†

0.95

M052401

1–2

1,500

89

0.77

†

M075301

1–3

400

96

†

0.94

M072901

1–3

1,000

93

†

0.94

M075801

1–3

1,700

82

†

0.86

M093801

1–5

1,000

91

†

0.96

M10

M142801

1–2

500

99

0.98

†

M143201

1–2

500

99

0.98

†

M143501

1–5

3,200

82

†

0.94

M11

M106301

1–2

400

99

0.99

†

M106901

1–2

2,600

84

0.66

†

M107501

1–2

400

99

0.99

†

M12

M110201

1–2

600

98

0.93

†

M110501

1–3

1,400

90

†

0.91

M111801

1–3

2,800

87

†

0.90

† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2005 Mathematics Assessment.