Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from 2000, rescored in 2005, by block and item, grade 4 science combined national and state main assessment: 2005

Block

Item

Range of response codes

Sample size

Percent exact agreement

Cohen's Kappa

Intraclass correlation

S3

K031001

1–3

3,500

94

†

0.88

K031002

1–3

3,900

85

†

0.78

K031003

1–3

3,700

83

†

0.65

K031004

1–2

4,200

88

0.74

†

K031005

1–3

4,000

78

†

0.68

K031006

1–3

3,400

88

†

0.89

K031007

1–3

4,100

83

†

0.85

S4

K031101

1–2

3,100

97

0.88

†

K031102

1–2

2,500

95

0.87

†

K031103

1–2

2,700

93

0.86

†

K031104

1–2

2,900

98

0.90

†

K031105

1–3

2,800

99

†

0.97

K031107

1–4

2,900

92

†

0.95

S6

K031301

1–4

2,800

91

†

0.87

K031309

1–4

2,500

90

†

0.90

K031302

1–3

2,800

89

†

0.54

K031303

1–3

2,400

89

†

0.91

K031304

1–3

3,500

92

†

0.92

† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2005 Science Assessment.