Range of response codes, percentage exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous assessment years, rescored in 2003, by block and item, grade 4 reading combined national and state main assessment: 2003

Block

Item

Range of response codes

Sample size

Percent exact agreement

Cohen's Kappa

Intraclass correlation

R3

R017001

1–2

1,400

88

0.74

†

R017003

1–3

2,000

86

†

0.86

R017004

1–2

1,400

94

0.85

†

R017006

1–2

1,600

92

0.78

†

R017007

1–4

2,200

73

†

0.86

R017009

1–3

3,000

85

†

0.78

R4

R012102

1–2

600

94

0.88

†

R012104

1–2

1000

93

0.86

†

R012106

1–2

1,000

86

0.72

†

R012109

1–2

200

89

0.74

†

R012111

1–4

3,000

89

†

0.92

R012112

1–2

1,000

87

0.73

†

R012113

1–3

800

88

†

0.87

R5

R012601

1–2

1000

85

0.68

†

R012604

1–2

2,000

93

0.84

†

R012607

1–4

3,600

80

†

0.83

R012612

1–2

3,000

83

0.67

†

R6

R017301

1–2

2,400

90

0.80

†

R017303

1–3

2,000

86

†

0.88

R017307

1–4

4,200

77

†

0.86

R017309

1–3

3,200

83

†

0.84

R017310

1–3

2,000

85

†

0.84

R7

R012702

1–2

1,000

91

0.79

†

R012703

1–2

800

87

0.74

†

R012705

1–2

2,000

91

0.78

†

R012706

1–2

1,200

80

0.56

†

R012710

1–2

600

89

0.76

†

R012714

1–4

3,800

80

†

0.82

R8

R017401

1–3

2,000

83

†

0.85

R017701

1–3

1,200

85

†

0.87

R017901

1–4

4,400

79

†

0.89

R018201

1–3

3,400

85

†

0.87

R018301

1–3

2,800

84

†

0.85

R9

R015803

1–3

1,800

85

†

0.83

R015804

1–4

3,200

78

†

0.81

R015806

1–3

2,600

81

†

0.81

R015807

1–3

1,400

82

†

0.81

R015809

1–3

1,800

87

†

0.87

R015810

1–3

2,200

88

†

0.87

R10

R012503

1–2

600

87

0.75

†

R012504

1–2

600

97

0.93

†

R012506

1–2

600

89

0.78

†

R012508

1–2

600

97

0.94

†

R012511

1–2

600

93

0.85

†

R012512

1–4

2,800

76

†

0.86

R11

R020401

1–3

3,200

86

†

0.88

R020701

1–2

800

88

0.77

†

R021201

1–3

2,400

84

†

0.86

R021701

1–3

2,200

79

†

0.81

R12

R023201

1–2

1,200

91

0.82

†

R023501

1–3

2,200

85

†

0.89

R023601

1–3

1,200

84

†

0.87

R024001

1–3

3,600

80

†

0.82

†The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items.
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2003 Reading Assessment.