A Comparison of the MEAP, SAT-9, and ITBS

The past generation of students in Michigan have taken the Michigan Educational
Assessment Program (MEAP) test to gauge their academic competency in five basic
subject areas: mathematics, reading, science, social studies, and writing.

The MEAP test is not the only option for the Michigan legislature to consider
using. A number of commercial firms publish high-quality, low-cost educational
achievement tests that could potentially replace the MEAP test. The following
compares the MEAP to two commercial achievement tests: the Stanford Achievement
Test, Ninth Edition (SAT-9) and the Iowa Test of Basic Skills (ITBS), and
analyzes the benefits and costs of each test.

The Michigan Educational Assessment Program (MEAP)

Some 25 years ago, the Michigan Revised School Code and the later-passed State
School Aid Act of 1979 determined that school standards and accountability
testing be used to gauge the academic achievement of all Michigan students.
Throughout time, the MEAP has been adjusted to conform to the state's
educational standards.1

As noted above, the MEAP exams are given to students in five basic subjects for
students in grades 4-11, as follows:

An advantage of the MEAP is that it is aligned with state standards, and uses
performance levels to grade the outcome. For example, in the eleventh grade
(high school) test the MEAP uses four levels to determine proficiency: Exceeded
Michigan Standards (Level 4), Met Michigan Standards (Level 3), Basic (Level 2),
and Unendorsed (Level 1). These performance levels give parents and policymakers
more understanding about what the test scores mean.

The Michigan standards themselves, though, have been roundly criticized for
being unduly vague and broad.2 Well
structured education standards should be focused on year-by-year learning, be
rigorous, intelligible, and measurable, so that teachers and parents alike know
what is expected each year. The Michigan Department of Education should consider
reviewing California as a model of quality math and reading standards.3

Another issue is the cost of the MEAP exams. According to a report from noted
Harvard economist Caroline M. Hoxby, the costs per pupil are lower in Michigan
than in other states; however, this is partially because of the infrequency of
the exams.4 Also, students only begin
MEAP testing in the fourth grade, so there is no objective way of ascertaining
how well a child, school, or school district is performing, compared to his or
her peers in other parts of the state, until then.

Further, there is no ability to gauge year-to-year changes in subject
performance. If a particular student is doing well in fourth grade reading,
there is no objective measure of reading mastery again for that student until
the seventh grade. If that student's reading achievement had begun to slip in
the fifth grade, for example, this objective measurement of achievement would
not be able to help his or her teachers zero in on the nature of the deficiency.

Under new federal standards (see below), and under legislation pending in the
Michigan House, some of these shortcomings will be remedied - and the cost will
rise - as more grades are tested.

There are a variety of popular commercial tests that can be used to gauge the
academic achievement of students. Two of the more popular ones are the Stanford
Achievement Test, Ninth Edition (SAT-9), published by Harcourt Educational
Measurement, and the Iowa Test of Basic Skills (ITBS), published by Riverside
Publishing/Houghton Mifflin. Both companies offer tests in a number of subject
areas, including reading, mathematics, language, science, and social studies,
among others, and these are packaged into a comprehensive battery of tests.

These are both norm-referenced tests; that is, the score results are
standardized into an easily-understood and comparable fashion. Both tests report
scores as percentile ranks. A student who scores at the 50th percentile has a
higher level of achievement than 50 percent of his or her peers nationwide. This
has an advantage over a state-based test in that it allows the state as a whole
to compare itself to the nation in terms of academic achievement.

School officials may also ask for performance standard scoring. The SAT-9 offers
four levels of performance (Level 1 through Level 4), which look analogous to
the Michigan performance levels (below satisfactory, partial mastery, solid
performance, and superior performance for Levels 1-4, respectively).

Both of these tests use nationally-recognized curriculum standards to draft the
test questions. For example, the SAT-9 uses criteria from the National
Assessment of Educational Progress for its reading test, the National Science
Education Standards from the National Research Council (a division of the
National Academy of Sciences) for its science test, etc.

A number of states use one of these tests as their state benchmark test. For
example, at least part of the SAT-9 is used by Arkansas, Arizona, California,
Delaware, the District of Columbia, Utah, and West Virginia as their state
assessment.

There are at least five major advantages for Michigan in using a commercial
test such as the ITBS or SAT-9.

1. Flexibility: When the comprehensive battery of tests is purchased, it comes
with a range of tests that may be chosen by the state to use. For example, the
ITBS complete battery of tests include reading, language arts, mathematics,
social studies, and science, among others. School administrators and
policymakers may pick and choose which assessments will be given.

2. Comparability: The percentile ranking scores that are returned can show how
well the entire state is performing in these subjects compared to the nation as
a whole. The MEAP lacks any kind of national comparability.

3. Political Neutrality: Michigan and many other states experience considerable
political turbulence over K-12 curriculum standards and test content and
methodology. The number of interest groups to satisfy is immense, the debate
heated and emotional, and the stakes significant, especially when school funding
and student scholarships are contingent upon the results. Adopting a widely
respected commercial test can avoid many political battles, and may even
increase public confidence in the state's performance assessment.

4. Value: By one calculation, it costs up to $22 per student to administer and
score the MEAP exam. If a set of reusable test booklets are purchased for the
population of Michigan public school children in grades 4-8, for example (almost
640,000 students5), there would be a one-time
cost of approximately $2.8 million ($4.40 for each test booklet for the SAT-9 -
the ITBS has a similar cost structure). Machine-readable answer sheets would add
another half million dollars each year (about $0.81 per sheet). Scoring the
tests each year would add between $1.95 and $5.90 per student (between $1.25 and
$3.78 million per year), depending on the reports desired by the state.

In all, the program would cost at most $7.1 million the first year and not quite
$4.5 million in subsequent years, assuming that Michigan did not negotiate any
kind of scale discounts from the publisher. This equates to slightly more than
$11 per tested student in the first year and $7 thereafter.6
A study of Georgia's testing system showed similar savings to that state if it
used a private test.7 These estimates are
in line with other states that use a commercial test. For example, West
Virginia, which uses the SAT-9, spends about $12 per student on its testing
program each year.8

5. Federal Law: The "No Child Left Behind" bill, signed into law last January by
President Bush, mandates that all states show yearly progress in improving
academic achievement. The nature of the SAT-9 or ITBS percentile rankings could
probably be used by states to show such progress, especially since a number of
states already use either of these tests.

There are also two disadvantages the Michigan legislature will have to weigh in
choosing a commercial test as opposed to the MEAP.

1. Standards: The state of Michigan would lose the ability to match its own
academic standards to the test used. This may not be a significant disadvantage,
given the earlier criticism of Michigan's standards.

2. Long-term Use: The state would, essentially, be "starting over" in its
educational assessment program. If the MEAP, which has been used for decades
now, is replaced, little if any of the information gathered from the decades of
testing could be used to help guide educational decision makers. There would be
no way the MEAP test scores could be accurately compared to SAT-9 or ITBS test
scores because they are different tests with at least slightly different
standards and objectives.

In the debate in what to do about assessing educational achievement, Michigan
has a number of options. The state could either keep its long-standing
achievement test program, the MEAP, or explore the advantages of using a private
test, such as the SAT-9 or ITBS. The question is whether the Michigan state
legislature is willing to lose some control in order to get more value and
flexibility out of its academic testing program.

_________________________________________

Stay Engaged

Receive our weekly emails!

email address

1 More of the history and design of the MEAP test may be found at
"Michigan Merit Award & MEAP Information" [WWW Document] at
http://www.meritaward.state.mi.us/mma/design.html.2 For example, see Lance T.
Izumi, Developing and Implementing Academic Standards: A Template for
Legislative & Policy Reform (San Francisco: Pacific Research Institute for
Public Policy), January 1999.3
These may be viewed online at http://www.cde.ca.gov/be/st/.4
Caroline M. Hoxby, "The Cost of Accountability" Working Paper [WWW Document] at
http://post.economics.harvard.edu/faculty/hoxby/papers/costofac.pdf. She
estimates that the cost per student is less than $7, but the actual cost is
probably closer to $22 per MEAP-eligible pupil because only those public school
students in grades 4-8 and 11 are tested, instead of students from all grades.5 National Center for Education Statistics, U.S. Department of Education,
Public School Student, Staff, and Graduate Counts by State: School Year 2000-01 NCES 2002-348, April 2002, Table 1. 6
If additional grades are assessed, the proportionate cost of the test would
remain roughly the same, at a little more than $11 per student for the first
year.7
Franklin Shumake, "Why Georgia Needs a New Approach to Testing" Georgia Public
Policy Foundation Commentary, November 8, 1996 [WWW Document] at
http://www.gppf.org/article.asp?RT=5&p=pub/Education/testing.html.8 Caroline M. Hoxby, "The Cost of Accountability" Table 2, pg. 17.