Our district faces a budget shortfall. The MAP® test has proven to be costly in ways beyond the initial subscription fee. Discontinuing the MAP® is one obvious way to save money and restore valuable class time, resources and learning to the children and teachers of Seattle Public Schools.

Some background

A subscription to the MAP® test was purchased by the Seattle School District from the Northwest Evaluation Association (NWEA) in 2009. (Then-Superintendent Goodloe-Johnson sat on the board of directors of the NWEA at that time and failed to disclose this fact. She was later cited for this breach of ethics by the state auditor. See below for more details.)

MAP® was used in some schools in the fall of 2008-09, and had a complete rollout in 2009-10. Every K through 9th grade student is now being administered the test three times a year (unless they opt out). [UPDATE: The pilot and rollout dates have been corrected from the original post. –sp.]

It is an “adaptive” test so it adjusts level of difficulty according to how the child answers the question. It is computerized, so unlike paper tests, schools require dedicated computers and lab space in order to administer it. It generates a lengthy report that teachers and administrators must be taught to interpret.

It is being administered to children as young as 5 in kindergarten some of whom are unable to read. So the test is given with headphones and read to the youngest children.

It only tests two subjects: Math and English.

The stated purpose of the test was to offer teachers a tool to gauge the academic levels of their students and adjust and differentiate their teaching accordingly. That in itself is not an objectionable objective. But that is not exactly how the MAP® has been used. It has proved to be a significant drain of time and resources, and of questionable usefulness.

Here are 15 reasons why the Seattle School District should discontinue the MAP®:

1. Lost class time. MAP® = lost opportunity for learning. Schools report spending as much as 9 weeks to three months of the year administering and analyzing/interpreting the MAP® test for hundreds of students. Seattle schoolchildren only spend 6.5 hours a day in school. The average American school public school year is 180 days — shorter than many other nations. Therefore, class time is precious. The thrice-yearly MAP® steals valuable time away from actual learning.

[UPDATE: The yearly subscription/licensing cost for MAP® was estimated to be $500,000 per year, according to SPS staffer Brad Bernatek and then-Broad Resident Jessica DeBarros in a report on April 2009.]

Also, most of the financial and logistic burden of administering and proctoring the test is falling to our schools who must give up precious space and staff time to administer it. Schools are effectively paying their librarians or other staffers to proctor and set up the tests. The cost of implementing the test in terms of administrator pay and purchasing computers is beyond the district’s means. This makes MAP® essentially an unfunded mandate.

3. MAP® creates unequal access and inequity. The MAP® requires designated computers and rooms to administer. Not all schools have computer labs, especially elementary schools, older buildings or those at full capacity and no spare space. Those that do not have computer labs are being forced to use their libraries or cafeterias to administer the test. Consequently libraries at a number of schools are off limits to students for normal use for as much as three months of the school year because of the MAP®.

[UPDATE: As many as 40 percent of Seattle’s public schools lose their libraries to MAP® testing for as much as three months of every school year, according to SPS’s Jessica DeBarros.]

So the MAP® is being used incorrectly despite having been sold to the community for another purpose – to help teachers understand the academic needs of their students. This has effectively made MAP® a high-stakes test in the Seattle School District. High-stakes testing has been repeatedly discredited by research and experts like Diane Ravitch and Yong Zhao. Such testing is shunned by some of the academically highest performing nations like Finland. This is not a best practice. It is a bad practice.

5. It is excessive. The MAP® test is being administered to Seattle schools kids in kindergarten through 9th grade three times a year. The test takes as much as an hour (or more) to complete, and hours to administer and process. For students in 4th grade and beyond who must also take the annual state MSP test (WASL replacement) this adds up to four high-pressure standardized tests a year. That is too much testing.

6. The MAP® test does not correspond to SPS curriculum. It is not aligned to our district curriculum, making it a poor fit for our district, rendering its relevance in question.

7. MAP® is narrowing the curriculum and leading to test-prep instead of teaching. Because teachers are feeling pressured to raise test scores, some are teaching to the MAP® test. This dilutes and distorts the curriculum for our children.

8. MAP® is inappropriate and unreliable for K-2. MAP®’s two main administrators for SPS, Brad Bernatek and Jessica DeBarros, told this to a group of parents in 2010, and said that other districts do not administer the test to these grades for that reason. (Indeed it seems excessive for such young children to be tested so rigorously three times a year.) Why is SPS the outlier, forcing its youngest students to take this test? Why should a kindergartener’s first library experience consist of a computerized test instead of the opportunity to check out a book?

9. MAP® is inappropriate for English Language Learners.

10. MAP® is of limited use for accelerated or advanced students (APP and Spectrum) because they very quickly hit the ceiling on the test. (It is also difficult to measure significant growth in these children’s levels and it would seem unreasonable to expect teachers to somehow raise high scores even higher, or penalize a teacher if a child’s score decreases, for example, from a 99 to a 98 percentage. Yet, this is happening elsewhere in the country.)

11. MAP® is not necessary. Many teachers are not finding MAP® that useful. A growing number of parents oppose it as well. There are alternatives like the Developmental Reading Assessment (DRA) and other less costly and less time-consuming assessments that teachers can administer to their students to gauge where they are at, academically at the beginning of the year. Good teachers already differentiate their lessons for their students wherever possible. The best teachers don’t need a standardized, computerized test to tell them whether their students are progressing or need help.

12. MAP® is not accurate. There have been reports of fluctuations in test scores from one session to the next, even within the scores of an individual child. The winter test of January 2009-10 reportedly trended district-wide so low as to render it unusable. This was attributed to the post-vacation slump (the test was given in January right after the holiday two-week break). If scores can be influenced by such outside factors as vacations, how can the data be accurate or useful? (And how can they be fairly used to evaluate a teacher?) This year, the district administered the test in Oct. then just 2-3 months later in Dec./Jan. and will administer it again 4-6 months later in May/June. That is inconsistent spacing of testing windows. It also seems irrational to expect significant difference in scores in the two months between the fall and winter tests.

[UPDATE: The MAP® test “recalibration” fiasco of Jan. 2012 – more evidence of the unreliability of MAP®

In January 2012, Seattle Public Schools families noticed that their children’s MAP® test scores had disappeared from the district’s database of student information. Then they reappeared, and some scores had changed from previous iterations, some dropping by as much as 20 points. Parents jammed the site anxiously searching for answers, and crashed it.

SPS district staff offered families this rationale:

February 1, 2012

Dear Seattle Public Schools families, As you may know, Seattle Public Schools uses a computer-based test called Measures of Academic Progress (MAP) to provide schools and families with information about student achievement. Schools administer MAP tests in math and reading two or three times a year to many of our students. (The fall MAP test is optional.) Teachers and principals use information from the MAP tests to monitor students’ academic progress and to design their instruction to help every student succeed. You have likely received your student’s MAP scores and percentile in the mail. These scores are also available online, via The Source, the online resource for families and teachers at http://source.seattleschools. org

As part of its ongoing efforts to follow best testing practices, the MAP test’s vendor, Northwest Evaluation Association (NWEA), recently recalibrated the percentile results associated with our students’ MAP scores, to better measure how Seattle students perform relative to students across the nation. This practice of recalibrating percentiles is common in standardized testing, and happens every several years.

The recalibrated percentile rank scores provide a more accurate snapshot of your student’s performance compared with other test-takers, since they are more representative of the national school-age population.

In order for you to be able to see your student’s academic growth over time more accurately, Seattle Public Schools has updated your student’s percentile scores for the past three years to reflect the recalibrated percentiles. You can find the new MAP scores online at The Source at http://source.seattleschools. org. You may notice only a slight change or a very significant change, depending on how your child scored compared to the larger national average of students taking MAP tests.

A student’s MAP results are reported using both percentiles and a RIT score. The RIT score shows what students are ready to learn rather than what they have already mastered, and is used to show a student’s current achievement on a scale that is independent of grade level. This percentile update does not change your child’s RIT score.

The recalibrated percentiles do not change eligibility for advanced learning (Spectrum or APP) students. Original scores from the Spring 2011 MAP test will still be used for eligibility for advanced learning for 2012-13. Next year, eligibility will be based on the recalibrated percentiles. If you have any questions or concerns about the updated percentiles or the MAP test, please contact research@seattleschools.org.

How can the district use a test based on a quicksand of shifting data points to measure student achievement or determine student eligibility for certain programs? How is it fair or accurate for the district to use constantly shifting test scores to measure, penalize or reward teachers?

The MAP® test recalibration confusion illustrated once again that the MAP® is an unreliable tool and should not be used for any high-stakes important decisions or evaluations.]

13. The manner in which the MAP® test product was selected and purchased is highly questionable. Seattle Schools Superintendent Goodloe-Johnson was on the board of directors of the vendor, NWEA, at the time the Seattle School District purchased the MAP® product, and failed to disclose this to the board or publicly, as is required. In 2010, the state auditor cited this as an ethics violation/conflict of interest and Superintendent Goodloe-Johnson was forced to step down from board of directors of NWEA.

Also, no bids from other vendors were sought. So it was a no-bid contract. A rationale for this lack of competitive bidding can be found here, from the former SPS employee once responsible for MAP® administration, Brad Bernatek.

The internal review of possible test products that led to the selection of MAP® was conducted by Jessica DeBarros of the Broad Foundation (who was subsequently hired by the district, at a salary of $90,000, to oversee the MAP® test). Superintendent Goodloe-Johnson was also affiliated with the Broad Foundation. She used the MAP® in her previous position as school superintendent in Charleston, South Carolina, so she may have favored the product already.

These factors have called into question the process and the motivations behind the purchase of MAP®, creating a level of mistrust associated with the test.

14. The test is being administered in an inconsistent and nonscientific manner. In some schools, librarians are being obliged to administer and proctor the tests, in other cases, other teaching staff are in charge. In still other cases, parents are proctoring the tests. How can this lead to consistent, accurate or scientific results? Is it fair to mandate a test that relies on parent volunteers?

15. The test can be gamed. Students can (and have) figured out that wrong answers will lead to easier questions and vice versa. This can lead to inaccurate and arguably useless results.

ADDENDUM: Troubling anecdotal evidence. A second grade child in the district’s accelerated/gifted program (APP) was reportedly asked the significance of the rose in The Scarlet Letter (clearly not an appropriate second grade book even for gifted 7-year-olds!). This is just one example of the MAP® asking inappropriate questions. A class of kindergarteners, unfamiliar with computerized tests in their first weeks of school at age 5, literally placed their mouse on the computer screen to follow directions that said “Put the mouse on your name.” The test is designed to give the children questions that are at times too hard for them. But this can lead to frustration and confusion. “Why am I being asked about something I’ve never studied?” my own child once asked me. There have also been reports of students in tears over the test. It is one thing to test one’s knowledge for the fun of the challenge, but to place a child before a computer and ask them to try to answer questions they are not expected to know seems bizarre at best, and almost cruel.

In short, MAP® has proven to be a costly and stressful misuse of precious resources. In times of financial scarcity, we need to ask: Is this the best use of our district’s limited resources? And, more importantly: Is this the best use of our children’s time?

–Sue Peters

More firsthand accounts and discussion of MAP among parents and teachers can be found here and here. [UPDATE: Still more concerns about the MAP can be found here and here. Jan. 2013]

My daughter’s school district has not been transparent with providing scores for the MAPS test. I’ve only seen my daughter’s scores at the beginning of the year. When I asked the teacher for her end of the year score all I received were two hand written scores with no detail.

I noticed that my daughter’s growth on the MAPS Math test from the beginning of the year to the end of the year was not realistic. She more than doubled the typical growth. So I asked my daughter why she scored so low at the beginning of the year. She said: “I could not hear out of the head set and no one would fix it for me”. Then the first time she took the test at the beginning of second grade she said “Some boy kept talking to her and no one would tell him to be quiet. The room was very noisy.” Hum…

If you are going to give a standardized test at least make sure you are administering it properly. The schools also need to be transparent with their findings and provide all results to the parents, not just what they feel like providing. If they are seeing huge amounts of growth or very little growth in the numbers, then they need to be honest about the test and discuss the problems.

I also find that giving a once a week in class room test in Reading, Spelling and Math as well as the MAPS standardized tests is way too much for children at the elementary level. Anyone who is defending the MAPS test at the elementary school level really needs to think back to their own childhood. Would you have enjoyed this amount of testing in elementary school? Really?

Special guest speaker Dr. Steven Strauss
Dr. Strauss is a Baltimore-area neurologist and a Ph.D. linguistics scholar. As a speaker and writer on education issues, he brings an anti-capitalist analysis and the passion of a parent with two children educated in public schools. In 2011 he marched with Save Our Schools in Washington, D.C., to protest corporatization and high-stakes testing.

Also featuring Matt Carter
Carter is a 4th/5th grade teacher and one of 12 staff members at Orca K-8 who are boycotting the MAP test. He is a shop steward for the Seattle Education Association and a member of Social Equality Educators. As the parent of a son and a daughter at Orca, he has opted them out of the MAP test as well.

Door donation $3, students $1, snacks served at 6:30pm for a $4 donation
Sponsor: Freedom Socialist Party

“7. MAP® is narrowing the curriculum and leading to test-prep instead of teaching. Because teachers are feeling pressured to raise test scores, some are teaching to the MAP® test. This dilutes and distorts the curriculum for our children.”
How? That’s your own conclusion. Teachers don’t even know what’s on the test and–in fact–are not allowed to review the test material.
Some of your other points are valid but I feel the commentary on some of them is your inappropriate extrapolation.

This is not a mere “extrapolation;” it’s what I’ve been told by teachers and parents. MAP only tests 2 subjects, math and English, and once you make that a measure of the teacher and have their job evaluations and possibly career hinge in part on these scores, that understandably can motivate the teachers to skew the classroom focus to the test and those two subjects at the expense of all others — especially in elementary school where one teacher is generally responsible for multiple subjects.

I have definitely heard from parents that some teachers have tried to prep the kids for MAP, or they have done “MAP work” in class before the test. Meanwhile, teachers have told me they have felt pressure from school administrators (principals) to “raise their kids’ MAP scores.” Ergo…

How can anyone prep for a test they are not allowed to see? Good question. So I would guess that teachers who do prep the class try to cover a little of everything in both subjects. That sounds somewhat futile and imprecise to me, which is all the more reason why we should not be giving our children a test that does not align to what they are being taught in class.

To those who love the MAP, would your students grow without it? My district uses the MAP, and my building had the most growth in reading for our grade level. The ironic part is, we never used Descartes. The discussion of growth needs to be prefaced by the idea that “growth” is a relative term, in the context of this particular assessment, separate from state standards.

It blows my mind how people who believe in the test: a. Know very little about it. b. Just want an “answer” to validate what they do in the classroom. c. Talk just like the NWEA reps.

I am so glad to see this post. Keep digging, people:). You’ll be amazed and horrified at what you find.

Our concern here in Seattle is the emphasis on testing. Our students are to take the MAP test three times each school year. Then in addition to that, they are to take a state test so that’s 4 tests in a school year.

Many of our schools do not have computer labs or additional computers to take the load of test taking so the library computers are used. The testing is taken by all grades within a one to two week period and the librarian is to assist in monitoring the test. So now there are one to two months of time that the libraries are closed over the school year.

So far in Seattle we have figured that it has cost us $11M to buy and implement this test. We have schools having to lay off counselors, nurses, librarians and even teaching staff due to budget shortfalls. Principals are having to make very tough decisions about how to use the limited resources that are available due to the lack of funding in the schools. I would say that this money could be better spent teaching rather than testing, supporting the students rather than cheating them out of time in the classroom.

Yes, I can see the value of such a test. One principal of a high school told me that they like to see where the incoming freshman are in terms of planning classes and curriculum. What has happened in the US is that we’ve gone from giving tests when they are appropriate to over testing which causes a shift in how teachers teach. In addition to that, teachers in the US are now being evaluated based on these tests in terms of pay or even being laid off. Principals are also being evaluated by these scores. A school can be closed, a principal fired or half of the staff fired over these test scores.

You can understand as a teacher how students vary in terms of ability and how well they do depending upon their family and financial circumstances. We see this as unfair pressure on the teachers who have so little support to begin with.

We as parents are very concerned that with this emphasis on testing, our students are not getting a broader education and an opportunity to explore subjects in there own way and at their own pace.

I would like to hear more about how the MAP test is used in China. How many times a year is it given? Are you evaluated by these test scores? Do the students use computers for the testing?

Honestly, this is a great tool! I have used it for 10 years in my classroom and I have even administered it. Is it perfect? No. Is it closer to perfect for students then having each teacher evaluate the students using their own method? Yes! True I can tell all about my students without a map test. Can I tell you all about my students individually in 30 specific areas and multiple subjects in a second? And tell you where they need to grow. I can with MAP. This makes my curriculum work better for my students. I see a lot of bitterness on this page and with bitterness anything can be evil. Guess what kids throw up for a multitude of different reasons. If that is a reason to stop something let us end PE or dances. It costs money. No argument there but it costs a fraction of the price for many districts to MAP test then it does to build a tank. You want help children, write a blog about why investing more money in the possibility war then the possibility of peace is ridiculous. I am currently teaching in China and we use it here too. Guess what? We use MAP testing. Why? It works if you use it correctly.

Are you a math teacher? It seems useful for that content area. However, just because it works there does not mean it works for all content areas. In my content area, science, I have been told by administrators the MAP is not ever going to be considered for use in our district.

Administrators as well as the federal government should keep their hands off all teaching tools. Although teachers I talk to like the MAP, I prefer the QRI just as another “set of eyes” on student progress. By no means should it be high stakes and the teachers wide range of assessments should be the dominating factor.

However, what I also see is the lack of an outcry to change the system as well as provide more support for teachers to take students from where they are. Students are still lumped into grade levels as an indication of academic achievement. Keep them in age groups but teach them from where they are using differentiated learning, centers, diverse projects, community experiences and proficiency based learning or whatever allows their skills to shine through. Students are still given meaningless letter grades based on a smaller version of those tests right in the classroom.

The advanced students must not be held back just because the major part of the class is at a certain level and those who are moving slower must not be pushed into an immoral failure pattern that forces the dummying down of grades or pushing kids out of school into a lifetime in the subclass. They could easily be smarter than the ones who score high on an artificial test.

Look at what is happening now and it has not changed over the last 80 years.
Find a way to make education individually appropriate and then exclude all tests that are not necessary including the multiple choice tests used in the classroom, especially if they are used to damage kids. No longer can we throw out learning and expect kids to catch it in a bushel basket.

What we are doing to kids is not just unethical, it is immoral! The challenge to all teachers is to take back their profession and provide education based on the reality that students learn in different ways and at different rates. And that is ok.

To change the system will be dramatic as many “dominoes” will fall and must be replaced with building blocks for success. Yes, we as educators will have to work harder than ever with no excuses. The only question is “Are the kids worth it?” We must all say yes.

In addition to students crying during the test, I had several students who had more serious bathroom accidents because of nerves and uncertainty of when they could ask to use the restroom. This type of high-stakes, high-pressure testing is WRONG!

“…I discovered then, in my early teaching career, that learning is best driven by ideas, challenges, experiences, and activities that engage students. My experience over the past 45 years has confirmed this.

We have come far from that time in the ’60s. Now the mantra is high expectations and high standards. Yet, with all that zeal to produce measurable learning outcomes we have lost sight of the essential motivations to learn that moved my students.

Recently I asked a number of elementary school students what they were learning about and the reactions were consistently, “We are learning how to do good on the tests.” They did not say they were learning to read.

It is hard for me to understand how educators can claim that they are creating high standards when the substance and content of learning is reduced to the mechanical task of getting a correct answer on a manufactured test.” (Summer 2009)

If the test works well for you, if you are a parent or a teacher, then great, use it in Wisconsin but here in Settle for it to be used three times during a school year is a misuse of a test particularly because the plan is to use it to evaluate teachers, something that the test was not designed for.

The MAP test, along with TFA and other policies, got pushed down our throats by our former superintendent. Now that she is gone we are re-evaluating all of her policies because she had an agenda that was not in the best interest of our students or our community.

We will figure out what’s best for our students. We have had enough of others landing in Seattle and telling us what’s best for us.

And about the “tons of information” I have herald that is the case, so much information that it is overwhelming to teachers. If we are going to provide our teachers with a “ton” of information three times each year then we also need assistants who can organize it and distill it down. From what I understand, not only is there information about how the student fared in a particular subject but also a course of action for each student.

Going though these “courses of action” three times each year for 30 students times however many classes a teacher has is way over what any human being should have to do when you add to that all of the other responsibilities that a teacher has.

We have the HSPE. Each teacher has a set of exams and quizzes that are administered to each student. I would suggest that we ask the teachers what they think. We have had the MAP here long enough for teachers to know if it works or not. Is it of value to them? So far, the answer seems to be a resounding “No”.

Cap Lee, we appreciate your input but we will figure out what’s best for our students.

Any assessment tool can be misused. However, my experience with this test is that it gives a ton of information to the teacher. This information can be used as a “jumping off” point to assure we are teaching all kids from where they are. We can no longer treat kids like robots where everyone is the same. Once we have this information,teachers take over with a wide range of assessments. The MAP, as well as other similar tests, are not perfect, and should never be used to judge.

This information is not designed to “teach to the test” but to teach children. There are no stepford kids. Use centers and differentiated learning after getting info from the MAP. And remember, this is only a snap shot in time. This is for information only. Dump the state test because it is worthless to kids. This test is valuabe for teachers and kids ….unless abused!

I don’t need an expensive program to give me tons of information about reading or math skills of my students. There are these absolutely amazing little devices called books. My room is full of them, and I know where I can get borrow more. I can use my ears to listen to a student read, (phonics, fluency, word recognition), and ask questions about the reading (comprehension). If the book is too easy, guess what I can do?

Cap Lee,
Do you really need a computerized test to prevent you from treating students like robots? Have you not or can you not use a myriad of information that comes from being a keen and active observer of your students to differentiate instruction? Have some faith in yourself.

1. Students cannot go back to check their answers, yet we have repeatedly instructed them to do so on ALL other tests.

2. One teacher cannot monitor an entire class to ensure that they are taking their time and thinking each question through with care. I had one student who completed the 52 questions on the math test in 5 minutes. That was not even enough time for me to get from one end of the room to another!

3. We have also instructed students, repeatedly, to skip a difficult question and come back to it. In MAP, if you skip a question, it’s considered a wrong answer.

4. Students who are absent for extended periods, or who take an extraordinary amount of extra time, may not be able to complete the test.

5. These tests are taken in my district without accommodations, even for students with an IEP.

MAP is a waste of time and money. Additionally, students are presented with material they have not learned. In some cases, 2 or more grades higher than their instructional level. This damages student self confidence and makes them more eager each time they take the test to just “get it over with.”

MAP needs to be revised and made more student friendly, or it needs to just go away. I can think of a lot things for my classroom that money could purchase which would enhance and enrich instruction and learning for my students.

Unfortunately even after using the MAPS test most school districts are teaching at the center. There is very little instruction for children who need help and most of the accelerated programs are far from accelerated. So spend the money not on a test, but on helping students at both ends of the spectrum. Yes, our children are not robots and they are tired of all the tests.

From what I am seeing and hearing the average child loves school, but the children above or below average still are not surviving in our public educational system. Most parents in our area with very bright children or children below average are either seeking out private schools or home schooling.

Over testing children is not instructing them, my daughter receives very little one on one instruction in school. If it wasn’t for what I do at home she would be lost at school. Stop testing the children and give the teachers the resources to teach. My daughter is in second grade and took a Reading, Spelling and Math test this week in the classroom in addition to a MAPS test. Come on…