Test Schools, Not Children

Instead of subjecting children to an endless series of tests and grades, we should be testing the schools for how engaged the children are, and how many ah-ha! moments there are in a typical day. In the long-term we should test how durable the learning is (how much 5th grade algebra and trigonometry do you remember?), how much curiosity the graduates retain, and how passionate they are for learning as adults.

Closing Statement from Gever Tulley

People perceive many issues in education today, and it is clear from our conversation (sometimes heated, sometimes brilliant), that any attempt to address one single issue necessarily leaves many unanswered issues. In general, those well served by a traditional education seem to see little problem with continuing the practice, and those who struggled through school embrace the notion of change - not for the sake of change, but for the chance that we might invent something better.

In seeking to change how we assess children, I started this conversation with the suggestion that we switch the focus of assessment to the schools. To many, this seemed to be a naive approach. A few commenters seemed to be saying, "the system can work, we just need to be more targeted with our testing." Many agreed that there was too much testing, but that some testing can actually be beneficial. Others found merit and suggested ways that it be implemented. Some concern was raised regarding the time it would take to know when a school was failing.

Eric Mercer turned the topic around, asking, "An educational system reflects, not creates, the habits and practices of a society. So which is broken?" My immediate thought is that the best way to fix an ailing society starts with fixing education, which seems to echo the sentiments of many commenters who suggested that schools aren't doing a good job of helping students discover what they truly want to do with their lives. Lee Wilkinson's high school experience seemed to be putting up more roadblocks when it could have been paving the way; "The problem as I see it is; I told them back then that this is what I wanted to do with my life and was told to stop day dreaming."

This has been a fascinating conversation, and I thank everyone for joining in. I am doing my small part to explore new ways to create meaningful education through Tinkering School and my newest project, Brightworks (http://sfbrightworks.org). All my best,
-gever

Feb 17 2011:
I have had the privilege of briefly running a school back about 20 years ago based on Montessori and Piaget guidelines. It was considered an "Integrated Day School" which meant that the learning came from the theme. Sometimes the children worked as a group, sometimes individually. We had mixed K-4th grade in the same room. When we closed the school, most of the parents chose to home school over returning their children to traditional classrooms. The ones who did return to traditional classrooms surprised the schools they returned to prompting phone calls to me to discover what "miracles" I had performed with the children. The only thing I can honestly say is that we set them free to learn what interested them. That's all!

In return for high salaries, the teachers in this school would be required to sit in on each others lessons in order to expose them to a wider view of things and other teaching methods. They would have to evaluate their own performance and that of their colleagues and show improvements. They would have to attend a required yearly amount of specialized conferences both about teaching and about their core subject etc.
Basically the idea is to attract high performing experts with the high salary, and then make sure they put in the right kind and amount of effort. Rather than 'test' the teachers, or for that matter the students, it is more a matter of keeping all engaged in the activity of learning.

The only thing a test and a testing service does is give a lot of people superfluous jobs to return statistics on how good the test-takers are at taking tests. It doesn't prove anything useful.

Feb 17 2011:
RE testing--I would add one thing testing has done on a huge scale: moved public school monies to private corporations such as McGraw-Hill. With those dollars going to testing firms for not only the tests but the scoring and then all the ancillary texts books and supplemental materials, we're talking big bucks each year. This is not reported on much. And standardized tests presume standardized kids. We all know this is not only not true but not possible--so we have an artificial situation that is a blind for moving tax dollars away from their purpose and toward for-profit corporations. That's one major thing wrong with schools today IMHO.

Feb 20 2011:
Over the past two years I have realized my children's most important school years were 1-5. They developed a worthwhile base for Middle School, High School and College while there. It was a good school, and I was happy with it. Perhaps they were doing more that would have given them a 'good grade'. I certainly wouldn't have given some Middle School and High School teachers a decent grade - but my children were able to move through the teachers silliness. It was because of that elementary school grounding.

Feb 22 2011:
Well said William. It amazes me that our culture only sees early education as a means to higher education. This is why we have a crisis in education. If you adopt a pet you don't wait till it is fully matured to train but keep it in a day care hoping that if it is behaved enough that it will have a chance to be taught by a more renowned teacher when it is older. Yet this is our educational model. Children are inquisitive creatures by nature, full of curiosity, but after a couple of years starring at a blackboard (Maybe an I pad if your school is tech savvy) and what have they learned. Not to talk to peers, how to tell what information will be tested, and how to disregard the rest, that after 45 min of studding something it is time to stop and do something new. The sad part is a great education would be affordable. A simple garden can be a multi- grade lab where one class is learning about how provide basic care for plants while the higher ups can be studding soil composition. Instead of spending money for state of the art computers where children learn "programs", the modern equivalent of secretarial skills, take in donated computers that need work. Teach them how to reprogram them so they work. A few years of this at an impressionable age and we will have a population that will be able to tach itself and each other.

FACT: The five billion poor are never going to sit in a school for 18 years.

FACT: Human brains (five billion poor especially) are the one inexhaustible resource we have with which to create infinite wealth.

BELIEF: Connecting the five billion poor to the Internet in a manner that cannot be corrupted (e.g. Google showing you what someone else has paid to put in front of you) or shut down (e.g. Egypt) is the single most effective thing we can do to "educate" the world. Solar-powered Internet meshes across every parish, satellite access from every diocese.

Feb 18 2011:
Has CORE been mentioned yet? I didn't see it as I rescanned the conversation here. CORE is designed to be a more holisitic assessment that provides 360 degrees of feedback, from students, parents and educators. From the website:

"The Idea:
The CORE (Creativity, Openness, Resourcefulness, and Engagement) test is an alternative assessment that flips standardized testing on its head. Right now students are required to take top-down standardized tests that measure only left-brain skills. In contrast, the CORE test will work from the ground up, inviting students to evaluate how well their school is providing a creative, engaging learning environment. Parents and teachers will be encouraged to take the test as well; it will be customized according to its audience.

The CORE test will:

provide the first set of standards for schools to develop right-brain abilities in students
allow students, parents, and teachers to critique their schools on creativity using a public platform
be widely appealing and easy to use."

Feb 19 2011:
Thanks for the information about CORE I shall be looking this up. I think the new TED Conversastions are such a useful means to pass on great ideas and developments such as CORE. Thank you.

Feb 19 2011:
As a teacher, I am concerned about how subjectively teachers are evaluated already. I think making it more subjective. Unfortunately, teacher evaluations, as well as opinions I hear from students and parents about teachers, essentially make teacher evaluation (on or off the record) a popularity contest. Opinions differ greatly on methods and what is rewarded one day is punished the next. I think the solution is a more set method of delivering what students need that is put into place nationally. This would have to include recommended methods and engaging materials for various content. Right now, teaching is like an independent quest for each and every teacher, rather than a body of professionals using the methods that research shows are the best for students.

By the way, schools are evaluated where I am from by boards of educational professionals who visit various schools and look at the atmosphere and the quality of education taking place there.

This is a great perspective. I understand that this issue of subjectivity and bowing to the pressures of being popular are what thwarts the integrity of some college level courses as some professors seek the positive praise of student evaluations at the end of the course.

The danger I hear in what you are saying is that its logical course leads to standardization. "Well if we can't rely on subjective feedback, we need objective standards by which we base our evaluation." This is the argument right? And this is what you are advocating for, correct? This is what is happening with common core standards, a national framework of standards for use in all public schools. I've argued about the virtues and liabilities of this system at www.coopcatlyst.org with other educators from across the United States, Canada, Europe, and Australia.

It also sounds like you work at a private school, where accreditation teams come to visit is this the case? I have been a part of this at both a private k-12 school and a college. It's an interesting process that requires a lot of work. I am not sure that all the work is worth it.

As for "acting like a body of professionals using methods that research shows are the best for students" there is no research that shows what is best for all students. There may be certain baselines like providing basic physiological comforts--warmth, food, water, physical activity--for students, but after that there is nothing I have seen that is clear cut. There are a myriad of methods that are equal but different, and it takes the mastery of a teacher to understand what method to use when. I've seen scripted classrooms (an extreme of what you propose) and they do not work to create rigor, only mediocrity and complicity of thought.

Feb 17 2011:
I haven't seen an actual standardized test recently, but watching IBM's Watson defeat the Jeopardy champions makes me wonder if schools and testing are currently geared toward Jeopardy-champion-style knowledge instead of the kind of knowledge and wisdom that is still beyond a Watson? As kids increasingly have external access to Watson-style knowledge, they should be taught more about meaning, connection, consequences, application of knowledge that may be harder to test en masse.

Whether I remember 5th grade algebra or trig is not as important as whether the process of learning it at the time strengthened my ability to learn abstract concepts in general. I know I can easily and quickly look up the law of sines, so I don't have to be tested on remembering it or not. What needs to be durable is understanding the concept and applicability of trigonometric relationships so I can be simultaneously surprised, delighted, and understanding when I see trigonometry applied to semantic spaces in search engine algorithms. I don't know how to test for that.

One of my favorite test questions was from my daughter's 3rd grade math workbook: "Harry had 36 oranges. He gave some to his uncle. How many did he have left?" It wasn't a multiple choice question; you had to provide an answer. How do you standardize on that?

Feb 17 2011:
Gever! Bravo Bravo Bravo! I stand with you in asking school reformers to rethinking their testing paradigm!

How bout the TED community creating a compassionate coaching retreat where the head honchos who set school policy meet up with gamers, designers, neuro, cognitive and somatic scientists along with progressive and conventional educators to reflect and learn what really tickles and inspires young brains to grow, develop new skills and become learners for life!

I'd be happy to lend my institute as a framing device for the conversation!

Feb 16 2011:
A very interesting concept, and NCLB is about evaluating schools. Of course it didn't take long before we decided we could measure a teachers worth with the same multiple choice tests given to students. The quickest way to improve student test scores and make AYP? Teach students how to better guess at the answers they don't know. In effect we are evaluating the effectiveness of schools and teachers on a students ability to guess on a multiple choice test. Yeah, I know that the test developers and number crunchers can find ways to validate these tests, but the basic premise is flawed! The overarching question is do we want students to be able to recall facts--my computer does that pretty well. Or do we want the to be innovative, problems solvers, to be able to make fine distinctions, to have compassion, to be able to adapt, to have an aesthetic sense, and to understand the larger world around them. If it is the later, how in the heck do you assess that? It requires a different paradigm.

One way to evaluate schools is to identify what the best practices are for developing these important attributes and then observing the learning activities at the school to see that these best practices are being used. My idea for best practices would be project based learning built around a theme that is relevant to the students and is managed by a facilitator to ensure that the important concepts (standards?) are addressed. This would mean a major rethinking and retraining in the existing education community, but the basic ideas has been around for very long time.

Feb 16 2011:
I believe the problem lies on the content of the tests. Basically all tests in school strive to find the kid that has the ability to memorize concepts and not necessarily understand them. Why do we spend so much time congratulating this kind of behavior when society does not have a need for this trait of memorizing. The true need is the ability to create and improvise not recite and memorize. We should restructure the paradigm of school, it is severely old fashioned and has little to do on what matters the most.

Feb 16 2011:
That is especially true considering we heavily rely on google and the internet to look up facts that most of us left behind long ago in school (or more accurately) moments after we took that exam. We have information at our finger tips via cell phones and computers 24 hours a day.

Critical thinking and creativity are needed in schools more then facts. People need to be taught the why's and how's of things to have a deeper understanding, not just memorize enough facts to get through a test. The test really means nothing in the end.

Feb 15 2011:
Learning is durable when students are empowered and have ownership over their learning. While this is being done in pockets around the world, it is far too infrequent. The idea of grading schools is great but I would counter that it is not a matter of grading schools but rather offering more examples of what kind of learning happens when students are engaged.

Examples like your Brightworks School, among others, offer the opportunity to provide evidence to school systems on the value of a more engaged and empowered student.

Changing "failing" schools is a massive cultural hurdle but if we build more schools that focus on students making a difference and engaging in something real and tangible, rather than simply throwing computers at them, we can offer some valuable examples to what is possible in schools.

I am in complete agreement with you on all points except for testing schools. Instead, let us reimagine schools as open, collaborative, creative, and hands-on with the real world. I think the problem lies with parents. Kudos for starting BrightWorks school as I believe is is absolutely the model for the future. It's very tough, though, to get parents to see alternatives to institutionalized learning when they believe that a letter grade is what matters most in getting into a good college. Perhaps if more universities openly embraced alternatives to grades and SAT scores, parents would feel more comfortable with alternatives like BrightWorks. I believe then that the tide will turn and BrightWorks-type schools will be part of every community!

Feb 26 2011:
Have we also thought about the fact that if we are to change the way in which students are taught and tested then surely we must also change the way that we are accepted into College and University. The system at the moment allows higher education to have a method of of testing student for acceptance (SATs, A Levels and IB) flawed as it is surely that would also need to change. How would we get the higher education system to change it's approach?

Feb 26 2011:
The many issues that have been raised still leave me with a unanswered question, primarily because the solutions offered seem directed at symptoms of educational failure rather than the cause of public school failure. I would like to suggest that the issue of purpose is avoided. Why and for who's benefit do we educate? I have a purpose for education that does not seem to be on the previous agendas; and here it is:

To promote in students a capacity for critical independent thought with a responsibility for the commonwealth (community). IN other words I would be more concerned with developing a student's ability to learn rather than spending too much time on what to learn. I am interested in promoting cognition as an aspect of social responsibility. I'd like students to develop a capacity for making meaning out of experience in a way that nurtures critical thought.

After reading the preceding comments about schools and who should be measured I felt that schools should be competent at learning as organizations (Peter Senge) because if they couldn't learn as a collective then they couldn't do a very good job of supporting learning in either the students or the staff.

I'd be interested in how these thought coincide or conflict with other views on this subject.

Feb 25 2011:
I'm 20y old and in the 7th grade I wasn´t aproved to the 8th grade, at that time I remember not having "happy" days in school, all I used to do was wrinting stuff that were on the blackboard and no interaction with teachers and all the material besides the black board were like a world map of the 60's and documentaries of the 80's.
The teachers weren't motivated, they didn't like their jobs and that reflects on the students,
At my second year teacher looked at me like I was a lost case and didn't bother about me and I became a part of a group of "lost cases" where teachers don't look to us and only see an F in my brain..

As my case there were a lot, like 20% of the students that weren't aproved each year, all them get lost in drugs, and thinking that they're ideias didn't worth the effort, for example a big friend of mine wanted to be an Geographer and he wasn't aproved 3 times and he droped out, now he is just a guy addicted to cocaine, alcohol and living with sadness and with a "phobia" to schools.

So, if schools should be tested, I guess yes, but first of all we need give the materials that a school needs to work.

Feb 25 2011:
Before even testing schools, teachers or students we should experiment with different approaches to teaching and understand what is valuable to learn and what is not?

I think students need to acquire the following skills and knowledge for life:
- creativity
- communication skills
- team work
- basic facts but not 'trivia' type of information
- concepts in science and math
- ethics and history

So instead of asking students to memorize formulas and facts that can be found at Wikipedia etc, why don't we engage them with local and national competitions in science, math, arts, creativity etc. They will learn great deal of skills needed for their careers as well as learn how to deal with peers.

In addition we should give students more flexibility in what they can learn and how, at their pace while ensuring some basic goals are met at the end of the year. (the founder of Khan Academy organization has some interesting ideas on how to do that)

Facts will become important once students attend apprentice, college or university but by then they will at least develop solid framework of life skills mentioned above.

Feb 23 2011:
I think that we should seek out educational diversity. An educational monoculture exhibits the same weaknesses that a biological monoculture does. If we are to thrive in a world of economic, political, and climatological shifts, we should promote the development of a diversity of education systems. We need a generation of adaptable individuals who exhibit the tenacity necessary to solve big complex problems in a range of career paths that don't even have labels yet, the resilience to work through and around a near infinite list of setbacks, and the creativity to imagine the solutions.

The best way to figure out what works best might be to start trying some different approaches. Summerhill, Sudbury, Waldorf, Montessori, Reggio Emelia - these are great experiments that have been running for decades. The children who graduate from these programs tend to graduate from college at rates equal to or better than the best traditional schools (private and public), and yet those successes are not integrated in classrooms. We need to integrate the best results and continuously innovate, never expecting a single permanent solution in education.

Feb 23 2011:
We must be careful using "students who attend college" as the success measure. This omits around 70% of the entire student population.

Would it be accurate to assume that families who can afford these different approaches can also afford the cost of post-secondary education? Also the success rate for first-generation college students is quite low when compared to second and third-generation students. What this unfortunately points at is the design of an educational experience for an educated few.

Feb 23 2011:
I agree Eric, the common use of "getting into college" as a metric is fundamentally flawed. I think it's more interesting (but still flawed) to look at college graduation rates, but we are always going to have an apples-to-oranges problem as long as the cost to attend an alternative school separates students along economic lines.

Our new school (http://sfbrightworks.org) uses a sliding scale tuition and deliberate outreach to make it accessible to low-income students, but we find that the social barrier is even more difficult to cross than the economic one. We find ourselves having to work constantly to overcome the perception that an alternative education is not a "real" education.

Measuring success is always going to be tricky when the pedagogy does not allow testing the students. We need to look longer-term and look at broader definitions of success. An oft-quoted study of the alumni of the original Sudbury Valley school, revealed a consistently happy and successful group of graduates (http://bit.ly/eZKOGt). Summarizing the study, one reporter said:
"The Sudbury Valley School recently completed an extensive study of former students that provides qualitative and quantitative evidence of the success of the model. Graduates tend to see themselves as creative individuals who feel successful and happy about their lives. They pursue a full spectrum of diverse careers and interests; 45 percent of those surveyed were self employed or owned small businesses; 82 percent of graduates go on to higher education. In addition, colleges and employers view a Sudbury-style education as an asset and a predictor of success."

1. Let the kids assess the teachers - nonsense. Kids are not qualified to assess teachers!
2. Students know what they are interested in - nonsense. The whole purpose of education is to broaden their interests.
3. Don't teach facts, that's what computers are for - ridiculous. Education is about learning the facts. Not to memorize them but to know where to find them later and knowing what to do with them after you have found them.

Here's what I think are the main problems with the current educational system.

1. Kids are taking way too many classes/credits. They are "drinking from a firehose". A 5 month semester is not enough time for kids to synthesize and master all the new material nor identify common themes among classes/subjects.

2. The testing regimen is ridiculous. I have kids taking mid-term exams in several subjects about once every 4 weeks. Again, not allowing enough time for the students to synthesize the material. They study to cram for the exam, not to learn the material - See Gever Tully.

3. Retention & firing of faculty depends too much on the "student opinion of instruction" surveys. They are student opinions that's all, they are not teacher evaluations nor should they be used as such - see Matthew Fisher.

Feb 22 2011:
"2. Students know what they are interested in - nonsense. The whole purpose of education is to broaden their interests."

Is that how our school system turns avid readers into people who absolutely refuse to pick up a book?

There is a very critical difference between introducing new ideas to a student and overriding the fledgling interests already present in the student. Our modern education has shown no ability to even understand the potential damage it can actually cause.

"3. Don't teach facts, that's what computers are for - ridiculous. Education is about learning the facts. Not to memorize them but to know where to find them later and knowing what to do with them after you have found them."

This point is so vague that it contradicts itself. Surely you're not intentionally suggesting that education is about learning the intricacies of quantum mechanics so that if you forget your can look it up on Google later in life.

Largely the only fundamentals which I can see a reason that they must be taught are simply fundamentals of communication and critical thinking. If a student is able to form and communicate ideas effectively as well as understand outside ideas and analyze them then what other skill could possibly be necessary for someone to teach themselves anything they could need in the future?

Feb 20 2011:
I loved school and by all accounts I was a pretty mediocre student, I am sure mainly because of the old industrial model of education in 1960s Great Britain. Fairly late on in my life I became an actor. I trained at a conservatory and later had the opportunity to study at RADA. I now also have a Masters from Kings, London and RADA. The point of this is that my love of learning and my ability to learn came together when I was 38. I heave learned and retained more about the world, maths, English, Physics and people skills since becoming and actor and a writer my two true passions than I did in school. The problem as I see it is I told them back then that this is what I wanted to do with my life and was told to sop day dreaming. So many people are like this and while academic study is wonderful for those who can work that way the avenue into education is not the same for all. Testing is really the proof that the system does not work and not the child.

Feb 20 2011:
The purpose of education is at the core of this discussion. Some suggest testing schools for relevance to one's ideal of life fulfillment is purposeful. Others believe success in ventures beyond and after the classroom is a good measure. Or whether someone retains information and skills in life (say, trigonometry) when perhaps they have gone on to positions that require minimal use or application of these skills.

It is a commonly held belief that a small percentage of humans control and consume the lion's share of world energy and resources. Do we feel that this is something we should all strive for - to grab our fair share before it runs out? We have but one life to live and it better meet all our expectations and desires of comfort and consumption even if it causes others to suffer shortages? Is greed, self-service and self-fulfillment at all costs driving this discussion to restructure education? An educational system reflects, not creates, the habits and practices of a society. So which is broken?

If it is guaranteed that our privileged level of consumption, leisure and exciting travel opportunities is sustainable then sure let's further build upon educating for the purpose of sole self-interest as the goal. If not, maybe some adaptation on our part to fair, reasonable, world sustainable outcomes might be in order.

We can't reach our true potential caring about others? Does an education system have to meet all our individual wants? You can't always get what you want but you just might get what you need (Jagger, 16,17 November, 1968).

Feb 25 2011:
I echo the comment "The purpose of education is at the core of this discussion." We should be discussing that first and foremost. Before we discuss how we are going to do and what tools and whether to use textbooks, we need to decide on the goal more than just "educating people". What is that goal?

Feb 25 2011:
My career, such as it was, was spent doing two fundamental things: researching, promoting and implementing new learning opportunities and researching, analyzing and questioning educational fads that promise but never deliver.

Telling the difference between the two is not a simple task, and a task that was not always done correctly. The fallout comes at a heavy price for the student.

Presently the buzz in education is early-learning. I believe that this holds merit however it is being promoted as a near 'cure-all' - for which it crosses, in my opinion, into the second category of promises without chance of delivery.

There are a number of what I refer to as 'key-stones' in education and neglecting any in hopes that any one will bring the success we so desperately desire will lead to another disappointment. One disappointment that comes to mind is our failure to continue to address the learning needs of males.

Feb 19 2011:
Broadly speaking, I think the goals of educations should be (1) to help young people choose a life path that gives them the most satisfaction, and (2) teach them the human and practical skills they need to excel at that path.

I'm 33, Ivy-League-educated, and in a stable job that challenges me, enables me to travel and interact with great people, and develop new ideas to use emerging technologies to improve lives. I've also been a teacher and a journalist, and lived in the U.S., Africa, and Europe. By almost all definitions I have "succeeded." I'm happy, yes. But I also feel like I haven't discovered my true passion—a fulfilling life path. I look back at my standard, U.S. public-school education and don't recall being taught to think about what makes me happy and why.

I think if our goal is to help young people grow into emotionally fulfilled adults (which are surprisingly rare in today's world) we have to do more to help young people learn to be introspective—to identify what makes them happy in life—and then give them the tools to go after those things.

I'll leave it to the education experts to say what such a system might look like, but here are some of the insights that struck me from this conversation:
- Pay teachers much more (to attract the best talent) and then require them to sit in each other's classes, both to learn from colleagues' approaches and to evaluate each other.
- Group learners by interest/learning style (rather than by age) and with teachers passionate about that interest.
- Find ways to individualize instruction more, by increasing money in the system to decrease class size, but also by pulling more of society into the role of teacher. Young people could spend more time learning from professionals, and more class activities can be guided automatically (computers, virtual reality).
- We don't need to learn to recall most facts anymore (thanks, Google!). Since our educational needs have changed so much, it's a great time to rethink the system

Feb 18 2011:
I've been thinking about four interrelated things regarding 21st century teaching and learning:

creativity - to what extent do we unlock the treasures within each of us?
citizenship - what does it mean to be a local, national, and international citizen?
connectivity - how do we relate to each other in a twitter/facebook world?
compassion - how do we deeply engage with our communities to bring about meaningful change?

... the challenge is how to measure these attibutes? As Patrick indicates, if the drivers of tertiary entrance continue to be standardized admission tests, then there is little stimulus for the system to be changed in the education world.

Feb 18 2011:
Great dialog so far... Enjoyed reading the various thoughts. This is one of my core areas of passion www.21foundation.com :-) and could write for days about this... Top 3 thoughts

Change the standards by which the top Universities accept students
Design accreditation standards to align with 21st Century Skills so the schools are evaluated based on these skills.
Reward learners and educators for embracing inquiry-based learning

Feb 18 2011:
This discussion reminds me of the question that usually comes to me when the subject of testing students is discussed. That is why nobody seems to ever realize that we are wasting vast sums of money and time when we test every student in a school to try to assess how the school is performing. It never ceases to amaze me how we can fully accept the results of marketing or political surveys that are typically based on the responses of a small number of people that represent a fraction of the full number of people impacted by a question, yet we think we need to test 100% of the students. We are more than willing to accept the results of a questionnaire that 200 or 1,000 people answer and apply it to millions of people. Apparently most polls, surveys and questionnaires have very small margin of errors even though they may be based on a small fraction of 1% of the total population that is affected. Why in the world do we insist on testing 100% of the students in a school. Let's test something like 5% and let the other 95% or so of the students use the time more effectively to learn more.

Feb 23 2011:
We talk about expensive books, but the expense of all this testing on school districts and governments is positively staggering. Millions are made on test creation and many more are made to employ and train people to do the assessments. It is my opinion that tests are not properly vetted,nor is the job of choosing which companies are engaged in the development of these tests given to educators. The job usually goes to the lowest bidder, but the company doesn't have to stick to the budget and they know that the gravy train will continue for a decade of of fees for updates and assessment training.

The situation is already unsustainable in my state - to much testing and not enough learning time.

Feb 17 2011:
Testing children is what life does. Human beings are a social species and by nature adaptive survivors with untold possible functions and combinations of functions in contrast to narrow specialists like cheetahs. Therefore testing for curriculum retention is incredibly meaningless. In any case one year after high school surveys of GRADUATES show a 20% retention. Testing skills or competencies may have more relevance but which skills? How many rocket scientists do we need? Why do Doctors need trigonometry? It is no longer possible to learn EVERYTHING relevant for even one specialty within a normal lifetime. Since curriculum committees are not Deity why do we let them define what is needed for our children? It is interesting that Einstein and the Dalai Lama seem to agree that technical information without moral-emotional-social wisdom tends to create monsters not happy humans. I hear that in Finland (which is rated #1 in the world) there are NO bad schools. Logically this must come in part from the fact that all their schools compete with all other schools by free choice of parents and students. If a totalitarian planned economy cannot feed the North Koreans why do we think it can work in a much more subtle system like education?

Feb 17 2011:
Doctors need to understand abstract scientific concepts and relationships. They may not have a daily practical application of trigonometry (though a little trig could help them understand things like medical imaging technologies), but the process of learning trigonometry strengthened their ability to understand and apply abstract concepts to practical situations.

Feb 17 2011:
I agree a matrix should be created that allows us to measure and evaluate how the school performs. For example, how child-centered it is? Do they have a recess and then a calm quiet lunch period, or do they do lunch first, so children hurry through their meal and toss most of their lunch so they can go outside and exercise? This matrix should be based on community expectations of schools, not how well the children perform on standardized tests. Several good models exist, including Tribes. I welcome comments and ideas on areas for the matrix/ rubric to evaluate how well schools serve our children and our community. Also, of course, how well they value their teachers.

Feb 17 2011:
School is both a vital and touchy subject, because everyone relates to their own school system, which is itself set up to produce the future citizens of its own country. School is basically a tool aimed at maintaining a society and has never been about the welfare of pupils/students. So there cannot be one single universal model and it's country leaders who need to be convinced that children's interest will also be the country's best interest. It means changing the way companies and institutions work as well, while they are based on rentability and individual promotion as a result of harsh competition, that the school system cannot but reflect. So the radical change should really come from the top.

Feb 17 2011:
Schools should undoubtedly be tested. But only if the tests are better than what we apply to the students. How are the critical thinking skills of the students that come out of the schools? How well do they think outside of the box? How do they handle situations they have never seen before? How do the students approach these problems?

In the future memorization is going to be irrelevant. Critical thinking skills, the ability to diagnose problems and come up with creative ways to tackle those problems are going to be important for the future of the children and our country.

Feb 17 2011:
I believe that when children get low grades, while having a wish to learn, it means that that particular school is not effective enough in educating them. School should be a system which is able to give the same knowledge to everyone, while adapting it in a way that each of students can conceive the given information in their own best way.
Now, If you think of how many children in this world get low grades and how many schools they attend, almost all schools will be in the list. The solution is not easy to find and it is not just simple. But we have to find and apply it.

Feb 16 2011:
This is a clever notion but logically leads to some challenges. Look at medicine and finance: We all basically agree that the right way to reward physicians is to reward them for outcomes. Testing schools for the right outcomes could entail evaluating them based on the students college acceptance rate OR the rate at which they start businesses OR their happiness with their education OR something else entirely. Worse, the mortgage crisis has been show to (in a substantial way) be a result of rewards for brokers who built deals which looked good initially but which fell apart within 24-36 months. This could imply that good testing of schools would only show results several years after graduation. I don't know how you test schools (or test students for that matter) to allow for actionable real-time correction in favor of better outcomes.

Feb 19 2011:
I agree. I also think we spend far too much time worried about test results from our public education system when we should be more concerned with if they are teaching the children how to learn, which is infinity more useful than memorizing for a test score.

Feb 19 2011:
As a teacher, I am concerned about how subjectively teachers are evaluated already. I think making it more subjective could make it more subject to corruption. Unfortunately, teacher evaluations, as well as opinions I hear from students and parents about teachers, essentially make teacher evaluation (on or off the record) a popularity contest. Opinions differ greatly on methods, and what is rewarded one day is punished the next. I think the solution is a more set method of delivering what students need that is put into place nationally. This would have to include recommended methods and engaging materials for various content. Right now, teaching is like an independent quest for each and every teacher, rather than a body of professionals using the methods that research shows are the best for students.

By the way, schools are evaluated where I am from by boards of educational professionals who visit various schools and look at the atmosphere and the quality of education taking place there.

Feb 16 2011:
Let's create an "INSPIRATION METER" !
Education should be more organic to each individual; it should be about ideas and conversations, instead of measuring all students with the same parameters.

Feb 16 2011:
While this may be more of a "meta" wish, I'd like to see schools evaluated on the degree of potential they achieve with a student. For example, gifted kids often get short shrift because they easily master the materials and concepts for a "grade level". The progression of these kids is on a very different scale than other kids in their classes. Once they show that mastery, the challenges disappear, resulting in a less-motivated student.

On the other end of the spectrum, kids with educational challenges have a much harder time grasping some the concepts that are being tested for. They get extra attention, and their measurements are less about the academics and more about relative improvement.

This is where I think the future of school testing lies - in relative measurement rather than against the usual yardstick of specific skills. Let's find ways to determine whether a student is progressing to his abilities, rather than against a standardized test. Instead of focusing on "one size fits all" solutions, change the paradigm and fit the solutions to the student. Then measure how they are progressing based on their previous level of skill and knowledge.

I don't pretend to know how to get to this solution, but Ih ope that it's a start.

Feb 16 2011:
The solution lies in front-end solutions. Teacher training. Currently, we do not train teachers to be leaders of the community who understand sociological systems and psychology. We train mostly mainstream learners to teach other mainstream learners how to be better mainstream learners. Everyone else is left out of the equation. Those mainstream learners become principals and administrators who make decisions shaping education. I am currently creating a program for this summer to train teachers in technology along with children to model what learning can and should feel like. It will take a bit of time to change teacher training and the cycle of micromanaging our learning. I think that we are ready and hungry for a way out so it is possible. It will be hard for many current teachers to grasp new ways though. They are used to talking way too much and wearing themselves out with lesson planning and grading and testing. The key is to focus on the new teachers and few willing experienced teachers. The good news is that people who become teachers are compliant fast learners and once a paradigm shift catches on then it will spread quickly to affect change.

Feb 15 2011:
Interesting idea, but the main point missing is the "how".

How do you measure a school's success? What constitutes a "success"?

Furthermore, how is each individual student affected by their school's rate of success?

Like it or not, there's no universal notation of an excellent teacher... what works for most people may not work for certainly skewed individuals*, so another question that raises from that is how to determine if the teacher is not right for a certain student for one reason or another (again assuming individual grades are abolished)?

I'm going under the assumption that this would be a sudden switch where everyone will be using this system, so no "compatibility" will be needed, making things easier... but the above questions apply even with this in mind.

* I've had cases where I had to paraphrase some stuff to fellow students that the teacher has said, and conversely, I've had cases where someone else had to re-paraphrase what I said before the fellow student got the point.

Feb 15 2011:
Gever- love this idea. Actually, how about we let the kids offer assessments of the schools and teachers as it relates to engagement, learning and mastery. Students know what they are interested in, when they have been supported to engage and discover their own interests.

Feb 12 2011:
Today I read 59. edition of Edge. Roger Schank talks about why in current education system students do not want to learn. He says it is because of grades and tests. Why there are grades and tests?

"The problem stems from the certification mission of schools. As long as the next school or employer expects that the current school will tell them who is good, the system can't change."

So we should create new certifications and remove tests and grades from schools.

Feb 13 2011:
Traditional public schools are a complicit link in a cyclic dependency chain that includes and defines many aspects of our global culture. One local public school in the San Francisco area that wanted to add hands-on learning to the classroom experience was thwarted by the Teacher's Union because the teachers did not want to have to learn how to run their classes a new way. Higher Education produces accredited teachers that are trained to run the classes in the "traditional" manner, and so it's very difficult for schools to find qualified staff (that meet the state requirements) that can, or will, teach in a new way.

Oddly, the place where this chain is breaking right now is at the employer level. Employers find that they cannot rely on the schools to accurately identify good employees. A hiring manager at HP stated that they fully expect to spend 3 to 4 years extending the education of a new hire before they start to make valuable contributions to the company. More and more, it appears that the student who deliberately breaks out of the traditional education path has a distinct advantage over those who do not.

Comment deleted

Feb 13 2011:
The irony is this; any system can be improved. The national focus in the United States is to keep trying to improve the current system. The problem is that the current system is nearing a local maxima. There are endless studies proving that great teachers can have a profound positive impact on the life of a child - but the limits of that impact are defined by the constraints of the school system. If the best thing for a given child is discovery-based learning, and another child would do better with a mastery-based approach, then neither of those children is being best served by the system - no matter how much the classroom experience is improved. And yet we keep pouring resources into solving the problem of making the system better, and we keep finding ways to measure and validate the improvements (typically by adding more testing).

What is needed is a series of long-term (10-20 year) experiments designed to evaluate and discover new learning systems.

Feb 15 2011:
Gever, the act of improving a system is reformation. However, the answer to problems in our current public education system is not reform. The fundamental premise of the system is to create economically viable products. The current reform effort is simply trying to move the system from creating employees suitable for a factory system to employees for the 21st century economy.

Sir Ken Robinson said it tersely, we need to transform education. Is this not what you have done at Tinkering School and now Brightworks? We have to fundamentally change the aims of education.

Furthermore, we know that schools and students do not exist in isolation. They are intimately tied to economics for example. Studies show us time and time again, regardless of what type of learning system a student goes through their socio-economic experience has tremendous effect on their learning trajectory. So perhaps we not only need to be talking about evaluating new learning systems, but entirely new human community systems. This purview would encompass education, economic, and ecological systems (and more) all at once.

Feb 15 2011:
I don't completely agree that most of the children tend not to like going to school. Remembering my first grade (and I mean first grade primary school), most of the children were actually excited to be "going to school". And that feeling was still (!) there when learning basics of mathematics, writing and reading. But gradually, they began not to like going to school...

How come school makes children gradually dislike learning? Yes, we should test schools, but in my view primarily to determine, where the mistake is. Some schools are better - where's the difference?

Feb 26 2011:
Aren't some of the most important qualities of a good education somewhat difficult to quantify?
For example, finding an area of interest.
I think this depends on the influence of mature role models, sharing their particular passion with students.Could a lack of academic role models leave a student clueless about where they have an aptitude for? I would venture that this is just as important for vocational jobs- though it ought to be noted that students have to find vocational role models outside the classroom.
The reason I suggest this is difficult to quantify is that the paths for a student to follow are already pre-made. If a given student comes to the end of their GCSE's and decides leave education it could be down to either a strong interest in an area that doesn't require A-levels or because of they underperformed in their GCSEs...

Feb 26 2011:
Having worked in an urban school district for 11 years I have come to see that one size does not filt all. Little is done with regard to authentic learning opportunities, teachers are overburdened with "must do" tasks to satisfy state imposed "duties" which dilute a teacher's ability to teach. I see the need for more parent, community education and allowing the teachers to do their job. Teachers also need more training for urban education. They now have little power. The ah-ha's come with the opportunity to think creatively and critically. The ah-has are when children truly learn and the lessons become encoded. Unless the time is given for this... there will be few ah-ha's.

Feb 25 2011:
I see two problems: the nature of testing and the actual paradigm of education.

Does memorization of facts and concepts equal brilliance? How do you grade curiosity, passion, or critical thinking? Testing schools is not the solution because the original intent of mainstream education is compliance: being just good enough to keep the gears of society well-oiled and running. Adam Burk previously stated: "the fundamental premise of the system is to create economically viable products." This was the educational archetype supported by major industrialists in the last century who steered and even funded education in this direction.

So, should a test exist, the answers to your questions would be evident.

It goes beyond the ineptitude of one public school, but society's definition of education and its addiction to standardization, and grades as proof of worth. Crude rote memorization and serial testing isn't the way to foster creative wayward minds: that's how you manufacture conformists who are likely to embrace everything at face value rather than innovate, improvise, or invent.

Ken Robinson said: if you fear being wrong, you'll never come up with anything original. He espouses taking risks, a big part of creativity and the entrepreneurial mindset. Not to say everyone is to erect their own business or enterprise, but that the grand wager of taking risks, no matter the context, is a burning part of innovation.The educational system does not nurture individual genius or creativity to even explore this.

Reforming the current system is especially hard in the American political milieu. A real paradigm shift in education is anathema to gatekeepers such as the NEA, teacher's unions, and the educational lobby who are powerful forces and where a lot of money and egos are at stake, and who'd have no qualms in quelling real reform. That, plus it's a cultural obstacle as many still revere the one-size-fits-all public education system as the future for all human advancement.

Feb 25 2011:
Are you serious sir? Just how are "we" going to effectively and realistically go about testing any given school on how engaged children are in lieu of tests and grades? Will we charge a group of sullen bureaucrats to prance through the classrooms inviting children to complete some series of nonsense rhymes? Just what kind of government system would you initialize throughout the state of California to determine how many ah-ha moments over 6 million students may or may not be experiencing on any given day. Furthermore, how could you ever track millions of public school alumni to determine if they recall the difference between polynomials and quadratic equations?And why would you want to do that? Never mind the outrageous cost of time and effort and taxpayer's money.

Feb 26 2011:
If an infrastructure can be built to test millions of children, then it seems just as feasible to test a thousand schools. Taken as a goal, the steps and procedures to query alumni would be no more complicated than running a statistically accurate poll.

The point of such an endeavor is to move us away from the known flaws of a grade- and test-based system that has been shown time and again to be both inaccurate and harmful. This idea only appears unrealistic because we're so inculcated in the current testing model that everything else seems strange in comparison. In truth, it might be much cheaper to run, and provide better feedback on what works in education.

Feb 19 2011:
While it is an interesting concept and I think Mr. Tulley was conceptualizing something a bit more abstract when he stated the question the fact remains that when you test the students of the school you are, in essence, testing the school. The question I keep hearing from parents when I talk about schooling is always "What are they teaching them at that school?" with the implication that the students of said school aren't learning anything. While I will admit that we, as Americans, are in a crisis when it comes to education, but we still produce some of the smartest people on the planet as an end result. Regarding K-12 education, which I believe is the main topic of discussion here, we need to focus on a couple of things first.

Number one being the high dropout rate, while we need people to work in fast food and other unskilled labor jobs, a close to 50% dropout rate is unacceptable by any standards. I believe that the reason for the high dropout rate is the common belief that if you want to be anything other than an unskilled laborer you have to go to a 4 year college, which is not the case; in fact 4 year degrees are becoming so common that most people that I know with a 4 year degree aren't even working in their field of study, it was just another "piece of paper" for the HR department.

Number two is focusing on fundamentals first then moving into specialization. The fact of the matter is that the old "reading, writing, rhythmatic" still needs to be taught to students, as boring as it may be these subjects may be they are the foundation for all future learning. The methods of teaching these subjects may differ among students but just because every student doesn't have an "ah-ha" revelation moment doesn't mean they aren't learning, and just because you don't remember your 5th grade algebra doesn't mean the education system failed you. While smarter testing systems will probably help gauge comprehension, we do need some conformity in our testing methods.

Feb 19 2011:
I think it's pretty clear that testing does not accurately indicate what is being learned - the time frame is too short. Classroom observations show that high school and college students have learned how to effectively cram information necessary to pass a given test, but when spontaneously asked the exact same questions from the same test just days later in a classroom, they have no effective memory of either the question or the answer.

We wouldn't need to test if the results of learning were self-obvious. Schools should put students in a context where their behavior and efforts produce outputs which a effective demonstrations of their understanding of a topic. The notion of a "test" is merely a convenience for the teaching organization, and offers no benefit to the student in the long run.

Feb 17 2011:
I think a related thread is home teaching. It's interesting to keep the annotations about all the legal cases of states against families and vice-versa. When families are doing ( or at least trying ) to do a good or better job than government to teach their kids. Here is a recent Brazilian article [1] on a family that was sued by the Brazilian government. In Brazil things are even worse than some places because our law is general and applies to the whole country. So because schools are poor is no way to scape unless you leave the country, so far.

Feb 17 2011:
The sad part is that when our assessments would not be authentic, we won't learn anything from it. Tests should not be give grades rather a vantage point for teacher to facilitate child's growth and let him/her know the hidden potentials.