A blog about reading, writing, teaching and the joys of a literate life

Main menu

Post navigation

More Thoughts on Craft and Those Pesky Test Questions

After reading my last post on craft, a friend and colleague emailed me saying how amused she was by the fact that I’d used the phrase ‘make no bones’ in the same sentence in which I’d compared close reading to a mouse dissection. I had, indeed, purposely chosen the simile to evoke the sense of desecration I think happens when we over-analyze a text. But the phrase ‘make no bones’ had just popped into my head, and I used it with no awareness that it echoed the lab mouse dissection until she’d pointed it out. Put another way, I didn’t consciously choose that phrase to create the effect she experienced, though I was tickled by what she’d noticed. And this reminded me of a quote from Samuel Johnson that speaks to the relationship between writers and readers: “A writer only begins a book. A reader finishes it.”

I think this is true for the simple reason that a writer’s words on a page are inert until a reader’s mind brings them to life. And while I do believe that writers make choices about words, details, images, and structure in order to convey what they’re trying to exploring, there’s also something intuitive and uncanny about the process, with writers making unconscious decisions as well as conscious ones as they craft a text. And that opens the door for readers to see even more than the writer might have intended and to come up with a range of interpretations about the words on the page.

Kylene Beers and Robert Probst address this very point in their new book Notice & Note, where they share an anecdote about the poet and playwright T.S. Eliot. A literary critic who’d seen one of his plays thought the play director had misinterpreted Eliot’s meaning. Eliot agreed that the production conveyed something different than what he’d intended, but he wasn’t really troubled by that. “‘But if the two meanings are contradictory,'” the critic asked, “‘is not one right and the other wrong. Must not the author be right?'” Eliot replied: “‘Not necessarily, do you think? Why is either wrong?'”

This anecdote suggests that, despite his connection with New Criticism, the literary theory behind the Common Core, Eliot believed that multiple interpretations can, indeed, co-exist and that in the end the writer’s intentions don’t necessarily carry more weight than a reader’s interpretation. And this raises some interesting questions about all those “Why did the author include X in line Y” multiple-choice questions on New York State’s tests.

More questions are raised by the memoirist Patricia Hampl who, in her essay “The Lax Habits of the Free Imagination,” looks at the fallacy and the presumption of those author purpose questions. In the essay she recounts the experience of having an excerpt of a memoir she wrote appear in a college anthology that she, herself, had used in classes. Initially delighted to be included, she had an unexpected and uncomfortable reaction when she received the new edition in which her excerpt appeared and saw the questions that accompanied the piece. “And there, at the end of the selection,” she writes,

in those shivery italic letters reserved for especially significant copy, were the study questions. There were several under the heading “Questions About Purpose.” One will do: “Why does Hampl establish her father’s significance to the family before she narrates the major incident?” Beats me, I thought.

I had no idea what Hampl’s purpose was. All the study questions looked quite mad to me.

These ‘quite mad’ questions are, of course, precisely the kind that appeared on this year’s tests, with four possible answers for students to choose from, only one of which was deemed right. And they’re also the kind of questions that appear on the new Teacher Performance Assessments that Pearson has developed for edTPA, the organization that will be testing pre-service candidates to see if “a new teacher is ready for the job.” Here’s the first paragraph of one of the passages from the sample literacy skills test online:

And here’s the kind of question that’s asked. As in Hampl’s case, one will do:

The repetition of the phrase does suggest some intention on the part of the author, but none of those answers seemed ‘right’ to me–including B, which the answer link said was correct. None, for instance, captured my sense that in her own unique and unconventional way, Gertrude Stein had a well-rounded life that was full of friends that were like a family, which might only have been possible because her family was prosperous. And none were connected to other details I’d noticed about her father, which suggested to me that she was repeating in reverse the journey that he had made. And when I re-read the passage, as close readers are supposed to do, I found myself thinking that the repetition had less to do with Gertrude Stein than with the idea that’s embedded in the title: that we cannot predict or control the future because we live in a world that’s disordered, in which the unthinkable happens. But that wasn’t one of the options.

One thing for sure, though, the question and answers forced me to abandon all the thinking I was doing and instead try to guess what the test-makers were thinking. And at that point I stopped being a reader and became a test-taker instead.

This has all made me think that when it comes to craft we might do better by remembering that readers and writers are both engaged in fitting details together to build meaning, with the writer ‘crafting’ the story out of details and the reader then using the details the writer’s chosen to ‘craft’ an interpretation. Any interpretation should be considered valid as long as it’s supportable by the details of the text, even if it veers from the writer’s intention. Most writers I know would agree with that because they respect and value the magic that happens when the words they’ve written interact with the mind of a reader. But one has to wonder what edTPA wants when they think that what demonstrates a teaching candidate’s readiness to become a teacher is the ability to second-guess the test-makers’ interpretations, which is what those answers are. What students really need are teachers who know how to help them craft their own ideas from the details the writer’s crafted the text from.

4 thoughts on “More Thoughts on Craft and Those Pesky Test Questions”

Vicki,
Brilliant! First of all, in planning my summer course my hope is that the teachers I will be working with will be able to identify their own writing process and to realize that everyone has their very own. Part of that for me and many is what you say about what happens unconsciously! I often envision dead authors rolling in their graves at how some wonderful books are so incredibly over analysed that all of the joy is sucked right out of them!
I also notice how nervous I got when reading the passage you posted. I am a terrible test-taker! There I said it! Mostly because I spend too much time considering how each of the answers could or could not be true! Instantly instead of reading I am wondering what am I “supposed” to get out of reading this passage and as you so eloquently state I become a test-taker not a reader. Again I am amazed at how quickly this stance changes for me!
Valuing the reader and what each individual brings to the page is a crucial part of raising kids who want to read!!
Thank you for this piece! It is scary to think our next generation of teachers may be “chosen” based on these TPA’s.
Tomasen

I thought of you as I was writing this,Tomasen, as I’ve been trying to say something about TPA since our brunch last month without just turning it into a rant. And for the record, those questions made me nervous as well. I had to read the passage again and again and still felt so unsure of the supposedly right answers that on several occasions I needed to click on the answer link to see what I was supposed to think. That lack of confidence bears no resemblance to what I usually feel as a reader because, I think, the reading wasn’t mine. How much better would it be if we asked students what they made of a piece and how and why. That would be a much more authentic assessment–though, of course, it might be hard and costly to score, and in the end I fear testing is more about money than trying to develop strong readers.

With this posting, you have successfully followed the evolutionary steps that led from New Criticism into Louise Rosenblatt’s Reader Response theory, the theory ditched by the Common Core because it cannot be standardized easily. As you point out, the multiple choice options are limiting and do not capture a reader’s understanding offering norm referenced “guess” questions instead.
Let us not forget that the CCSS were created under the guidance of David Coleman, the current President of the College Board. When asked about student responses using reader response theory at a NY State Department of Education presentation on the Common Core (April 2011), Coleman said, “As you grow up in this world, you realize people really don’t give a sh*t about what you feel or what you think.”
Actually, I think that is exactly what we should be teaching… the tools for students to tell us what they think about any piece. That reading is an intimate collaboration between writer and reader (he/she is in your head! How much more intimate is that??) and that there is no one answer.

Somewhere online (Burkins & Yaris, Susan Ohanian, you?) I read a wonderful analysis of the David Coleman’s session in Albany where he said that famous ‘people don’t give a shit’ line. Whoever it was pointed out how much of his speech asked his audience to care about what he felt or thought and how much his reading of Martin Luther King was grounded in his own emotions and experiences. In a sense he was demonstrating Reader Response theory at its finest–a thoughtful, attentive reader transacting with a thoughtful writer by bringing his mind and his heart to the text–but he was billing it as New Criticism by denying all that he brought.

Of course, we know it’s impossible for a reader not to bring themselves to a text, as does the writer of a great piece in the Times this weekend that you might have seen, http://www.nytimes.com/2013/06/09/opinion/sunday/no-learning-without-feeling.html?hp. And we know that answering those norm-referenced ‘guess’ questions don’t in any way reflect the kind of thinking that’s really required in college, let alone in the world where more and more we need people who can think outside the box. It’s yet another way that the corporate world is taking us down the wrong track.