Know The System is a bipartisan organization dedicated to elevating public understanding and public dialogue, to take democracy in the United States to the next level.

Scene 38: Sample test questions reviewed

The four senators peered up at the sample test question together. After a moment, Globalist senator Horace Daly spontaneously began to read aloud.

“All right, lemme... ‘Suppose two adjacent countries both have the capability and interest in producing and selling Widgets. But country A can produce each Widget at half of the cost that it takes country B to produce them. For country B to maximize its own welfare, should it’ yadda yadda... ‘everything else being equal’?! How the—who’s going to understand, what does ‘everything else being equal’ mean?”

Jamaal Anderson had been asked to preview a few of the sample questions being drafted by a nonpartisan Fed Ed committee, as early groundwork for the legislation that would be required if the Amendment was ratified. The senators, of course, did not have test design expertise, and so Anderson had expected difficulty. He tried to keep it simple. “Yes, great question. Many of these—particularly for economic theory—many of these require setting up simplified, hypothetical examples, to avoid—”

Joens interrupted, “I don’t see anything wrong with that. It’s just a basic part of critical thinking to be able to separate out parts of the problem that aren’t relevant.”

Florez wished to keep the discussion friendly, so she interposed herself. “Jamaal, why don’t you make a note about that—we can’t have that amount of ambiguity in a question, even if they’re just samples.” Anderson nodded obediently, and wrote a note, while carefully hiding his scorn for Daly’s potshot. Florez moved on. “I particularly was interested in question sixteen, about the tax expenditures, and what your reactions to it are. No, not that one, Jamaal, below... Yes. Here, I’ll quickly read it.

“‘When the Congress enacts a tax credit benefitting a small number of corporations, how is the tax revenue recovered? Explain in one to three sentences.’ Okay, then, Jamaal, could you show us the sample response that—”

Daly blurted out, “What? It’s not! It’s not recovered. Other taxpayers have to pick up the slack!”

“Right,” said Anderson emotionlessly. “That’s right, that would be a correct answer. This is a case where we’re most interested in seeing if that even occurs to them. So we’d be checking for—”

“But what if,” interrupted Senator Arya, “what if they answer, ‘By helping an important new industry sector with the tax credit, the increased jobs and GDP will result in greater tax revenue’ ?”

“I don’t like it,” said Daly. “Seems like a trick question.”

“Sure,” said Anderson in reply to Arya. “That answer would be classified as inconclusive – there’s an implicit acknowledgement in that answer that the revenue needs to be made up, but our question didn’t provide any particular justification for confidence that it would be. So we’d have to—remember that the testing would be adaptive.”

“Sure, sorry. So, for short-paragraph responses, if they clearly get it right, then the test would move on to different, new topics. But if their answer is muddled, or ambiguous, then it goes to more questions within the topic—usually to more elementary questions about what they do or don’t know. So the test adapts as it goes, trying to be as efficient as possible in diagnosing what knowledge the test-taker does or doesn’t have.”

Joens asserted, “I think I do share Horace’s concern though about… this strikes me as being fairly tough stuff for your average American. Or like that one: ‘Approximately how many US citizens are there, in millions?’ Who’s going to guess that one? I don’t even know!”

Arya, also Socialist, piled on, “Agreed! Is the goal to weed out three-quarters of the electorate?”