Now that the discussion of the various tournaments has finally petered out, I thought I'd promulgate some dogma on question writing, seeing as how I'm always yelling really loudly about question quality and such. I encourage other players to contribute, and I'll incorporate good suggestions into the original post. Before I get going, I'd like to point out that many of the things I'll be writing about are settled issues with respect to question writing, but many others reflect my own stylistic preferences, and I'll be sure to try and point out which are my own biases. To begin:

1. Before Writing Questions

Look around you. Look around you. Just look around you. Do you see what we're looking for yet? That's right, the subject of today's lesson is sources.

I'd like to start this guide with a general discussion of what makes a good question-writing source. There are many things that one can use, but obviously not all sources are equally reliable. Notably unreliable sources include Wikipedia and random geocities web pages. Reliable sources include scholarly journals, textbooks, published books, and encyclopedias of various sorts. If you can get your hands on it, the electronic version of the Encyclopedia Britannica is superb for many topics, particularly history. Masterplots are a good resource for literature plot summaries, and there are many topic-specific encyclopedias that are surely available at your local university (or public) library (I'm assuming that virtually all of you have some sort of university affiliation).

Wikipedia may be worthy of a special topic of its own; I mention it here to point out that while it may be a nice source for brainstorming before you actually start writing, you should be thrice-wary of anything you read on Wikipedia. I don't want to get into a discussion of the problems that inhere in the Wikipedia model. Suffice to say that it is not a trustworthy source, though it does in fact contain much that is true. Anything read on Wikipedia should be cross-checked with a more reliable scholarly source. For that matter, pretty much anything you read on the internets that doesn't come from a journal should be double-checked. There are some sites which are pretty reliable and contain some useful information; those are usually associated with universities in some way. The worst part of using Wikipedia as a source is the noted problem of Wikiplagiarism, which is what occurs when people just copy sentences out of Wikipedia. Wikiplagiarism can be easily spotted by competent editors because it contains weird phrasings that usually shouldn't appear in questions, as well as editorializing. I'll have more on style topics below.

Another useful source is online lecture notes (thanks Dwight), whether they come from a class you're taking or just from some stuff professors at other institutions put up. These are generally reliable (they have been in my experience) and ordinarily are pretty easy to get, since many professors just make their notes available to whoever. A notably good source for such notes is MIT's Open CourseWare site, which contains a lot of excellent information on various scientific topics (and some others too).

All this makes it sound like sources are hard to get to and require a lot of work. For the most part, you should be able to get a lot of mileage out of a few books available from whatever library is easiest for you to get to.

2. Deciding What to Write About

This is arguably the hardest phase of the whole process; I know for me it's usually the stumbling block, whereas once I have my topics picked out things generally go pretty smoothly. How can you make a good decision regarding what to write? Read below to find out!

First, you have to know your audience. Are you writing for a novice tournament, a standard invitational, or ACF Nationals? This will determine, to some degree, what answer selections are appropriate in your packet, and later on, what clues you use to construct this question. Second, you should ask yourself whether the answer you are thinking about will make for a good question; although most answers are fine, all other things being equal, some answers don't lend themselves well to good questions and should be avoided if possible. I'll try to outline some of that below. Finally, you should ask yourself whether writing on this particular answer choice is going to require a lot of research. This has less to do with writing a good question and more with budgeting time for doing so. If you find yourself spending an hour per tossup, you're either doing something wrong or picking topics that take far more time to research than they should.

With regard to difficulty, context matters a lot. If you're writing for a tournament like ACF Fall, which is going to include a lot of new players who are, at best, familiar with the high school canon, the goal should be for every tossup to be answered by every team if read to that team to completion against empty chairs. Obviously, this is an ideal; sometimes, people blank on the answer, and sometimes they just don't know it, but at tournaments which are supposed to be introductions to collegiate quizbowl for most participants, the great majority (90%, say) should at least have heard of the answer. So, a tossup on Herbet Spencer would be fine, whereas a tossup on Social Statics is almost certainly too hard.

For harder tournaments, the field of allowable answers is expanded, but it can still be hard to know whether what you're writing is appropriate. I think the following method provides a decent rule of thumb that one can follow: consult past instances of this tournament. If a clue appears multiple times in a tossup, but doesn't appear as a bonus part, it may be all right to make it the hard part of a bonus. If something appears as the hard part multiple times, you could probably promote it to one of the easier parts. If something appears multiple times as a middle or easy part of a bonus, it's probably good tossup material. At first, this might seem like a daunting task, but I think you'll find that if you read packets in practice and go to tournaments, you'll quickly get a handle on what's hard and what's not. At the very worst, by following this method, you will likely err on the side of an easier topic, which is fine. It also makes a difference whether you're writing tossups or bonuses. For example, a European history bonus that included parts on the War of the Spanish Succession, the Treaty of Utrecht, and Philip V would be perfectly reasonable for ACF Fall, but a tossup on Philip V himself would probably be excessive. On the other hand, Philip V might be acceptable as a tossup at ACF Nationals.

I'd also like to address a particular misconception that has formed around some tournaments like ACF Nationals and Regionals. It appears that these tournaments are erroneously viewed by some teams as the perfect opportunities to drag out that tossup on minor works of Selma Lagerloff. Please resist the temptation to do this; before writing, you should really do what I suggest above and check to make sure that whatever it is you're going to write about has a decent chance of being answered. Depending on the results of the above procedure, you might do better by making this subject a bonus part or a clue to something more gettable. Many times, an inexperienced player will hear something that he thinks is tossup-worthy in class and rush to write about it, only to stump an entire tournament. Also, it's generally a good idea to avoid things that are too highly specialized. Some specialist material, especially in science questions, is inevitable, but even within the category, no one should have to have a Ph.D. in the subject to answer the question.

Mostly, I've been writing about tossups. As far as bonuses are concerned, a good way of constructing a bonus is the "easy-middle-hard" model which is now pretty much predominant in quality tournaments. The easy part of the bonus should be answerable by pretty much any team at the tournament. The middle part might be answerable by about half the teams, whereas one might need to have really good knowledge to get the third part; probably no more than 20% of teams should reasonably be able to 30 a bonus. Of course, bonuses vary in difficulty for different teams, but in general, a tournament with well-distributed bonus difficulty will have bottom teams converting not much less than (preferrably around) 10 PPB, whereas the best teams could conceivably top 20 PPB, and bonus conversion should be well-correlated with overall place in the tournament.

This brings me to the somewhat peripheral but still important issue of consistency. This is less important for tossups, which contain internal difficulty gradations, than it is for bonuses. If all the science bonuses at a tournament are super hard, you're going to screw some teams really badly. For this reason, following the above bonus writing model is a good way to ensure that you aren't making any one category too hard. Some teams are going to just not know some areas, and that's fine, but if some bonuses are consistently harder than others, that has the potential to really damage teams. In accordance with this principle, the practice of the impossible third bonus part should also be avoided. The possibility of getting 30 points on a bonus should always be there for the talented team; overly difficult bonus parts effectively turn 30 potential points into only 20, which tends to really narrow the gap between teams.

Ok, this is the part where I address some objections to my own advice. There are some people who adhere to the view that one should write about things people are likely to know about instead of things that have come up before. Matt Weiner is one notable proponent of this view. I'd like to note that I don't completely disagree with this, but I am of the opinion that things that have come up in packets before are things that people are likely to know. One might make the argument that this rewards people who read or hear lots of packets, and this is of course true. But I contend that these people will be rewarded anyway, because if you write about what people know, and people play on those packets, the people who played on them will have the same advantage when the same topic comes up, regardless of the reason for why it comes up in quizbowl.

Ok, with that long preamble out of the way, we can get down to the business of actually writing some questions. I'll try to provide some guidelines for general question writing and also dissect some questions to demonstrate what makes for good and bad tossups and bonuses. Here we go!

3.1. Tossups

The first principle of good tossup writing is pyramidality. What is pyramidality, you ask? Pyramidality is simply the principle of putting harder clues earlier in the questions, with the idea being that the more knowledgable team will answer the tossup first because they know more about the subject. I like to think of the pyramid as representing the proportion of teams that would know the answer to the tossup at some particular spot in the question. So the apex of the pyramid represents a specialist in the subject, and the base represents most people who play quizbowl and have heard of the answer. Ideally, there should be a relatively smooth gradation between the apex and the base, although of course real tossups resemble ziggurats (or perhaps mastabas) more than actual pyramids. Nevertheless, the idea is clear: there are gradations of clue difficulty in tossups which is intended to distinguish between more knowledgable and less knowledgable players.

There's no magic formula for writing tossups, unfortunately. If there were, we could have computer write tossups for us and the problem of generating questions would be solved once and for all. While Ray Luo labors to come up with such a machine, I'll just go ahead and describe, in general terms, what the various levels of the tossup pyramid might look like and what kind of clues they might contain. Keep in mind that this is not meant to be an exhaustive list, but I hope it will prove instructive.

The first clue of a question should contain information that is unlikely to be known by anyone who has not spent at least some time studying whatever the answer might be. When I say "studying" I don't necessarily mean studying in school, but rather this word should be read as meaning "absorbing information about," and I write "studying" for short. So, reading a book by or about, say, Herman Melville, qualifies as studying in this sense. If I were determined to learn a lot about Moby Dick I would certainly read the work itself, but I might also read various critical interpretations of it. That kind of clue would be pretty useful to me then; it would give me an advantage over someone who has just read the book but not the criticism. In science, such a clue might reference a little-known application of some effect, for instance. In history, it could be a clue about an academic historian's interpretation of some event. And so on.

I think this is a good point to insert a brief caveat about such first clues. It can be hard sometimes, especially when employing clues of the type "Joe Blow said this-and-that about this work," to decide if a clue will actually be useful to anyone. I address this below in the section entitled "The Usefulness of Clues." Moving on:

The next couple of clues (what I'm going to refer to as "the middle") should successively narrow the answer space down to the actual correct answer of the tossup. This is the part that is usually the hardest to execute, since clues that are really easy or really hard might be obviously so, but the arrangement of middle clues is tough because you're trying to create a relatively fine gradation from the start to the end of the tossup. It's hard to say exactly what those clues should be, but maybe some examples would be instructive. If the question is on a work of literature, (say, the very same Moby Dick), you might describe a somewhat-little-known episode in the book, followed by the actions of some better-known characters, followed by some even better-known names or events. In a science tossup, you might talk about a lesser-known consequence of some effect, then maybe the equation that describes this effect, followed by a better-known consequence. Or something like that.

The key component of clue ordering is to ask yourself, what would a person who knows a lot/a decent amount/not so much/very little (pick one) be likely to know about whatever it is that you're writing about. This requires some intuition in some cases, because unless you've been around the circuit for a while, you probably don't know that certain clues for certain things are considered "stock" and thus shouldn't be anywhere but in the end of a tossup. However, for many situations, you can figure this out; in literature, for example, it should be straightforward to see which characters are minor and which are major, which events are central to the work and which are just vignettes. If you're writing on some idea or theory, the name of its formulator probably belongs in the end, unless it's obscure. Likewise for common definitions (like in a math question), or major scenes in novels, or most famous lines of poetry, or whatever. You should quickly get some sense for what the clue ordering might be like by reading a bunch of tossups; that's not to say that you have to slavishly emulate what came before you (maybe you found a neat new clue or something) but it does mean that if you follow the conventional clue order you will probably write a decent tossup.

The giveaway is self-explanatory. It should contain the most famous bit of information about whatever the answer is. It should usually not, in my opinion, contain tangential clues (the James Webb telescope comes to mind) about what the answer sounds like or anything that requires lateral thinking. Note the emphasis on "usually." Sometimes an oblique reference to another thing that may help someone get the question is fine (mostly I'm thinking of common-link tossups), but most of the time it's not.

3.2. Bonuses

There's a lot less to say about writing bonuses than about writing tossups, mostly because the confusion associated with pyramidality is to some extent absent in bonuses. Bonuses should generally follow the "easy-medium-hard" model of bonus parts and the majority of the bonuses you write should be of the 10-10-10 variety. We can loosely define "easy" to mean that we would like 90% of the field to convert this part, "medium" to mean that maybe 40-50% of the field will convert it, and "hard" to mean that perhaps 10-20% of the field should convert it. These are not hard-and-fast numbers; they are just vague signposts to explain what is meant by the difficulty levels. Obviously, these may be adjusted downward or upward for various tournaments, but I think they are generally a good indication of how to break down bonus parts by difficulty.

In writing bonuses, avoid the "impossible third part." A sufficiently comptetent team may not get 30 points on each bonus, but the potential for 30 points should be there. A bonus part on something that no one who is not an expert on some particular subcategory wouldn't even have heard of is a bad idea. It is also bad to have the bonus difficulty differ substantially across categories within a tournament (and especially within a single packet). Such bonuses tend to screw an unfortunate team and can make the difference in close games. Of course, there will be variability between teams, and some categories will be known better than others, but the potential for the structure of the packet (rather than the clue content) to influence the outcome of the game should be minimized.

3.3. Style

This may be the most subjective section of all, but I think it's necessary to say something about the stylistic issues inherent in question writing. It should go without saying that packets should be grammatically correct, but they're often not, so please read your questions out loud to yourself to make sure the sentence structure makes sense. We all make mistakes of course, but some tournaments I've been to have featured missing words or spelling errors or grammatical mistakes in almost every question. This is the minimum that can be expected of any writer. I'll say it again: PROOFREAD!

The first clue of a tossup should always be uniquely identifying (to the best of the writer's abilities; sometimes information may match two answers despite the writer's best attempts). Also, the first pronoun should refer unambiguously to the answer and should appear IN THE FIRST SENTENCE. I can't stress this enough! If the first sentence of a tossup finishes and I don't know what category the answer goes in (person, place, or thing) then this question is bad because it doesn't make clear what it's looking for.

More generally, there are different ways to word questions, and different approaches to writing. On one level, you may choose between simple, declaratory sentences (e.g. "This guy wrote about Joe Blow in Work X. Then he wrote about Jane Doe in Work Y.") or compound sentences with clauses and subclauses (e.g. "This guy wrote about Joe Blow in Work X, after which he used his experience in the Pastry War to write about Jane Doe in Work Y."). You get the idea, I'm sure. I personally like the latter style, but some people like the former. I don't think there's anything wrong with either one, provided you are economical about your word choice. Since questions are typically limited in length by editors, you should make sure that all the words you use are meaningful. If a clue can be rewritten with shorter words in place of longer ones, you should probably do that. This saves space and helps you pack more information into a tossup.

Within clues, there are different ways of ordering information. For example, notice how above I used the form "wrote about Joe Blow in Work X," rather than, "His Work X concerns the adventures of Joe Blow." This is deliberate; any random quizbowl player is less likely to know the main character of any given novel than the name of the novel itself. And so on. I recommend using this construction to create pyramidality within clues themselves.

Finally, a word on gender. For a long time, there has been what I think is a nasty trope in quizbowl, which involves writing things like "this writer," or "this scientist," or whatever in an attempt to disguise the fact that the person in question is female. This is annoying, because instead of saying "she," which is shorter, you're now using up more space and you're not even hiding the fact that it's a woman, because everyone now knows to watch for this. So my suggestion is that people use "this [blank]" in the first couple of clues, but then just transition to the male or female pronouns. In most cases, the field of women whatevers is not nearly so narrow as to shrink the potential answer space down to something obvious.

3.4. The Usefulness of Clues

This is another problem that I see come up over and over again. Clues get used in tossups that are just not useful to anyone. Egregious examples of these kinds of clues are "Joe Blow notably studied this thing." These clues are unhelpful because unless you know exactly who Joe Blow is (and even then, he might have notably studied several things), you can't possibly get anything out of this. Such a clue could be converted into a useful one by writing, "Joe Blow notably addressed this issue in his tract 'On Stuff'," provided that "On Stuff" is actually about only that one thing. Clues containing numbers in science questions (e.g. "Its albedo is a million," "This quantity is 234.9 for water," etc.) are also useless, because no one memorizes these numbers. Vague clues like "In this novel one character goes to the store for some lettuce," are also not useful since they are probably not uniquely identifying. Clues like "This river is 543 miles long" are also dumb because no one, not even geography wizard Jeff Hoppes, knows these things. In short, if you are tempted to pad a question with the kind of information found in an almanac, don't. Make sure that all your clues convey useful information that someone could actually get the question from.

All this leads me to the culmination of this lengthy opus, which is perhaps the most important part.

3.5 What to Avoid, and Why

I've done my best to cover how questions ought to be written; now I'll point to some mistakes people make in writing questions and explain why you shouldn't do these things.

Tossups

Giveaways in the first line are bad. Since pyramidality is good and this makes the question not pyramidal, it is bad. Ok, that was the obvious one

Stock clues. These are clues that were abundant in old, pre-2002 or so quizbowl, like Michael Faraday being apprenticed to a bookbinder or Saussure writing about Indo-European languages (must be the most reused lead-in ever). These clues are bad because instead of learning anything about the subject at hand, people just memorize clues which end up being recycled by lazy writers. Sometimes, the recycling of these clues by beginning writers is inevitable, but editors should know better. Keep in mind that not ever lead-in is a "stock" clue, since there are plenty of different ways to write a tossup on, say, Herman Melville, and some of them may recycle clues about his lesser-known works. However, clues that come up again and again (the Saussure clue is bad because it's his only other published work, so it gets referenced all the time, becoming "stock"). My suggestion for avoiding this is to search the Stanford Archive for the topic and then see if the lead-in you want to use appears many times in older packets. If it does, don't use it.

Question Transparency. This is a huge one, and I didn't really address it above. I'll do so now. The idea of question transparency is that if you write the question poorly (or if you choose your answer poorly), it will become very obvious what the answer is without the player actually having to know anything. Case in point, picked at random from whatever Moon Pie packet I happened to be looking at: a tossup on Grimm's Law, which mentions something about exceptions to it being described by the spirant law and later in the first sentence mentions voiceless dentals. Now, I know nothing, but nothing about voiceless dentals. All I have is a little radar that beeps "linguistics" when I hear that line. Also, I speak English well so I know that laws have exceptions. What is a law of linguistics? Why, Grimm's law is! This is the classic example of a question that rewards lateral thinking and knowing that an answer vaguely matching the characteristics being described exists. Basically, if someone can "figure out" the answer based on the fact that it becomes obvious (for reasons having to do with poorly placed clues, linguistic hints, whatever) within the first two clues or so (as opposed to because they actually know it from information provided), then the question is bad.

Editorializing and needless verbiage. Everything you write in a tossup should contain information that helps players answer the question. I've seen at least one packet from Moon Pie that contains all sorts of needless editorial content and mountains of words that don't help at all when playing. Any words that don't contain useful clues are just hindrances for the players; moreover, since space is limited, these questions tend to suffer from sharp difficulty drops and create buzzer races. Speaking of which...

Buzzer races. If you write a question that contains clues on nothing but Melville's poetry and then a giveaway on Moby Dick you have engineered a stupid and pointless buzzer race, because one either has to be a Melville expert to get this question or wait until the end and hope one is fast enough to beat everyone else who will buzz at the same time. This goes contrary to the idea of pyramidality and having clues which get progressively easier. It also fails to discriminate between two teams with potentially different Melville knowledge, neither of which contains Melville experts.

List tossups. These kinds of questions violate several rules of good question writing, including the one that mandates that the subject of the answer be revealed in the first sentence. This has the added effect that potentially any commonality between the listed things should logically be an acceptable answer. For example, a question that begins "Julius Caesar, John F. Kennedy, William McKinley," with the intended answer being "they were all assasinated" would be terrible anyway, but logically would have the answer "they were all men" be acceptable. This is stupid and wrong, so don't write these questions, as nothing good can come of it. There does not exist a list tossup that cannot be remade into a better question with the same answer just by changing the wording of the question.

Bonuses

List bonuses. These are dumb and uninteresting. These bonuses amount to almanac memorization and tell you nothing interesting about the subject at hand. Authors from works, VSEPR shapes, rulers from dates of rule, whatever. All these are equally dull and should be avoided. The presence of such questions is one of the best indicators of lazy question writing.

Almanac bonuses. These include memorizing Nobel Prize lists, physical constants, and so on. A relative of the list bonus, and bad for all the same reasons.

Wacky bonus forms. The 5-10-15 is deprecated; so is the 30-20-10, which is better as a tossup anyway. The 5-10-15 is bad because it penalizes the team twice: once for not knowing the hard part, and once again for making the hard part worth more. The 10-10-10 is king; you should follow this bonus convention unless you have a good reason to change the value of some of the parts (such as if you have two answers required for one part and you'd like to make them 5 each).

Varying bonus difficulty. Already discussed above, but all your bonuses should aim to be roughly the same in difficulty. They won't be of course; some teams will know some categories and not others, and that's ok. But if all teams are doing poorly on a certain category, then your bonus difficulty is unbalanced. Also, randomly difficult bonus parts in supposedly easy tournaments are also bad.

Calculation bonuses. These are bad for the same reason that the 5-10-15 is bad, only more so. These bonuses tend to doubly punish teams that lack a specialist since they require you not to just be somewhat familiar with a topic but to know the actual formula. They also punish specialists who may just not be very fast when it comes to calculating something, or who can't remember the exact formula.

Marzipan, a kick in the balls, both or neither. This is a lousy format because it too punishes a team multiple times, especially if it's used in a science context. These bonuses are annoying because they tend to contain trick questions and encourage guessing. While it's logically possible for such a bonus to not be awful, chances are that if you write one, it will be awful. So don't.

Ok, this has been long for me to write and for you to read, so I'll wrap it up right here. Some closing comments: contrary to what Charlie Steinhice will tell you, standards are not "optional" and while there remain debates about specific things in question writing, most of the core issues really are settled in good tournaments. The outstanding debates are like the outstanding debates about evolution: no one with any expertise in the subject disagrees on the core, though they may debate the specific mechanisms. I hate to make it seem like I'm some sort of quizbowl pope speaking ex cathedra, but it's pretty much true. These issues have been hashed out over the years by experienced players and a lot of thought has gone into making quizbowl questions good. If you are a new player, do not ignore these guidelines and attempt to reinvent the wheel. Rather, take note of these rules, read them once, maybe twice, and then read them again when you feel compelled to write a crappy question.

I welcome everyone's comments on this document, including any suggestions for things I might have missed. I will incorporate good ideas into this original post, which will hopefully be stickied for future reference. If it occurs to me to add something, I'll do so (I plan a little addendum on time management in editing in the future). I also plan to do a dissection of good questions and bad questions somewhere down the line in this thread to give people an idea of what distinguishes one from the other. If other experienced players would care to pitch in with their own ideas, I'm sure that working together we can make this quizbowling world a better place.

Last edited by grapesmoker on Mon Apr 30, 2007 11:59 pm, edited 1 time in total.

One notably reliable source Jerry fails to mention is online lecture notes. If you're lucky enough to find online lecture notes for a lecture dealing with your answer choice, this will more or less give you both lower-level and mid-level clues and a good idea for ordering them. Generally, I tend to trust any pdf/PowerPoint from a .edu domain name that appears to be written by a professor who knows what he's talking about.

Also, "niche topics" are a very tricky group. It is absolutely true that you should not be using these things for tossup answer choices, since it is highly unlikely that anyone in the field is going to convert it. However, that same inaccessibility makes it a good lead-in. Now, I'm not advocating that you start every tossup you write with a clue about 14th-century Bulgarian economic history, unless you're writing some kind of theme packet on that (theme packets are an entirely different thing and I believe there is already a thread somewhere about that). But, every once in a while, starting a tossup on THING PEOPLE HAVE HEARD OF with a lead-in from PET TOPIC NO ONE ELSE CARES ABOUT can be a good idea.

What is the general consensus on having Bonus Parts that cross distributions, i.e. a geography bonus that asks for a work of literature that makes significant reference to the area in question-to be avoided all together, or only to be used sparingly?

Also, there were some complaints after ACF nationals about tossups whose answers were general terms or concepts like: novel or spider. Any particular rules regarding such inclusions?

And other than 10-10-10, are there any other acceptable point formats? Perhaps 5-5-10-10 or something?

I will add the suggestions made by Dwight into the original post sometime today.

SnookerUSF wrote:What is the general consensus on having Bonus Parts that cross distributions, i.e. a geography bonus that asks for a work of literature that makes significant reference to the area in question-to be avoided all together, or only to be used sparingly?

Well, I would be on the side of counting that as literature, but it all depends on the clues, really. If the bonus is obviously held together by the literature (say a bonus that tied Alexandria, Avignon, and Corfu by means of Lawrence Durrell), then I would call that literature. Otherwise, it might be geography. For me, I guess the decider is whether someone is more likely to answer the bonus parts by knowing the literature or the geography (or whichever of the two categories you're mixing). Whichever one is easier to get should probably be the category in which the question falls.

Also, there were some complaints after ACF nationals about tossups whose answers were general terms or concepts like: novel or spider. Any particular rules regarding such inclusions?

I actually like these sorts of questions, since they allow you to incorporate a lot of interesting material and still have the question be pyramidal and gettable in the end. I think they shouldn't predominate in packets, but as long as you can pull it off by being unambiguous and having a decent giveaway, I think that's fine.

And other than 10-10-10, are there any other acceptable point formats? Perhaps 5-5-10-10 or something?

The 5-5-10-10 can have its uses if you find some really cool clue for one of your parts and you just really feel the need to use it and thus end up with 4 parts. I don't write such bonuses, typically, but I think they can be ok if used sparingly.

I also plan to add computation bonuses and "x, y, both or neither," bonuses to the list of things to avoid. Stay tuned.

What is the general consensus on having Bonus Parts that cross distributions, i.e. a geography bonus that asks for a work of literature that makes significant reference to the area in question-to be avoided all together, or only to be used sparingly?

Personally, I have no problem with this, as long as you're just generally fair about distribution. If the bonus is really 2 parts geography and 1 part lit, that's fine, but I would maybe make sure to include a geography tossup in that round (if you have the choice, granted often you're not editing - in which case it's the editor's job). Basically, I just try to be careful not to screw over someone who might like a particular subject - if I feel the packet as a whole does that, then I compensate.

Also, there were some complaints after ACF nationals about tossups whose answers were general terms or concepts like: novel or spider. Any particular rules regarding such inclusions?

I think the general consensus is that there's nothing wrong with these types of questions - in fact they can be really good - they're just tricky to write. Whatever the answer is, I'd say there should be at least 2 and probably 3 quite gettable things related to that topic - if it's a spider myth tossup, that's okay, because you have Arachne and Anansi and probably some other stuff. Also, you really have to look critically at the totality of the clues and think "am I encouraging someone to buzz and just say something logical without knowing anything?" If you're answer is spider, you probably don't want a part that says "one of these is hairy in myth x" and "one of these has eight legs in myth y." It's really easy to think "hey, I've got this cool concept idea for a tossup because I have all these interesting or important books/things related to this concept"...while that may be true, you have to carefully consider when people are really going to buzz and why (with knowledge or with a logical guess). If you're in doubt, you might just decide to use the clues you'd use for a concept tossup as parts in a related bonus.

It's also notable that some tournaments have "other academic" distributions. I think it's fine to put interdisciplinary questions in those (for example, leadins on literature about a work of art, terms shared between fields, etc.)

Here is some additional information that I always found helpful from Roger Bhan's Question Writers' Manifesto. Some of the original I deleted as no longer relevant to current trends:

"QUALITY:
I guess this is the part most people have problems with. Here are a few general guidelines before I get into anything specific:
â€¢ First, please make sure your questions are factually correct. DO NOT submit a question that might not be entirely correct.
â€¢ Is there more than one possible answer? Is it possible to give another answer before a certain point in the question? These things must be specified in the answer to the question if applicable.
â€¢ Most importantly, please write about things that most people have heard of or at least would be interested in hearing. Expansion of the "canon" of quiz bowl knowledge is an inevitable and good thing, but must be done in small increments lest we alienate players. If you'd like to introduce something new and never heard before, do so in a bonus part first.
Toss-ups should be pyramidal in that they start with obscure clues and get progressively easier until a giveaway follows the phrase "for ten points." In this manner, toss-ups reward depth of knowledge about a particular subject. Here's an example:
Literature: American
This book is prefaced with a verse from a poet that was a character in the author's first novel. The pianist Ewing Klipspringer and the title character's employer, Dan Cody, are minor figures in this novel. All through the story, the eyes of Dr. T. J. Eckleburg stare over the ash heaps as the characters travel between East and West Egg. For ten points, identify this novel about a man in love with another man's wife by F. Scott Fitzgerald.
Answer: The Great Gatsby

SOME CATEGORICAL IMPERATIVES:
Here are a few guidelines regarding the specific categories. Please adhere to these strictly.
Literature: You may not write more than one author biography toss-up per packet. Please keep your subjects diverse and interesting. Novels, plays, short stories, essays, journals, literary haracters, and poetry, among many other things, are acceptable topics, as are authors in moderation.
Science: Please do not write science biographies that have nothing to do with a scientist's relevant work. While it is interesting to know that Michael Faraday was a bookbinder, that is nothing more than SCIENCE HISTORY and not real SCIENCE. In any case, you may not write more than one science biography toss-up per packet, and that must fall into the miscellaneous category. DO NOT write questions about elements. Nobody cares about the conducting properties of indium or that Hans von Dorkmeyer discovered vanadium in his butt in 1393. Elements are a cheap way out of a potentially good chemistry question.
History: You may not write more than one history toss-up about a specific person.
Social Science/Geography: You may not write more than one social science toss-up about a specific person.
Religion/Mythology/Philosophy: You may not write more than one biography toss-up among these three categories. Stay away from philosophy biographies if possible.
Fine Arts: Do your best to avoid biography toss-ups in this category. It is very easy to write about pieces of music and art works/movements, so there really shouldn't be a need to write biography toss-ups."

allythin wrote:DO NOT write questions about elements...Elements are a cheap way out of a potentially good chemistry question.

I don't see things this way; I may be in the minority here, but there isn't that much in the Fall-level canon that isn't introductory organic chemistry, so unless we want the ACF Fall editors to cut like 10 tossups on entropy, we need to relax these standards a little bit. That said, it is difficult to write a GOOD question on elements, mostly for the same reason that it is difficult to write a GOOD question on rivers, that being that 90% of the clues most people include are almanac crap no one cares about. People should not be discouraged from writing element questions, but they should be advised to use "real science" clues (e.g. This element acts as a catalyst in reaction X) instead of the fake science stuff that comes up.

Also, I don't get the "stay away from writing on a person at all costs" maxim in that manifesto. Good, pyramidal tossups can be written on people (especially in literature, science, philosophy, arts, and social science), and these questions are more likely to make it through the editing process for novice and some invitational tournaments than a comparable tossup on that person's works (as Jerry said, Herbert Spencer is fine for most tournaments, Social Statics is not). The main thing here is that the question about the person should actually focus mostly on the works of that person (e.g. Joe Blow, his classmate at Random School, served as the model for the waifish figure on the right of his Painting Y), and biography should only be used if it is directly important in the person's work. There is a big difference between Category Biography and Questions about People, and I think a lot of people confuse these to one extreme of writing lots of Category Biography or the other extreme of dismissing any Question about People as horrible.

I think the idea was to prevent half the packet's answers from being people's names. This encouraged diversity in topics/answers. If you notice, he's not saying questions about people are bad in most contexts, just limiting the number overall per packet.

allythin wrote:DO NOT write questions about elements...Elements are a cheap way out of a potentially good chemistry question.

I generally agree with this, but I propose one notable exception. At tournaments with difficulty levels (maybe significantly) higher than ACF Fall I endorse and enjoy the occasional biology tossup on an element. While one can critique this idea by saying, "There aren't that many elements that have recognizable enough functions to justify a tossup," I'm pretty sure the answer space is varied enough to avoid transparency, especially because the clues tend to be fairly specific and identifying.

Hoss Cartwright wrote: generally agree with this, but I propose one notable exception. At tournaments with difficulty levels (maybe significantly) higher than ACF Fall I endorse and enjoy the occasional biology tossup on an element. While one can critique this idea by saying, "There aren't that many elements that have recognizable enough functions to justify a tossup," I'm pretty sure the answer space is varied enough to avoid transparency, especially because the clues tend to be fairly specific and identifying.

I'll second this notion, with the caveat that I'd throw something at anyone who writes a tossup on carbon in this vein.

Yeah, I think the Bhan piece really overstates how much you should avoid writing on specific people...probably because it comes from a time when most "biography" tossups featured the same hackneyed and boring clues and just basically sucked balls.

It would be perfectly defensible, I think, to write an entire packet just on people - as long as you use in-depth clues that give specific information (not just strings of titles, but characters/theories/descriptions of paintings, etc.) and you should also avoid using too many unimportant tidbits of trivia about the person's life (stuff generally unrelated to their academic contributions). Maybe such a packet would be a little weird, but it could be fine. So, don't think that tus on people are disfavored or should necessarily be limited.

cvdwightw wrote:...there isn't that much in the Fall-level [chemistry] canon that isn't introductory organic chemistry...

I strongly disagree with this. I think chemistry can often be written better but I know for sure there there are any number of laws, principles, variables, and other non-organic (or not exclusively organic), non-element chemistry answers accessible at that level. In fact, due to the widespread teaching of chemistry in high schools, it seems to me that there are a lot of thing accessible at that level.
That said, I don't think element questions should always necessarily be banned. They should just be held to the same standards as other questions (i.e. that they must be interesting, be relevant, be important, and not comprise too large a portion of the chemistry distribution.) Unfortunately, it's definitely the case that most element questions drastically fail on many of these criteria.

I'll have more to say about the guide later, but there's one little thing that I've asked about elsewhere on this forum and got no answer:

grapesmoker wrote:
The giveaway is self-explanatory. It should contain the most famous bit of information about whatever the answer is.

Paul and Seth's primer says essentially the same thing, and adds: "Don't be afraid of making the giveaway too easy". If this is the current paradigm, then it's something of a departure from quizbowl from back in my day, when "curved yellow fruit" was a popular catchphrase that was used to mock College Bowl.

The New York Times wrote:
The questions are distributed among academic subjects, sports, pop culture and current events and steer clear of the giveaways, even at the ends of questions, that the College Bowl often indulges in, infuriating purists.

"There was a famous one in 1988," said Michael Zarren, the captain of the University of Chicago team, which is considered the nation's strongest. "It began with some obscure description of a tropical plant, and then finished with, 'For 10 points, name this curved yellow fruit.' "

Is it fair to say that today's "purists" have the exact opposite point of view about "giveaways"?

As a practical matter, if you write a tossup about Washington, D.C., should you really end it with "capital of the United States"? The guidelines appear to agree that that is what you should do. But it seems almost insulting to end a question that way. If the earlier clues are mostly about architecture, for example, you might be a little subtler and end by mentioning the Supreme Court and the Capitol. But that wouldn't be the most famous bit of information about Washington, so it would violate the rule.

cvdwightw wrote:
it is difficult to write a GOOD question on rivers, [...] 90% of the clues most people include are almanac crap no one cares about.

I find this surprising, because rivers are one thing I find it easy to write tossups about: where they start, where they end, their tributaries, any dams or waterfalls or rapids, bridges over them, cities they pass through, boundaries they mark. Do these make bad clues? What kinds of clues about rivers are GOOD?

vetovian wrote:As a practical matter, if you write a tossup about Washington, D.C., should you really end it with "capital of the United States"? The guidelines appear to agree that that is what you should do. But it seems almost insulting to end a question that way. If the earlier clues are mostly about architecture, for example, you might be a little subtler and end by mentioning the Supreme Court and the Capitol. But that wouldn't be the most famous bit of information about Washington, so it would violate the rule.

Wouldn't writing a tossup on Washington DC violate the rules about answer selection and considering your audience? If you're writing for middle schoolers, perhaps a giveaway of "It's the nation's capital, kids!" would be appropriate. For a college audience, Washington DC as an answer (assuming you haven't written, say an architecture tossup with DC as a common link, which I would still be a bit dubious about) is, obviously, too easy, making your point about the ridiculously easy giveaway moot.

Mountain yellow-legged fr wrote:say an architecture tossup with DC as a common link

Even if one wrote such a tossup, the correct thing to put at the end would be the most famous thing about the architecture of DC (like the White House), rather than that it's the capital of the USA, since this is what's appropriate to the category.

It depends what clues came before the giveaway. If you absolutely must write a tossup on bananas, it's inadvisable to go from nobody buzzing (this sounds a vague description of a tropical plant of some kind) to everybody buzzing (I KNOW A CURVED YELLOW FRUIT!) The giveaway should be the easiest clue in the question, but if it's way easier than the clues before it, you face this problem.
So, in short, you can go as easy as you want with the giveaway, but it's not good if it's way easier than the clue before it, which shouldn't have been way easier than the clue before it, etc. That holds equally well for any question you might want to write.

vetovian wrote:As a practical matter, if you write a tossup about Washington, D.C., should you really end it with "capital of the United States"? The guidelines appear to agree that that is what you should do. But it seems almost insulting to end a question that way. If the earlier clues are mostly about architecture, for example, you might be a little subtler and end by mentioning the Supreme Court and the Capitol. But that wouldn't be the most famous bit of information about Washington, so it would violate the rule.

Wouldn't writing a tossup on Washington DC violate the rules about answer selection and considering your audience? If you're writing for middle schoolers, perhaps a giveaway of "It's the nation's capital, kids!" would be appropriate. For a college audience, Washington DC as an answer (assuming you haven't written, say an architecture tossup with DC as a common link, which I would still be a bit dubious about) is, obviously, too easy, making your point about the ridiculously easy giveaway moot.

The answer "Washington DC" wouldn't necessarily be inappropriate. There are many tossups you could write with that answer--a literature or trash tossup on works partially or fully set there, a geography tossup featuring clues about parts of the city tourists don't necessarily visit, whatever. But the giveaway thing remains an issue I suppose, depending on the tournament. I'd have no problem with a washington dc tossup at acf fall with the giveaway ftp name this city, the capital of the United States. The problem with "as easy as possible" is that this often means trash, and while the rule for naqt may be "if you wait long enough, it'll devolve into trash," the rule for ACF and most mACF invitationals is "academic is academic, trash is trash."

vetovian wrote:I find this surprising, because rivers are one thing I find it easy to write tossups about: where they start, where they end, their tributaries, any dams or waterfalls or rapids, bridges over them, cities they pass through, boundaries they mark. Do these make bad clues? What kinds of clues about rivers are GOOD?

The simplest definition of a good clue that I can give is the following:

A good clue will perform two functions. First, the clue will provide uniquely identifying information so that people with knowledge of that clue can buzz in and be assured of being correct. Second, that clue will help people who have some inkling of the answer, but no concrete knowledge yet, to narrow down the answer space, so they are cued toward quicker recall of the answer once a piece of information they do recognize comes up, or at least a reasonable guess at the end of the question if they've heard of the thing. It's not enough to just "wait for a clue you know, then buzz" anymore, as we've seen with umpteen reflex buzzes that go horribly wrong. If the preceding clues make (reflex buzz answer) not a part of the "possible answer space", then it's the player's fault for a stupid reflex buzz, but if they don't, then it's the writer's fault for writing a bad question.

So: a good lead-in will narrow the answer space from "everything that we could possibly write a question about, and then some" to person, place, thing, idea, etc. With rare occasions (e.g. "This dude wrote about Character X in Book Y", but we're looking for someone whose literary career is secondary to his other accomplishments) it will narrow down the subject of the tossup. Mid-level clues will successively narrow down the set of answer choices someone could give with an educated guess (e.g. a thought process might go from "This is a person who wrote a book" to "This describes someone who wrote stuff in the past two centuries" to "This is clearly describing an English novelist from the past two centuries"). At all points these should serve to "cue" people away from obviously wrong answers and toward the correct one. Giveaways should reduce the "possible answer space" to one (e.g. "This is a law of thermodynamics that says entropy is always increasing), at which point one of only three things should happen with every single player. The first is that the player has knowledge of the clue and buzzes, either getting 10 points or being beaten in a buzzer race. The second is that he recognizes the clue, but cannot accurately remember the answer, and the third is that the clue rings no bells and the overlap between "possible answer space" and "answers the player knows" is zero (e.g. suppose a player in a match between two literature-poor teams has narrowed the "possible answer space" down to "This is a contemporary Portuguese author"; either the player's going to recognize it's Saramago and buzz, take the entire time trying to come up with Saramago, or have never heard of any contemporary Portuguese authors).

My issue with most rivers questions is that this "almanac knowledge" often doesn't help narrow down the possible answer space at all. If you write a tossup on the Yenisey River, 90% of your audience is going to have the "possible answer space" narrowed down to "Siberian Rivers" in about two sentences, and then have no further revision to that set. It's not that good tossups on rivers can't be written, it's that many almanac clues (start point, end point, length, tributaries, etc.) result in no substantive cuing beyond what's already been established as the "possible answer space", so there is essentially no utility in hearing anything between the point that answer space is established and the FTP giveaway, which presumably finally has a clue people know and will buzz on. Also, these can quickly lead to transparency (OMG it's a major South American River that isn't part of the Amazon Basin), which is bad, no matter how well-written the actual clue that provides that transparency is.

One thing that Dwight just mentioned that I think warrants highlighting and expounding is this:

It's not enough to just "wait for a clue you know, then buzz" anymore, as we've seen with umpteen reflex buzzes that go horribly wrong. If the preceding clues make (reflex buzz answer) not a part of the "possible answer space", then it's the player's fault for a stupid reflex buzz, but if they don't, then it's the writer's fault for writing a bad question.

If you've got a bunch of technical clues on platinum that neither team buzzes on and only narrow the answer space to both teams in any particular match down to "it's a metallic element," then you say it dissolves in aqua regia, you have written a bad question. If you have a bunch of clues for Parmenides that narrow it down to "Greek philosopher" then the next clue is "On Nature," you've written a bad question. And so on.

You must use qualifiers in these cases (It's not gold, but it dissolves in aqua regia...; His "On Nature" differs from those of Empedocles and Anaxagorus in that...). Or simply do not use them.

If these guidelines are intended to be read by new players, I think we need to define terms that are in common use among us but may not have an obvious meaning to the uninitiated. "Pyramidality" is such a term, and so Jerry gave a nice definition of it in the original post here. But an undefined term used in the original post and also in some comments by others here on the subject of tossups is "answer space" or "possible answer space". A couple of notes about this phrasing:

1. It comes out of the jargon that engineers and physicists use, and it is a natural term for them. But I think most other people are intimidated by any phrase with "space" in it, and for them, the word doesn't call to mind the same intuitive notion that it does for us.

2. Perhaps more importantly, I think it needs to be made clear that what we're talking about when we refer to "(possible) answer space" is not quite "the set of all possible answers to the question up to this point", because as the original post says, "The first clue of a tossup should always be uniquely identifying". "Possible answer space" is different for each player and means the set of all possible answers that haven't been ruled out so far according to that player's knowledge. We all understand that, but I think it needs to be said. Maybe someone else can come up with a more elegant definition.

At one point in our discussion about VETO last year, Jerry and I had a misunderstanding over this distinction. I was trying to say that at a certain point in a certain tossup, the possible answer space for most players would be fairly large, but he thought I was saying that the set of possible correct answers was large. I didn't use the term "answer space", but he presumably would have understood what I was trying to say better if I had. So it's a good concept to have a term for, just like pyramidality.

I hope everyone intuitively understands what I mean by "answer space." I'm not sure that a rigorous definition of it is necessary. All I have in mind is just "the things it could be at this point provided you don't know the actual answer," so if you're hearing a Melville tossup, you might know halfway through that this is an American author, but you wouldn't know much more than that. The next clue might tell you that this is a 19th century American author, and so on. I'm not going for mathematical precision here.

Dwight's first two paragraphs are great, but I completely disagree with the third.

many almanac clues (start point, end point, length, tributaries, etc.) result in no substantive cuing beyond what's already been established as the "possible answer space",

For me only "length" from among these really qualifies as an almanac clue. These are the clues ("it's eight hundred and ninety-seven miles long!" "its drainage basin is the same size as Belgium!" and so forth) that actually derive from memorizing a list of figures in an almanac. By contrast, clues about sources, outflow, tributaries can be acquired by looking at maps, reading books about the history and culture of the region, etc., and are thus substantially more academic in nature. This has always struck me as identical to the difference between a literature question that begins with clues about the critical reception of a play and a literature question that begins "the second act of this play contains 5,387 words..."

If you write a tossup on the Yenisey River, 90% of your audience is going to have the "possible answer space" narrowed down to "Siberian Rivers" in about two sentences, and then have no further revision to that set.

The problem here isn't that the clues themselves are somehow structurally illegitimate. If 90% of your teams know nothing about a proposed tossup answer except the giveaway clue, banish the Yenisey to a bonus part and write about an easier river.

transparency (OMG it's a major South American River that isn't part of the Amazon Basin)

Yeah, I think you're right, Sanatorium. Tributaries and starting and ending location (if you give cities or mountains or whatever) are very useful. I only wish people liked and wrote about geography as much as they like talking about it here...back to our regularly scheduled program: why people can't write good questions.