I've read over a lot of RFD's in the past few days as I evaluate potential judges for Bobbert's upcoming gauntlet tournament. I've looked over RFD's for both judges who volunteered, and RFD's for those who didn't but whom I'd consider asking to volunteer and RFD's from random people on higher profile debates for points of comparison and a few general things stick out to me. I'm going to talk about them now, but I'm not going to name specific examples because the purpose of this is not to target anyone specifically but to identify and comment on generalities. I'm sure some of you would like examples, but this thread's purpose of beginning a discussion on how people vote just isn't furthered by including them.

The first thing I notice is a tension between voters seeking to confirm what they already believe at odds with an extant desire to seem unbiased. How this works is pretty predictable: the arguments which judges at least cite as the most persuasive are ones consistent with their understandings/views of specific topics, but in RFD's there is usually a rationalization of why that argument was more compelling which may or may not capture the precise nature of what was said in a debate. This isn't a bad thing to the extent that judges have well grounded backgrounds in the subjects on which they debate, and possibly even a good thing when judges penalize debaters for factual errors, but is also a bad thing when judges' normative beliefs delineate wins and losses.

The second thing I've noticed is emphasis on dropped arguments, and this sort of ties into the first trend. I think as a rule it might be said that only if in a debate one debater drops or mischaracterizes an argument that a judge finds compelling for reasons which may or may not be due to the particular way the debate was conducted, will the debater who dropped the argument be penalized by judges. Conversely, if a competitor is not penalized by a judge for a dropped or mischaracterized argument, it can be inferred that the judge did not find that argument compelling for reasons which may or may not relate to what was precisely said in the round.

The third thing I've noticed is that there is often at least a semantic divergence between what was precisely said by competitors in rounds and what judges understand, but even more frequently the difference between what is said by competitors in rounds and what judges understood is stark. I suspect three possibilities in why that is the case: (1) judges do not carefully read debates and therefore fail to understand them, (2) competitors rhetorical imprecision misleads judges or (3) some combination of the prior two may all contribute to shoddy RFD's.

There are other trends, but they tend more to deal with bias which stems from personal animosity or outright intellectual deficiency, or both, than mere normative bias or imprecision. While those voters who fit into that trend tend to consistently cast both the worst and the most obviously bad ballots, they are a definite minority and discussing them further doesn't seem purposeful.

I think it may be said that to the extent that judges offer either caviler or extraneous objections to competitor's arguments, especially when those objections are based on imprecise understandings of competitors arguments or other aspects of the debate, that judge shows undue bias. But, only when that bias is levied on one competitor to a degree proportionally greater than another, does that bias necessarily indicate that a judge is "bad." Naturally, there are numerous reasons why a judge might show bias to one competitor over another and those reasons may stem from personal feelings about one of the debaters, normative beliefs about the resolution or for other reasons. Presumably, it is also the case that even when a judge is not consciously showing deference to one debater or another, that judge might not be consciously aware of it.

The reason that people usually credit dropped arguments 100% to the other side is to avoid bias. If the reason we find the argument implausible is a better argument than what the opponent *would have* said had he responded, we are crediting the opponent with a better response than he would have made, absent the drop.

Granted, *most* tabula rasa judges admit that they won't accept *all* propositions as true if dropped. An assertion that the sky is orange most of the time, not blue, does not have to be accepted as it is patently absurd.

There are three possible standards to require of an argument before you will accept it (even if it is dropped). These are standards legal judges use as well. (1) Possible, (2) Plausible, (3) Probable.

Personally, I will accept arguments that are dropped if I consider them to be "possible." YYW, you seem to only accept arguments if you consider them to be "plausible." I have seen RFD's that have even gone so far as to require a "probability" of truth, even absent a response. I think this goes to far though since "probable" gets awfully close to the preponderance standard of proof, which requires that a proposition be shown to be "more likely than not. A "probable" standard gets awfully close to the judge refuting the argument himself.

Personally, I'm okay with a judge adopting either a "possible" or "plausible" requirement before accepting an argument, although a judge who accepts a "plausible" standard should be sure to only require that the argument is logically plausible, not that it is plausible in light of all of the judge's outside knowledge of the subject matter. Debaters can't refute what is in the judge's head (because they don't know what it is), so it seems unfair to hold this knowledge against the debater. And we -- as judges -- can be wrong, so it's not even valid for us to assume that our knowledge is probative of whether a statement is plausible true, in some objective sense.

Confirmation bias is a huge problem, but hopefully less of a problem the better job a debater does. If a debater provides lots of warrants and explanations for his arguments, the judge does not have to rely much on outside knowledge to evaluate the debate. But if the debater under-explains things, this requires a lot of outside knowledge. For example, if a debater says, "quantum fluctuations disprove the KCA because they show something can come from nothing," and that's *all* the debater says, he is begging the judge to insert their own views into their decision. A judge who understands a lot about quantum fluctations might vote for the debater in question. A judge who doesn't know a lot about physics and who believes in God is going to vote the other way. In contrast, if the debater has explained what a quantum fluctuation is, this insertion of outside knowledge wouldn't be necessary.

Of course, we have to assume some level of knowledge in our judges. We can't take time to explain each debate that "the United States" is a country in North America, whose history includes such instances as X, Y, and Z. The US Congress works in the following ways when we pass bills: ..... etc. We have to assume some basic education level in our judges in order to make higher order arguments. The problem is often when the debaters assume *too high* a level of understanding in their judges and under-explain arguments. That's when confirmation bias will be at its worst.

Of course, there will always be vote bombers on the site who don't understand that they are voting for the better debater and confuse the debate section with the poll section. People without any background in competitive debate sometimes have a hard time grasping this. If you look at old debates on this site, even well-respected debaters like Roy and Danielle voted 7 points to themselves. It's somewhat about evolving community standards. But it's also partially that a 12 year old (more of whom are now allowed on the site every day) doesn't have the educational background that most of us would assume. These are people who haven't even really learned much about World War II in their history classes. So when they judge our debates, of course they look really biased. Basically the entire debate went over their heads. There are also plenty of 18 year olds on the site who were under-educated in school, and also lack requisite background to understand complex arguments. I'm not really sure how you deal with this, except possibly requiring a "test" of basic science and history before getting voting privileges and/or having a high ELO minimum threshold. But the ELO system can easily be gamed, as we've seen with the 3 debate requirement. Certain people did 1K character debates against their relatives to gain voting privileges. You could do the same with ELO, even if the threshold were set relatively high (like at 3K).

The third problem is just a failure to communicate effectively. It happens all the time. It is usually a mix of faults -- of the speaker and listener. Although in some (rather rare cases), one party bears zero fault. You can have a really awesome debater and just a really dumb judge, or a really amazing judge and a debater that is just really really unclear. On this site, it is definitely usually both the debater's fault and the judges. Although there are certain judges -- I would say -- that if you accuse them of "not getting your argument," it was definitely your fault as a debater and not theirs. It's hard to say the opposite -- that there are certain debaters so good that it is always the judge's fault -- because we all sometimes fail in our explanations of complex issues, no matter how good we are.

I don't really have a tldr; for this, but there ya go.

You can't reason someone out of a position they didn't reason themselves into - Jonathan Swift (paraphrase)

At 4/22/2014 2:36:24 PM, bluesteel wrote:All of these same issues come up with legal judges and juries.

The reason that people usually credit dropped arguments 100% to the other side is to avoid bias. If the reason we find the argument implausible is a better argument than what the opponent *would have* said had he responded, we are crediting the opponent with a better response than he would have made, absent the drop.

And in some (at least 1, less than 100% of the time) cases, that makes sense -but if an argument is implausible, makes no sense, is incoherently articulated, etc. then "dropping" it isn't deleterious to the other side. Ignoring irrelevant stuff and focusing on salient arguments isn't a bad thing where character space is limited. But, that's not a big deal because all debaters should, even if an argument is stupid, explain why it's stupid.

There are three possible standards to require of an argument before you will accept it (even if it is dropped). These are standards legal judges use as well. (1) Possible, (2) Plausible, (3) Probable.

Yup.

A "probable" standard gets awfully close to the judge refuting the argument himself.

Maybe. I'm reluctant to discuss that hypothetically, though.

Personally, I'm okay with a judge adopting either a "possible" or "plausible" requirement before accepting an argument, although a judge who accepts a "plausible" standard should be sure to only require that the argument is logically plausible, not that it is plausible in light of all of the judge's outside knowledge of the subject matter.

Sure, and I didn't suggest that knowing all the stuff I know is the metric against which I weigh rounds. If it was, to the extent that I applied that standard to both debaters, it wouldn't be unfair -but it also wouldn't be in the spirit of the exercise of debate, either. So, when I said what I said, I'm talking about general stuff and simple questions of fact -on both of which I'm going to reasonably expect accuracy, as a judge. I also don't expect highly technical or specialized knowledge that may only be knowable through the most recent publication of the American Political Science Review or the Journal of Politics. But, I do expect that if sources are cited, that evidence derived from them is accurately represented.

Debaters can't refute what is in the judge's head (because they don't know what it is), so it seems unfair to hold this knowledge against the debater.

Yeah, of course.

And we -- as judges -- can be wrong, so it's not even valid for us to assume that our knowledge is probative of whether a statement is plausible true, in some objective sense.

Depends on the "knowledge." I think that you might be thinking I'm talking about opinions and beliefs rather than only factual stuff, and that might be because I didn't explicitly say that I was only talking about basic facts and figures. So, only where one debater makes a statement that is demonstrably false is it ok for a judge to penalize a debater for that mistake to the extent that impacts their overall case relative to their opponent.

Though, if we're talking about "opinions" then, a judge penalizing a debater for having a different one than they do (which happens all the time) is a really bad situation. So, we've got to distinguish between a judge who is punishing a debater for factual errors and a judge who punishes a debater for differing normative beliefs. In the former case, it's not only acceptable -it beneficially creates an incentive for debaters to be honest with their evidence. But in the latter case, it is profoundly damaging not only to the exercise of debate, but the integrity of the practice of casting a vote at all.

There are also plenty of 18 year olds on the site who were under-educated in school, and also lack requisite background to understand complex arguments.

Yes, there are. There are also plenty of dimwitted adults as well, but I digress...

I'm not really sure how you deal with this, except possibly requiring a "test" of basic science and history before getting voting privileges and/or having a high ELO minimum threshold. But the ELO system can easily be gamed, as we've seen with the 3 debate requirement. Certain people did 1K character debates against their relatives to gain voting privileges. You could do the same with ELO, even if the threshold were set relatively high (like at 3K).

I'd like a 3k ELO score threshold. Here's why: it's not impossible to get a 3k ELO score, but it's not exactly easy either. It requires real effort, and to the extent that the effort is advanced, voting becomes not something that everyone can have a say in but only something which committed members who demonstrate noteworthy skill can reasonably attain. What's lost is the egalitarianism of the debating process, and I'm sure that right now there are some voters who are both good and have a lower ELO score than 3k, but they could rise to the occasion if they wanted to vote or they could not. I can't imagine that their capacity to cast good ballots would be harmed by pursuing a 3k ELO score.

The third problem is just a failure to communicate effectively.

That's really common, in every aspect of debate. It happens here. It happens at NFL tournaments. Sh!t, it happens even at nationals when people are debating the summer month resolution for the first time. And I agree with you completely that fault for that lies with the debater. But, if I started thinking about what I'm going to have for dinner mid-sentence in a debate and miss the point of what's being said without rereading it, whether the rhetoric was clear or not, that's on me as a judge. There are really stupid judges out there, too. Everywhere. Here. At the National Forensic League. Etc. But, as you said, that doesn't absolve a debater for failing to persuade. Likewise, it's also a problem here. Though as of right now, the only answer to bad votes is more votes.

At 4/22/2014 3:32:52 PM, Jifpop09 wrote:Voters are very choosy in what they include in their RFD's. He made a good point here, so he wins and I'll ignore the rebuttal kind of situations.

A perfect example of failure to communicate. I assume jiffy means that some voters put almost nothing in their RFD's -- they just reference one argument made in the debate that they claim to have liked, and ignore the entire rest of the debate in their RFD's, including rebuttals.

But I'm only 80% sure that's what jifpop meant, since it was really unclear. I'm reminded of Samuel L. Jackson's famous quote from Pulp Fiction [https://www.youtube.com...].

You can't reason someone out of a position they didn't reason themselves into - Jonathan Swift (paraphrase)

At 4/22/2014 3:32:52 PM, Jifpop09 wrote:Voters are very choosy in what they include in their RFD's. He made a good point here, so he wins and I'll ignore the rebuttal kind of situations.

A perfect example of failure to communicate. I assume jiffy means that some voters put almost nothing in their RFD's -- they just reference one argument made in the debate that they claim to have liked, and ignore the entire rest of the debate in their RFD's, including rebuttals.

But I'm only 80% sure that's what jifpop meant, since it was really unclear. I'm reminded of Samuel L. Jackson's famous quote from Pulp Fiction [https://www.youtube.com...].

You can simply ask for clarification rather then a personal attack. But yes, that's what I meant.

At 4/22/2014 3:32:52 PM, Jifpop09 wrote:Voters are very choosy in what they include in their RFD's. He made a good point here, so he wins and I'll ignore the rebuttal kind of situations.

A perfect example of failure to communicate. I assume jiffy means that some voters put almost nothing in their RFD's -- they just reference one argument made in the debate that they claim to have liked, and ignore the entire rest of the debate in their RFD's, including rebuttals.

But I'm only 80% sure that's what jifpop meant, since it was really unclear. I'm reminded of Samuel L. Jackson's famous quote from Pulp Fiction [https://www.youtube.com...].