I made a quick little Google form for the Preseason Poll. I'd ask respondents to list their top 25 in order from best to 25th. I also included optional questions about who respondents feel is going to win ICT and ACF Nationals. I will close this form September 15 unless I don't get a good amount of responses. I'd like to get responses from players of different skill levels as long as the respondents feel they are knowledgeable about the collegiate circuit.

Please email me if there are any issues with the form. Thanks.

Harris Bunker
Grosse Pointe North High School '15
Michigan State University '19 B.S. in Economics & B.A. in Mathematics
UC San Diego Economics 2019 -

"Whereof one cannot speak, thereof one must be silent."- Ludwig Wittgenstein

Mnemosyne wrote:Here is the postseason poll. It would probably be useful for people to post about who is leaving, joining, etc.

Cheynem wrote:
4. Stanford

Stephen and Benjy are gone. I'm still here; Kyle Sutherlin, Jennie Yang, and Eric Kilgore are still here; Nikhil Desai is back (!!) and, so far as I know, intends to play; Max Rong (formerly of Northwestern) has joined up; various freshmen (Ali and Young from Stevenson, Ashwin from Hooch) are in the mix as well.

vengefulsweatermensch wrote:Max Rong (formerly of Northwestern) has joined up

Max is a big loss for us, though obviously not as big a loss as Dylan is. Also worth noting is that Northwestern had essentially no bench last year: of the two returning players other than Adam and I, one is studying abroad in the fall and the other has indicated that he's done playing quizbowl. We'll be replacing Dylan and Max with new recruits.

Greg Peterson

Northwestern University '18
Lawrence University '11
Maine South HS '07

Thanks a lot to the people who have taken the time to fill out a ballot. I've had the poll up for about a week now and I've gotten less than 10 responses. I'd like at least a dozen or so before I look through the results after September 15. Again, you don't need to be an expert in the collegiate circuit to submit a ballot and I'd like to have opinions from many teams, regions, and players of different ability levels and experience. Thanks!

Harris Bunker
Grosse Pointe North High School '15
Michigan State University '19 B.S. in Economics & B.A. in Mathematics
UC San Diego Economics 2019 -

"Whereof one cannot speak, thereof one must be silent."- Ludwig Wittgenstein

Hello all. I have some results made from the 14 responses I got. I felt the poll results didn't have too many outliers and were relatively in line with what people have said in discussions, previous polls, etc. Therefore, I thought it would be appropriate to post what I have got. I used python to go through a large CSV that I had to spend a lot of time cleaning up. For example, I had to make sure Minnesota A was equivalent to Minnesota when different voters put one name or the other for the team. By my count, 40 teams received at least one vote.

A note on the points: It's a simple formula that says if you get a 1st place vote, you get 26 points, and if you get a 25th place vote, you get 2 points, and so on. This is similar the way the AP College Football poll does it I believe. [Edit: thanks to Stephen Eltinge for pointing out the initial problem here. Damn 0 index.]

(Unlabeled teams are A teams) Please note that the lowest ranking does not include a ranking of not ranked.

I also asked two optional questions about who will win the two collegiate national championships. Yale got 10/14 votes to win ICT, with Penn, Berkeley and a long, multiple team answer also receiving votes.Yale got 9/14 votes to win ACF Nats, with Penn and Berkeley also receiving votes.

Thank you to everyone who voted. If anyone is interested in receiving the raw, anonymous voting data, or the program I wrote to analyze the poll, just send me a PM or an email. I had to "clean up" a lot of the data to make sure that A teams were treated the same as unlabeled teams, and to eliminate some errors such as erroneous spaces in team names, so please let me know if you think anything is amiss here.

Last edited by Progcon on Sat Sep 16, 2017 11:19 am, edited 1 time in total.

Harris Bunker
Grosse Pointe North High School '15
Michigan State University '19 B.S. in Economics & B.A. in Mathematics
UC San Diego Economics 2019 -

"Whereof one cannot speak, thereof one must be silent."- Ludwig Wittgenstein

Progcon wrote:if you get a 1st place vote, you get 26 points, and if you get a 25th place vote, you get 1 point, and so on

Shouldn't the point range go from either 26-2 or 25-1, not 26-1?

Yeah. My program increments 26-i points where i ranges from I think 0 to 24 inclusive, so the highest score is 26 and the lowest you can get is a 2. This means there are 25 possible values, and thus 25 places. I originally incremented by 25-i but I thought 25th place teams wouldn't get any points because 25-25 = 0 but the 0 indexing confused me and caused me to write something in the forum post that was incorrect. You are right and I apologize for the confusion there. If I do this again, I'll have a 25-1 system instead.

vinteuil wrote:Thanks Harris! Is it also possible to include the traditional "lowest" and "highest" info?

I have added this. I wrote a way to keep tracking of lowest and highest ranking in the program, but this doesn't account for people who chose not to rank a team. There were many teams in the 20-25 range (and even higher) that were not ranked by every voter, so many "lowest ranking" would be "not ranked". If people are interested, I can also add a count of how many people ranked a certain team, but I feel the points do a decent job of making tiers of the ranked teams.

Harris Bunker
Grosse Pointe North High School '15
Michigan State University '19 B.S. in Economics & B.A. in Mathematics
UC San Diego Economics 2019 -

"Whereof one cannot speak, thereof one must be silent."- Ludwig Wittgenstein

Using a script is a creative idea, but it seems more suited to a poll with 14,000 voters than a poll with 14 voters. I received the poll results from Harris and used Excel to manually create rankings in a more familiar form, using the standard system of 25 points for #1 down to 1 point for #25. This changes the rankings somewhat!

Going forward, it would be inadvisable to continue using a Google form, since with e-mail we have some idea of whom the voters are. I was planning to do a writeup of Top 25 teams, as has been done in past years, but the wild variance in the results of this poll (of the 14 voters, four had WUSTL in the top ten and four did not rank WUSTL at all) indicates that the level of information possessed by voters is so low that these rankings are not really useful for any purpose.

Last edited by 1.82 on Mon Mar 12, 2018 3:22 pm, edited 2 times in total.

1.82 wrote:Using a script is a creative idea, but it seems more suited to a poll with 14,000 voters than a poll with 14 voters. I received the poll results from Harris and used Excel to manually create rankings in a more familiar form, using the standard system of 25 points for #1 down to 1 point for #25. This changes the rankings somewhat!

Going forward, it would be inadvisable to continue using a Google form, since with e-mail we have some idea of whom the voters are. I was planning to do a writeup of Top 25 teams, as has been done in past years, but the wild variance in the results of this poll (of the 14 voters, four had WUSTL in the top ten and four did not rank WUSTL at all) indicates that the level of information possessed by voters is so low that these rankings are not really useful for any purpose.

First of all: thanks for doing that Naveed. I appreciate the work it took to remake the rankings by hand. I apologize for any silly mistakes I made.

I fail to see how using a Google form is inadvisable. It allows there to be a ton of different responses, and saved me a ton of arithmetic which I liked. The data I sent to you was also highly edited from what I received, as I removed the person's name and their time stamp and cleaned up a lot of inconsistencies in team naming. All individuals who submitted a poll with the Google form used names of real quizbowlers and several confirmed their submission with a PM. Therefore, I fail to see how email is a better system. There is no incentive to impersonate a quizbowler in an anonymous poll, but maybe people would have to PM the pollster and say "yeah I submitted a ballot" to verify. I had no issue with how it was done personally aside from my error on the 0 index which led to the 26-2 point range. For that I apologize. I also wrote the script in Python for future polls that have larger response rates. It could work for any arbitrary ranking of things which is helpful to me.

I am not sure what there is to gleam from these rankings, but I firmly believe they do an accurate job of showing the perception that the community at large has of each team. Even the high school poll, where there are many more knowledgeable and connected people with many more results and more static teams (i.e. players are in high school for 4 years only for the most part), has strange results, and people who are not knowledgeable about the entire circuit. The high school poll does not have a huge response rate either. Given that only about a dozen people felt comfortable submitting anonymous responses to this poll request, I am not sure there could then ever be rankings that were "useful for any purpose". Are there a ton of people who have in-depth knowledge on all 40+ circuit regulars and all the freshmen coming in? Even if there are a ton of people with that kind of knowledge, and those people happened to not have voted in this poll, why then not make a write up of the 25 teams? If you have the time and the interest, I think it'd be great if you could write a few sentence blurb on each team. In your opinion, do you personally think any supremely talented teams did not make the 25 team cut? I'd genuinely like to hear your thought here.

In summary, the poll does a good job of precisely doing what a poll should do, which is to show the perception of teams in the quizbowl circuit as a whole. I think my confidence interval on final placement or team strength grows as we go down the rankings, but I think there is some value with knowing that, say, Yale, Penn, Berkeley, Chicago, Columbia, Ohio State, etc. are some of the favorite this year, even if that fact is somewhat trivial when looking at how good their players are.

Maybe in the future, it would be advisable to only have pollsters rank the top 10 teams because those are the teams people have the most knowledge about. It's kind of unrealistic to expect a large amount of people throughout the American, Canadian, and British circuits, have a good working knowledge of Chicago C, Stanford B, Toronto, and Duke when those teams are in different regions and some teams, such as my own, might play those teams once a year at most. Cursory quizbowl statistics also do a poor job of estimating team strength for teams that are not the strongest for reasons that have been discussed.

Harris Bunker
Grosse Pointe North High School '15
Michigan State University '19 B.S. in Economics & B.A. in Mathematics
UC San Diego Economics 2019 -

"Whereof one cannot speak, thereof one must be silent."- Ludwig Wittgenstein

Progcon wrote:
In summary, the poll does a good job of precisely doing what a poll should do, which is to show the perception of teams in the quizbowl circuit as a whole. I think my confidence interval on final placement or team strength grows as we go down the rankings, but I think there is some value with knowing that, say, Yale, Penn, Berkeley, Chicago, Columbia, Ohio State, etc. are some of the favorite this year, even if that fact is somewhat trivial when looking at how good their players are.

It should be trivial that WUSTL is minimally top 15 if you know how good Charles Hang is! Anybody who left them or the three teams above them out of the top 25 (!!!!) is, as Naveed says, so ill-informed as to add a big wash of noise over the results.

Progcon wrote:
In summary, the poll does a good job of precisely doing what a poll should do, which is to show the perception of teams in the quizbowl circuit as a whole. I think my confidence interval on final placement or team strength grows as we go down the rankings, but I think there is some value with knowing that, say, Yale, Penn, Berkeley, Chicago, Columbia, Ohio State, etc. are some of the favorite this year, even if that fact is somewhat trivial when looking at how good their players are.

It should be trivial that WUSTL is minimally top 15 if you know how good Charles Hang is! Anybody who left them or the three teams above them out of the top 25 (!!!!) is, as Naveed says, so ill-informed as to add a big wash of noise over the results.

Well I agree both that WUSTL is a top 15 team and that those that don't think so dilutes the results, I'm not sure what the direct implication of this is. Do we toss out the results of those people who supposedly don't have the requisite knowledge to do the poll, or do we just naively say that the poll is meaningless or invalid, etc.? More people should have voted if we want a way to correct for people who mistakenly think that WUSTL isn't a top 25 team. Part of the reason I made the poll anonymous was to increase participation, and I had the poll open for 3 weeks. I would of course liked to have gotten more responses, but I wanted, and I think is a reasonable idea, to get the poll out before people starting playing real tournaments.

There are two questions about the collegiate poll that must be discussed. First, is what is the value of the poll, and how seriously do we take it? Is it about projecting future performance, peak team "skill"/knowledge, gradating teams about a potential match up (eg. the Number 17 team would be unambiguously favored over the Number 18 team). Further, do we start putting the team's preseason poll ranking next to the team name ala college football to hype up, say, a top 5 match up between Yale and Penn? If we decide that the poll matters, then we need to get more people to vote--and vote soon after the poll goes up.

That leads into my second question. Do we need a minimum number of respondents--or some other restriction on pollsters before the poll can be published? I made a judgement call that 14 voters was enough, and maybe it wasn't, but I was getting less than one response a day even up the last day so I didn't see a whole lot of desire to either make a poll or discuss the rankings. Hell, the high school ranking threads get load of responses: both here and on Facebook, so I figured that these are the responses I have, and I might as well publish them promptly. I apologize if these rankings lead any team to feel slighted or subject to bias. This was obviously not my intent. I wanted to stick to the deadline. Maybe in the future, we can make a requirement before people vote that the poll won't be published until, say, 30 different people vote who all don't attend schools in the same region.

Another idea, and I think this one is a bit better, is to have a panel made of knowledgeable players out of the game make polls and then look at who they think the best teams are. A lot of the respondents were very biased towards their own team, which is fine and understandable, but I think making it feel like more the College Football Coaches Poll is a good idea. Alternatively, restricting the poll to people who have been in college for at least 4 years or have graduate could be an idea, but the majority of respondents fell into this camp anyways this year. You could also add a skill bar that must be reached of, say, averaging 40 ppg at a regular tournament. I think we can all agree, though, that there should be more than 14 respondents.

Again, I wish I had gotten more data and hopefully all my mistakes illustrate ways the collegiate polling process can be ameliorated or changed.

Harris Bunker
Grosse Pointe North High School '15
Michigan State University '19 B.S. in Economics & B.A. in Mathematics
UC San Diego Economics 2019 -

"Whereof one cannot speak, thereof one must be silent."- Ludwig Wittgenstein

I'm surprised that Stanford and Northwestern both were left out on one ballot; upthread Will Alston posted the team changes so they couldn't even make a mistake like with Toronto, who by the way, is losing Jay for ACF events but will still be able to play NAQT.

I don't think this ranking is useless. The top 10 this poll generated seems more or less correct to me, and the rest of the poll looks plausible enough, no less so than it's been any of the past few years.

Personally, I am getting farther away from the game every day and I honestly have no idea who plays for some of these teams and what they are good at (although I guess I was at least a replacement level voter since I ranked Wash U). I don't see why it's impossible that someone even more removed than I am might find it interesting that Yale, Penn, and Berkeley are favorites this year and teams like OSU and Cambridge are looking to play spoiler. I think even a flawed poll has a lot of value in that respect. Is there actually a negative outcome to having this?

Even though we're apparently the most prominent example of a team that was voted on by uninformed voters, I do think the overall poll result is fine. Our placement was--in my personal opinion--a little low, but certainly not outside the margin of error.

Richard perfectly expressed my views about how the poll still has value, even if individual voters might have made mistakes. As far as I can tell, the wisdom of crowds seems to have delivered a perfectly reasonably result.

1.82 wrote:I was planning to do a writeup of Top 25 teams, as has been done in past years...

I can't speak for anyone else, but I would find this really enjoyable to read and share with my teammates. We have a lot of new players who are very excited about quiz bowl this year, and I think this sort of thing would be a fun morale-boosting thing to present to them, especially since many of them are totally new to the game and have asked about how our team ranks relative to other schools.

Given that the top 25 teams listed in the poll results are still (roughly) the 25 best teams in the game, despite the poll's flaws, I don't see why you would let a few bad ballots keep you from using these teams as the basis of your writeup.

1.82 wrote:I was planning to do a writeup of Top 25 teams, as has been done in past years...

I can't speak for anyone else, but I would find this really enjoyable to read and share with my teammates. We have a lot of new players who are very excited about quiz bowl this year, and I think this sort of thing would be a fun morale-boosting thing to present to them, especially since many of them are totally new to the game and have asked about how our team ranks relative to other schools.

Haha same with McGill honestly, a lot of signer uppers at Clubs Day asked how good we were.

I rescind my criticism earlier somewhat, if people forgot teams by accident then that's unfortunate but it does show the team is somewhat forgettable. Shouldn't apply to any of WUSTL and above though.

I made a spreadsheet compiling all stats from Penn Bowl and ACF Regionals, excepting Connecticut and the UK mirrors. I rushed it and there are probably numerous errors, and I didn't make a very deep attempt to get all the rosters right (and gave up partway through). I set the spreadsheet to comment-only, I'm hoping to crowdsource the rest. If you can fill in stuff for me, I'd appreciate it.