[eDebate] Side bias - Data from CEDA Nats
Wed Mar 28 12:56:33 CDT 2007
Like Will I observed that the last seven debates at CEDA Nats were won by the Negative. At various times during the tournament, I shared my "impression" with several folks that side bias had become a pro-negative phenomenon.
But then, being a statistics geek, I looked at the actual data. I was actually doing preliminary work on reporting the preference results at CEDA Nats and whether or not it appeared to impact outcomes. Any discussion of outcomes identifies at least four variables, only one of which we intend to have an impact on the outcome of a debate. We would hope that the debate is won by the team that debates better and by extension that the debate is won by the team that deserves to be considered "better" based on the outcome of the tournament. But three things might conspire to frustrate that judgment. I am obviously concerned about judge assignments and the question of whether mutuality and/or preference impact the outcome. In other words, is it the case that the team that prefers the judge more is more likely to win the debate? But before I can make that call, I have to control for other variables. Will's post reflects a long-standing concern that side assignments might impact the outcome of a debate (in perhaps unfair ways). Others worry about the strategies for pairing rounds, particularly with side contraints requiring pull-ups. More on that some other day.
Now to Will's immediate concern. Is there a side bias (either because of topic construction or reigning theoretical paradigms) that favors the negative, as evidenced by the last seven rounds of CEDA Nats. Of course, the immediate observation that needs to be made is that based on seeding, five of those seven debates were won by the favorites. Based on what will likely be the pre-seeding at the NDT, even fewer of the rounds might be considered "upsets." So we need to be careful about identifying side assignments as the reason the victors prevailed.
But what about the entire tournament?
In prelims the AFF won 358 debates while the NEG won only 332 debates. But that might just be a lucky artifact of who got assigned AFF and NEG in each round. Once again, it would be better to control for "quality" by asking how many of those decisions were upsets, probably a superior measure of side bias.
There are two ways to calculate who's favored in the debate. In the first, you can identify the final seed of each of the team in the debate and come up with a measure of how much of a favorite each team is based on 8 rounds of performance. Based on that measure, there were 114 "upsets" out of the 690 debates. Of the 114 upsets, 65 were won by the AFF and only 49 by the NEG. And whether the upsets based on final seeding were mild or significant, the AFF advantage remains constant.
But with that measure, the final seeding is impacted by the result of the debate itself, thus underestimating the number of debates that might be called upsets. A better measure is to subtract out the win and the points from the current debate and to identify a favorite based on the outcome of their seven other debates. Based on this methodology, there were 245 debates that were won by the team that was not favored. Of those 245, 141 were won by the AFF and 104 were won by the NEG.
So far no real evidence at all of a pervasive NEG side bias.
But what about ELIMS?
The data there is a little different but still impacted by the fact that there were VERY few upsets. Of the 64 elim rounds actually debated, 54 were won by the higher seed. Of all 64 debates, 24 were won by the AFF and 30 were won by the NEG. Now it might, in fact, be relevant that of the 10 upsets, 7 were won by NEGS and 3 by AFFS (though we're dealing with a rather small n here).
Now, assuming that I take this latter fact as proof of a side bias that might differentially impact the elite teams as opposed to the rest of the field, we need to be very careful about how we translate that fact into topic-writing. While there is an undeniable temptation to write topics so that no side bias appears to occur in the last seven rounds of a tournament, the topic must also effectively serve the 100's of teams that will be debating it during the year. Getting that right might be even trickier.
PS I will wait to report on pref results until after the NDT. I applaud the community for their willingness to supply data at both CEDA and NDT that will permit a number of careful analyses on the preference systems that we use at tournaments.
More information about the Mailman