[eDebate] Response to Kloster
Wed Nov 22 21:35:42 CST 2006
Thank you for your thoughts on this topic.
Allowing teams to specify preference when considering both side of the
resolution and opponent should not fundamentally change the problems
encountered when trying to place judges. It will, however, remove
another illusion of mutuality created because people now have to rate a
judge without knowing the context. If the result of adding this
information to the system is that it is harder (or impossible) to find a
1-1 judge for a given debate, it just reveals a problem that the two
opponents don't have a mutual 1-1 available for that debate. The
additional information does not cause that problem.
As to how allowing the consideration of side and opponent may help
increase mutuality, I have one reason in mind. Sometimes people rate
judges low to avoid having them in certain situations. An otherwise
great judge may be perceived to have a bias that favors a given
opponent. When ranking judges, some teams may lower the preference score
of a judge simply to avoid having them against that opponent. Allowing
consideration of side and opponent in the pref sheet would allow a team
to effectively strike certain judges only when debating against certain
teams (or sides of the resolution) while still rating them highly for
all other situations. This would increase the pool of highly preferred
judges for that team in most pairings.
Thanks again for your thoughts and for all the work you do for the
Gary Larson wrote:
> In the next few days I will weigh in on the mutual preference discussion
> by discussing the experiment run at Kentucky and Wake, but while it is
> fresh I thought I should comment on Kloster's suggestion for increasing
> On one hand I agree with his observation that actual preferences might
> be much more complicated than can be fully captured by a single
> context-free rating or ranking that a team is permitted to enter for a
> judge. It is probably true that a fully "accurate" judge rating would
> include data about side and opponent (and to be honest also could be
> different in presets, break rounds, elims, etc). But even if I just
> encoded the variables Kloster identifies, at a tournament such as Wake a
> team could assign 43,884 ratings to complete the ratings task (138
> opponents, 159 judges, Aff and Neg).
> But my principal concern is not that we couldn't manage the data or
> that teams wouldn't be able to intelligently provide them. My real
> concern is whether such a system could actually provide more mutuality
> once implemented. If the only concern was that a single debate between
> team X and team Y needs to find the most mutual judge among 159
> possibilities, Kloster's suggestion would be a very good one. I might
> discover that most of the mutual 1's aren't really mutual when I
> consider side and opponent. Maybe none of them are. But presumably I
> could find at least one BEST match among the 159 options.
> But the task for judge assignment is not a local maximization problem
> but rather a global maximization problem. A successful assignment
> algorithm at Wake needs to simultaneously assign judges to 69 different
> debates where the "ideal" judge for one debate might also be the ideal
> or perhaps the only acceptable judge for a number of other debates as
> well. At the same time, among the 159 judges, a significant number of
> judges might be obligated to hear the round (meaning that they might
> have to trump some more ideal judge placement). Additionally, among the
> partial commitment judges, some lower preference judges need to hear
> each early round to preserve committed rounds of higher preference
> judges for later in the tournament. To complicate matters further,
> among the rounds within the pairing some rounds might be break rounds
> and hence considered more important for the maximization of pref.
> Finally, the goal is to equalize the opportunity for high preference for
> teams with the same record across all prelim rounds. In other words,
> those teams that have been least fortunate in previous rounds should
> have the maximum opportunity to receive their prefs in later rounds.
> With the interaction of all of the global constraints, it is perhaps
> unrealistic to expect that Kloster's increased mutuality could ever be
> successfully balanced against all competing interests.
> A much more critical concern impacts the proposal however. While
> Kloster suggests a number of factors that might make a putative mutual
> assignment less than mutual, he doesn't suggest any that would make them
> MORE mutual. Given all of our competitive instincts, presumably equally
> well informed teams would discover that they really had NO truly mutual
> judges (unless by mutual I meant that the outcome would be totally
> unpredictable). Say that I decide that such and such a judge is a
> negative hack. I rate them highly on my neg sheet and low on my aff
> sheet. But of course, my equally informed opponent rates them high as
> neg and low as aff. If the whole tournament has the same knowledge, the
> judge is never a match and we're worse off than we are now. At the end
> of the day, every time my opponent knows what I know about a judge, the
> more likely that I am to believe that the info matters makes it less
> likely that the judge matches. So the only judges left are those that I
> have no good intel on OR the really rare case where I just conclude that
> the judge is absolutely "neutral" in the upcoming debate. But since we
> all hate uncertainty, this genuinely "best" alternative might be
> rejected by both of us.
> Currently, judge preference selections are made without consideration
> for the side of the resolution the participant will be debating, nor
> participant's opponent. This leads to a situation where for a specific
> debate if you look at the preference sheets a judge placement may be
> exactly mutual (say a 1-1 placement) but in fact, one team believes
> the assigned judge gives an advantage to their opponent.
> The solution is to allow judge preference selections to be
> differentiated based on which side of the resolution a participant
> be on and who their opponent will be.
> 1) I may rank a judge 5 when I'm aff, but 1 when I'm neg.
> 2) I might rank a judge 1 in general but 8 if I'm debating against team
> 3) I might rank a judge 1 if I'm debating team Y on the neg, but 8 if
> I'm debating against team Y on the aff.
> By allowing consideration of side and opponent the judge placement
> system can do a much better job of placing actually mutually preferred
> judges in particular debates.
> eDebate mailing list
> eDebate at www.ndtceda.com
More information about the Mailman