[eDebate] Wake preparations
Mon Nov 6 16:48:18 CST 2006
Since I will once again be helping Ross with the pre-tournament
preparations for Wake, I would like to ask for everyone's help.
It would be greatly appreciated if the entries (with appropriate judge
commitments) could be completed on Bruschke's site by tomorrow evening.
I would like to post the pref sheet no later than early Wednesday
morning and have all prefs collected by NOON on Friday.
With Ross's approval we will continue the research on an alternative
system for collecting prefs and managing mutuality. Once again we will
ask all teams to rate each judge on a scale of 0-100 with no explicit
quotas regarding the distribution that you create.
A couple of strong pleas:
1) It was very interesting to talk to folks at Kentucky who tried a
variety of "strategies" or "games" to optimize their results. While
some might have felt that they succeeded (depending on their definition
of success), it appears that most unusual distributions DID NOT in fact
2) As both a research enterprise and as a test of potential
improvements in the judge assignment process, I need to know everyone's
ACTUAL assessment of the judges rather than a set of values that are
designed to improve the odds without revealing actual preferences. It
is impossible to determine whether a system is assigning judges "fairly"
or even if it is improving aggregate preference and/or mutuality if the
participants aren't revealing their true assessments on the sheet. We
only end up with anecdotal reports where some say, "I liked my judges
better" or "I didn't like them as well."
3) While the Kentucky experiment worked very well from an empirical
perspective (and showed evidence of an opportunity to improve the
algorithm's performance even better with practice), I was struck that
the real definition of fairness that most of us apply to such a task is
"It did or did not advantage/disadvantage me." Obviously my goal would
be to improve everyone's satisfaction, but if satisfaction is measured
by whether I feel advantaged against my opponent, the algorithm's
success is ephemeral.
More information about the Mailman