[eDebate] Does mutuality matter?&In-Reply-To=BAY106-F2041920DA6FF6629D12658D3D90 at phx.gbl

Gary Larson Gary.N.Larson
Wed Mar 22 21:21:41 CST 2006


Three quick reactions:

On one level, Neil is absolutely right.  Mutuality (and for that matter
preference) don't matter as much to the outcome of the debate as we
might think.  Of all of the things that might presumably affect the
outcome of the debate - the preference or mutuality of the judge, sides,
pairing strategy, etc., the good news is that by far the best predictor
of who is going to win the debate is the team that debates better.  Now
that is not an excuse to not adopt the best pairing strategies possible,
to maximize both mutuality and preference and to write balanced topics.
But like Neil, I've similarly noted in dozens of tournaments that there
is not a statistically significant correlation between which team
prefers the judge more and which team wins the debate, particularly when
the mutuality difference is small.  Participant satisfaction, on the
other hand, raises most with higher preference so Neil is probably right
that a 23 is better than a 33.

One of the most important observations from Neil's data is the overall
small mutuality difference that he is discussing.  With categories that
have only 11 members (out of roughly 100 judges), an off-1 match has a
maximum mutuality gap of 21 (out of 100).  While Neil might discover
that small gaps may not have a significant impact on outcome (in fact,
they might be within rater error), the same might not be the case if he
was evaluating a large number of larger mutuality gaps (say 30, 40 or
even 50).  One of the simulations that Fritch and his staff will be
conducting with the NDT data is the impact on mutuality and preference
using larger or smaller categories (and potentially larger and smaller
numbers of categories).  This work will potentially answer a large
number of questions about mutual preference strategies.

The final observation is an artifact of thinking in terms of categories
rather than ordinal rankings.  Within Neil's data, an off-1 match (12 or
21) might actually be MORE mutual than a 11 match.  An off-1 match might
have an ordinal gap as large as 21 but also potentially as small as 1
(!) while a 11 match could have an ordinal gap as great as 10.  While
teams may not prefer the work of completing ordinal rankings, ultimately
it provides a better true measure of preference and a much better
measure of mutuality.  Now, given the challenge of rating judges, we
make necessary trade-offs with precision.  But it means that we need to
be more understanding about what mutuality might really mean.





More information about the Mailman mailing list