[eDebate] Speaker Points - Why we are using .25 at USC

Gordon Stables stables
Tue Nov 6 13:45:54 CST 2007


As we all get ready to utilize the Wake experiment with a new point scale I
wanted to provide some background about how the USC 'Alan Nichols'
tournament will be recording speaker points.

Our decision to pay closer attention to this situation has developed out of
a concern that speaker points in practice have largely lost their ability to
differentiate between speakers. Inflation is a related problem, but if
inflation were uniformly influencing both the bottom and top of the scale it
would be less of a concern. Our view was that given the incredible
significance of speaker points in so many elements of a tournament (prelim
pairing, elim seeding, and teams clearing) it was important to begin a
conversation about how we could restore greater differentiation among
speaker points.

The actual items on the scale (5, 30, 50, 100) are far less important that
the values associated with that scale. It is somewhat intuitive that if we
only really utilize 3-4 items on a scale it would simple to just use
1,2,3,4. The problem, however, is that this requires the community to adjust
to this system and then also provide points in a consistent fashion on this
new scale. Of all of the problems with the current system it is hard to
argue that there is inconsistency of points. If anything, we are worried,
the system has worked 'too well' in terms of normalizing community
expectations.

In our invite we provided both a scale that offers a minor repair to the
current 30 point system (using quarter points) and asks judges to strongly
consider providing differentiation among the speakers in the debate. The
full scale and instructions are reposted below. The scale itself doesn't
translate well in email formatting, but you can view it at
http://www.usctrojandebate.com/college_tournaments.html

Gary has offered the correct concern that this system will require
additional data entry time. We are fortunate that Adam Symonds of ASU is
running our tab room and has agreed to shoulder this additional labor. We
would be pleased, however, in future years to adopt a more simple to enter
scale (say 1-5 or 1-10) after the community has adjusted their judging
habits to use more elements of that scale. I am very concerned that we may
be trying to adjust move community norms and the instruments that we use to
capture those norms at the same time. Quarter points is not a long-term
solution, but perhaps it is an initially cumbersome, but effective, means of
getting us as critics to return to a more differentiated scale.

We have been, for the last two years, strongly considering the shift to a
system that evaluates the points a judge gives in a given debate against
those points that the same judge has awarded all season. We are not using
the system this year for a number of reasons, but the decision to not have
all tournaments utilize a 30 point scale is a significant factor. I know
there are concerns about moving to this system, but I would suggest that as
we should be concerned that the intersection of MPJ and non-differentiated
speaker points does create a competitive incentive for teams and a prestige
incentive for judges to award points in excess of the norm. Limiting the
ultimate autonomy that any single set of points can provide is one of the
few options available to address this intersection. I would welcome
additional discussion on this item.

Our instructions ----

Speaker Points - We are very concerned that the current speaker point scale
and system doesn't help differentiate the quality of individual speakers.
Specifically we are worried that the norm has evolved to giving most
debaters the same points with only a few judges willing to vary from that
norm. Our research, for example, shows that at one of the larger recent
tournament (hosted by the University of Kentucky) almost half of every
speaker in every debate earned the same speaker points, a 28. The community,
it appears, now considers this to be the default setting for speaker points.

With 46% of the speakers earning exactly a 28 and then an additional 24%
earning a 27.5 it also appears obvious that the actual scale in practice is
too small. When 70% of the debaters earn one of two items on the scale it is
obvious why tiebreakers, speaker awards, teams clearing and seeding appear
so arbitrary.

To that end we are going to provide additional options to judges and in
exchange we ask you to consider your points as part of the larger scale. Our
charge is that we would prefer that judges utilize speaker points to
differentiate between the performances of individual debaters in each round.
We ask you to use the following scale

26	Needs improvement
26.25
26.5
26.75
27	Below Average
27.25
27.5
27.75
28	Average
28.25
28.5
28.75
29	Very Good
29.25
29.5
29.75
30	Perfect

The determination of how these points should be awarded still obviously
resides with individual judges.

We do ask you to consider two additional guidelines:

    * Speakers should be differentiated in each debate. It is certainly
conceivable that debaters would earn similar points, but we would ask judges
from awarding more two speakers with the same points in a given debate. The
quarter points are a means you reflecting differences in performances.
    * We ask that judges not regularly award the bulk of their points at or
below the 'below average' (27) standard or at or above the 'very good' (29)
standard. In order for speaker points to have some sense of meaning
individual judges must feel comfortable providing evaluations across the
range of items.

These are guidelines that we are requesting your assistance in implanting.
We feel it is in everyone's competitive and educational interests to have a
meaningful speaker point scale.

Gordon

Gordon Stables, Ph.D.
Director of Debate and Forensics
Annenberg School for Communication
University of Southern California
Office: 213 740 2759               Fax: 213 740 3913
http://usctrojandebate.com





More information about the Mailman mailing list