[eDebate] Solt on the point scale
Tue Nov 6 17:47:50 CST 2007
Hi Ross, and all,
The point that was made yesterday, that I think may have been missed, is
that it is not necessary to do an uncontrolled experiment, and guess how
things will turn out (as was done in Roger's note). You could do a
controlled experiment: simply have people double-enter their points, both
on the 30 point scale and the 100 point scale. For Wake, use the 30 point
scale, but also publish the data-set to see if you get a single
distribution, or the feared multiple distributions, and to see if people
make finer distinctions within distributions (for example, if you just take
27 and multiply by 3.3333 and give a kid 90, and do the same with a 28.5,
you are not doing anything different on the 100 point scale; the two scales
at that point are identical .)
If you observe multiple distributions (some people translate a 27 as a 90,
others translate it as a 75, others translate it as a 97), that helps to
show that a 100 point scale would for a transition period have multiple
meanings to different people, causing results which are "meaningless" -
skewed into non-overlapping meanings based on the multiple possible
translations different people utilize. One suggested way of trying to
instill a faster transition is to actually give the translation
guidelines... but even that should be studied first via experiment and data
Last point : if you simply switch from one scale to another, without doing
the experimental controls suggested, you will have no way of knowing what
you have done, because you can't compare the new system to something we all
know and think we understand.
It was suggested by Gary that the experiment itself would switch behavior -
but first this is speculation, and second, this might be desirable if it
will make people think more, and take the time to use the suggested
I just don't see how one evaluates the results of the new scale as a
"success" or a "failure" if comparative data is not generated. There would
be no possible criteria for making such judgements.
On 11/6/07, Ross Smith <smithr at wfu.edu> wrote:
> Roger Solt said I could post his message to me if I thought it helpful.
> I do. Here it is:
> The new point scale is a worthy experiment. I like the 100 point better
> than the 50 point option.
> The value of the scale is that it opens up a wider range of evaluative
> options; the danger, of course, is that people will use it so
> inconsistently that a major element of arbitrariness is introduced.
> The real question, I think, concerns what benchmarks one should employ.
> For example, what would be the equivalent on the 100 point scale of a
> pure 27? Or a pure 28.5?
> I have gone through three stages in my thinking about this issue.
> First, I simply converted the 30 point scale into a mathematically
> equivalent 100 point scale.
> Thus, 27=90 (since 27 is 90% of 30); 28.5=95; 30=100.
> Even this scale is something of an improvement over the status quo, but
> it soon became clear from yours and others edebate posts that this was a
> higher scale than was intended.
> My second thought was to make 27=80. This would make 28.5=90, etc. This,
> I assume is closer to what you intended.
> Still, I was struck by one of Josh Hoe's comments about the danger of
> one set of judges using a 90 to 100 scale (my initial response) and some
> using a 70-100 scale (a still wider range than my second scale
> This led me to contemplate using a 70-100 scale, with 27=70; 27.5=75,
> 28=80, 28.5=85, 29=90, 29.5=95, etc.
> It struck me that one not negligible benefit of this scale is its easy
> convertibility from the old scale. Perhaps this familiarity is also a
> drawback, since one benefit of the new system is presumably to encourage
> people to rethink the basis on which they assign points. Still, all told
> its ease of use makes the 70-100 point scale the one I think I would
> like to use. That is?if I think that that's the scale that most others
> will use (since I don't want to be a point Grinch). At this point, it
> seems impossible to tell, so I may well use the 80 to 100 scale instead.
> I think that a lot of people would find it useful if the tournament
> would publish its suggested baselines. Where would your 27 point
> equivalent be? 28.5? Etc. Maybe you don't want to be that definitive,
> but I do fear that absent such guidance points will be all over the
> place and that this will be a one shot experiment.
> Ross K. Smith
> Director of Debate
> Wake Forest University
> 336-251-2076 (c)
> 336-758-5268 (o)
> eDebate mailing list
> eDebate at www.ndtceda.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Mailman