[eDebate] Solt on the point scale

Gary Larson Gary.N.Larson
Tue Nov 6 20:21:46 CST 2007


Now that I?ve been instructed on experimental design 101 and even offered David?s presumably excellent book J, let me suggest that I don?t think I?m the one missing the point.  BTW, I?m open to collecting two sets of data, though the act of doing so wouldn?t constitute an adequate ?control? given the contamination of the data by the task.  But even more importantly, this isn?t the kind of research where a ?control? is possible in the sense that given two different outcomes, we have absolutely no objective means of defining which one is correct or even ?better.?  More on that in a second.
 
For what it?s worth, even given the mutual interactions between the control (30-point scale) and the test condition (50 or 100-point scale) and the fact that the experiment itself will likely change judge behaviors (at least temporarily), I?d support collecting both scores.  A couple of caveats:  we dare not over-interpret the value of the 30-point scores.  We started from the premise that they are the problem rather than the control. 
 
As I noted, the place where David is absolutely correct is in his worry that judges will use significantly different distributions.  At the point that David first raised the question, it was not yet clear that Wake would offer clear guidelines as to how the distribution should be constructed.  Of course, even with a defined scale and defined guidelines for ?translation? what do we do when some judges just DECIDE for whatever reason to do it differently.  Apart from the fact that we already face that problem, putting both scores on the ballot provides a check for those who just don?t get it.  Probably not a bad idea.
 
Where I part company is the suggestion the old scores should continue to be official.  IF the experiment can ensure (within appropriate limits) that the scale translation is the same so that the principal difference is essentially one of increased discrimination, I can?t imagine reasons that we would still prefer the old outcome and, if so, what grounds could be used for making the choice. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ndtceda.com/pipermail/edebate/attachments/20071106/4073b522/attachment.htm 



More information about the Mailman mailing list