[eDebate] Solt on the point scale
Tue Nov 6 20:31:47 CST 2007
okay, so my turn to make some concessions... I don't feel too
strongly about which scale is used officially, as long as we know what
happened as a result of the change
(pause for group hug)
so... haven't heard if Ross is actually gonna do the control and offer
has that been said?
On 11/6/07, Gary Larson <Gary.N.Larson at wheaton.edu> wrote:
> Now that I've been instructed on experimental design 101 and even offered
> David's presumably excellent book J, let me suggest that I don't think I'm
> the one missing the point. BTW, I'm open to collecting two sets of data,
> though the act of doing so wouldn't constitute an adequate "control" given
> the contamination of the data by the task. But even more importantly, this
> isn't the kind of research where a "control" is possible in the sense that
> given two different outcomes, we have absolutely no objective means of
> defining which one is correct or even "better." More on that in a second.
> For what it's worth, even given the mutual interactions between the control
> (30-point scale) and the test condition (50 or 100-point scale) and the fact
> that the experiment itself will likely change judge behaviors (at least
> temporarily), I'd support collecting both scores. A couple of caveats: we
> dare not over-interpret the value of the 30-point scores. We started from
> the premise that they are the problem rather than the control.
> As I noted, the place where David is absolutely correct is in his worry that
> judges will use significantly different distributions. At the point that
> David first raised the question, it was not yet clear that Wake would offer
> clear guidelines as to how the distribution should be constructed. Of
> course, even with a defined scale and defined guidelines for "translation"
> what do we do when some judges just DECIDE for whatever reason to do it
> differently. Apart from the fact that we already face that problem, putting
> both scores on the ballot provides a check for those who just don't get it.
> Probably not a bad idea.
> Where I part company is the suggestion the old scores should continue to be
> official. IF the experiment can ensure (within appropriate limits) that the
> scale translation is the same so that the principal difference is
> essentially one of increased discrimination, I can't imagine reasons that we
> would still prefer the old outcome and, if so, what grounds could be used
> for making the choice.
> eDebate mailing list
> eDebate at www.ndtceda.com
More information about the Mailman