Speaker point scale changes?

[A Numbers Game]

The Georgia State tournament is moving to a 100-point scale this year. Are any other tournament directors planning on switching point scales? If you considered doing so, but decided not to, what stopped you? If you are switching, why did you choose the scale you chose?

This change will make Georgia State the third large college tournament to use a non-traditional scale: Wake Forest uses a 100 point scale, and USC has used a 30-point scale with quarter-points allowed and a 30-point scale with tenth-points allowed. I have put up some charts showing the effect of these changes on point distributions. I was surprised to see that judges are a bit reluctant to use the new range made available by the changes.

[RedHead]

I've never debated or judged at a tournament with any scale other than the 30 pt normal scale.
That will be interesting to see how the 100 pt scale plays out at gsu this year. I can't figure out what the average points would look like on a scale that big- im guessing it would be good because speaker points would seem less arbitrary since people would use a wider than general 25-30s. I think the quarter or .10 scale with the 30 sounds interesting- allows for tight speaker points but less chance of ties.

[A Numbers Game]

Originally Posted by RedHead

I can't figure out what the average points would look like on a scale that big- im guessing it would be good because speaker points would seem less arbitrary since people would use a wider than general 25-30s. I think the quarter or .10 scale with the 30 sounds interesting- allows for tight speaker points but less chance of ties.

If you inspect the charts at the link above, you'll see that the most common point assignment at Wake are the multiples of 5, peaking at 90. My guess is that the guide given to judges listed these scores as examples; for instance, they might have said, "90 is average, 95 is superior, 85 is missed-on-points . . ."

This guess is based on the popularity of the example scale given by USC at their most recent tournament. They used example scores ending in .0, .3, .5, and .8, and many judges did not deviate from these, even though any tenths place was allowed.

It will be interesting to see what happens at GSU this year. On e-debate, Jacob Thompson revealed that UNLV will be using a 100-point scale this year for both the round robin and the invitational tournament.

The link I gave in the last post also shows how dramatically the number of ties decreased when non-traditional point scales were used.

[Gargamel]

I gave most of the charts a quick look when you posted this to eDebate and this information is pretty interesting. I like the non-standard 30 point scales the best because on the 100 point scale I just figure out the equivilent anyway. Obviously if the norm was 100 points and the shift was to 30 I would probably prefer the older scale because that's just what I'm used to. I haven't heard (though I haven't sought any out) any compelling arguments for a 100 point scale over the 30 point scale.

[A Numbers Game]

Originally Posted by Scrooge McDuck

I haven't heard (though I haven't sought any out) any compelling arguments for a 100 point scale over the 30 point scale.

One argument might be that many judges simply don't use the finer granularity when the 30-point maximum is maintained. That is, they still only assign scores like 27.5 and 28, and rarely or never assign scores like 27.7 and 28.2.

There was a big difference between the two instances of this problem at the two USC tournaments that used finer granularities. When dropping granularity to a quarter point, all of the clustering around the values ending in .0 or .5 was due to judges who didn't use the new scale at all; judges who occasionally used the new scale seemed pretty comfortable with it. When USC moved to the tenth-of-a-point scale, even judges who occasionally used the newly available values preferred to use the traditional half-point granularity.

It seems that dividing up the scale too much just scares judges. :-)

In the quarter-point case (which was during the '07-'08 season), perhaps the clustering could be solved by education. It's not clear that all the judges knew or remembered that quarter-points were permitted. In the tenth-point case, what could be done to convince judges that 27.9 and 28.1 should be almost as common as 28?

[Organization XIII]

What about the winners bracket? Is there a higher degree of difference away from the gradiant inside of the down 1 (2-1,3-1, so on and and so on) and down 2 (3-2, 4-2 etc.)? Or does this fall across the board? I say winners bracket theoretically those are the better teams debating with the more preferred better judging. I don't think anyone really cares if the 0-5 bracket really has a constant 5 point interval system.


Edit:I'd like to say I love the 100 point scale.

[Antonucci23]

Independent of statistical effects, a 100 point scale just makes more sense. We grade generally on a 100 point scale, so most people are familiar and comfortable with it. The 30 point scale was just a weird anomaly that people maintained because they're afraid of change.

[A Numbers Game]

Originally Posted by Organization XIII

What about the winners bracket? [. . .] I say winners bracket theoretically those are the better teams debating with the more preferred better judging. I don't think anyone really cares if the 0-5 bracket really has a constant 5 point interval system.

I think this is a good question that deserves to be addressed at length.

A shorter answer is that there may be a difference, but not enough to make the point distribution in rounds where a team might break look smooth. I suspect this is partially because it takes many rounds to identify even a few teams that are definitely going to clear and a few that will definitely not. This means there is no sub-pool of highly preferred judges that judge rounds near or above the cut.

[A Numbers Game]

Originally Posted by Antonucci23

Independent of statistical effects, a 100 point scale just makes more sense. We grade generally on a 100 point scale, so most people are familiar and comfortable with it. The 30 point scale was just a weird anomaly that people maintained because they're afraid of change.

Another scale to consider is 1 to [number of competitors in division]. Judges would be asked to rank a speaker based on the judge's expectation of the speaker's final ranking out of all speakers in that division.

I think, if this scale were used, the actual average would be better than [number of competitors/2], for the same reason we have grade inflation.

[topspeaker70]

Kudos to Georgetown.

Speaker points have become a joke. Over the past four years, I have seen printed judging instructions at both the high school and college levels saying that the judge should "almost never" give anything under an 18. That, in effect, means that the "traditional" 30-point system is, in fact, a 12-point system. And some of the specific instructions on some college ballots (I mention no names) are just plain stupid* (i.e. "an average debater should receive 24-25 points").

Since so many judges are teachers/former teachers, giving them this wider - and more familiar - range of "grading" will produce "fairer" results. And it is consistent with a great tradition started by Jimmy Unger, who, as early as 1970, had written on the Georgetown ballots "Award speaker points as if you were giving grades in school."

One final thought: I would hope that, with the 100-point ballot, ties in speaker points are discouraged, if not prohibited.

*This type of rhetoric is but one of the many reasons I will never be POTUS.

[dropk2day]

statistically speaking isnt it the same thing? either way the trend is about the same, there's still going to be a normal distribution (bell curve, roughly) with 68% of the data being within one standard deviation from the mean and 95% within 2. anyone who does better than that would have still gotten the same speaker awards, breaks would be roughly the same. i dont see what the point is.

[Fox On Socks]

At a certain level, increasing the number of discrete values available (either by extending the scale to 100 or allowing more fractions of points on the 30-pt scale) doesn't help significantly and introduces greater chances for arbitrary scoring. For example, if someone someone delivered a 28.7 (or 82) point speech and their opponent delivered a 28.8 (or 83) point speech, would you be able to tell them apart?

This is not as much a problem in individual rounds as it's usually easy to rank-order the debaters (even if you don't know whether someone was one tenth or two tenths of a point better than another). But when aggregating the scores for awards, it's tougher to maintain uniform judging standards when more values are available precisely because it is very difficult for an objective observer to say a given speech is definitely a 27.5, not a 27.3 or a 27.7...

[A Numbers Game]

Originally Posted by dropk2day

statistically speaking isnt it the same thing? either way the trend is about the same, there's still going to be a normal distribution (bell curve, roughly) with 68% of the data being within one standard deviation from the mean and 95% within 2.

The average open points awarded at non-Wake tournaments this year was about 27.7, and the standard deviation was about .77. This makes the usual speaker points represent:

26.0: 1st percentile
26.5: 6th percentile
27.0: 18th percentile
27.5: 40th percentile
28.0: 65th percentile
28.5: 85th percentile
29.0: 95th percentile
29.5: 99th percentile

I suspect many judges feel that they can make finer distinctions than this, especially between debaters in one round, just like I suspect most teachers want a finer grading scale than A+,A,B,C+,C-,D,F,F-.

Originally Posted by dropk2day

anyone who does better than that would have still gotten the same speaker awards, breaks would be roughly the same. i dont see what the point is.

But the coarse scale magnifies little errors, which could make missing on points more random than it has to be. Point-based seeding also affects who gets a bye in partial elims and who gets to debate a 5-3 vs who has to debate a 6-2.

[coach_hanes]

Very interesting stuff. Thanks for the thoughtful analysis.

There's a line of psychological research that people aren't really capable of keeping more than 6-8 "objective" distinctions: more than that and colors, notes, and the like start to seem to blend together.

It seems like this does happen when judges give one holistic score for speaker points. Perhaps no matter what the scale, judges' scores will converge on 6-8 common scores. (For example, it looks like the 100-point scale is really 7 basic scores: 70, 75, 80, 85, 90, 95, 100.)

Just a thought -- tournaments might be able to break this obstinate human habit and encourage finer granularity by giving judges a rubric: separate categories, say five of them (e.g., analysis, cross-examination, organization, persuasion, use of evidence), each scored separately on its own 1-6 scale, and then the tournament or judge adds them up to come up with a composite, 30-point score. To be sure, there would still be a lazy option, giving an "average" score in each category (a 5?), but most judges aren't that lazy. A rubric would help them identify weaknesses in otherwise strong performance, and the range of scores might increase as a result.