22 replies to this topic

[A Numbers Game]

magicmasterk said:

My general question though was does your analysis assume the nature of MPJ including the fact that not as preferred judges judge at the JV/Novice level which may alter their side tendencies.

I should have been more clear above: the numbers I posted do not take into account MPJ or division. Many of the judges listed, as you noted, are highly preferred, and so probably judged some of the best rounds. These judges also, however, appear on both sides of the side-bias charts.

The next bias numbers I plan on posting will be about resolutional side bias, and the numbers will take into account opponent strength.

[DJ Kazic]

A Numbers Game, on 12 May 2009 - 12:25 PM, said:

I should have been more clear above: the numbers I posted do not take into account MPJ or division. Many of the judges listed, as you noted, are highly preferred, and so probably judged some of the best rounds. These judges also, however, appear on both sides of the side-bias charts.

The next bias numbers I plan on posting will be about resolutional side bias, and the numbers will take into account opponent strength.

Although I feel A Numbers has done extensive research, a conclusion can't be made until sufficient data is gathered anyway. Right now it's too little amounts of votes to take the judges voting habits into account. Notice that I say "voting habits," and not bias. This is because at the smaller amount of data, even a small difference in voting can attribute to a large change in deviation. Thus, the data can easily be skewed.

Just my $0.02.

[nathan_debate]

In response to the initial post as well as the one below.....bias doesn't mean unfairness. Bias doesn't mean bad judge. That side bias could just mean consistency. Consistency is actually something to be valued....as long as you can find out what the modus operondi of the consistency is.

That said....those judges with 10 to 1 ratios I might want to stay away from (ie Holland & Carson).....thats a rather profound tilt in one direction or the other.

Quote

I'm no statistician but are you all accounting for the fact that debate round assignments are not random. Mutual Preference Judging at nearly all tournaments in theory assigns judges to rounds both teams want them to judge. Perhaps it is something about the teams and the types of rounds these particular critics judge that attribute to these figures.

I am not making accusations but if you are perhaps suggesting that these judges will vote based on these figures, you are discrediting some fairly respected judges in the community. I don't know 80% of the people on your website but I do know that Brian McBride, Jason Russell, Greta Stahl, Toni Nielson, and Erik Holland don't make arbitrary decisions based on side. If you are a college debater / college coach and you're looking at these numbers, I do not believe that these statistics should be an important facet in how you fill your pref sheets out.

That said, I do believe your project is interesting. I do think better statistical work should be done analyzing whether there really are statistically significant aff biased / neg biased topics as people claim on different years.