Friday, July 13, 2012

Meta-debate tournament tabulation post

I have been at this blog for three and a half years. I have discussed neat calculus, geometry, and statistics problems; I have laid out some discussions for a critical thinking course; but in about half the posts, I have discussed tournament tabulation procedures. I have been working through a lot of ideas as I try to make a case for building a new generation of programs.

I just received a new book today, Who's #1? The Science of Rating and Ranking, by Langville and Meyer, two math professors. They state that they were frustrated that the methods they discuss in their book were not collected in any one other single book. I can say I share that frustration. At an initial look through it, the book is amazing. A lot of the methods they discuss I have looked at in one form or another in considering a tab program; many I have not. It is both validating and humbling. I have a lot of work to do! So, for quite a while, I am going silent on tab programs while I read this book. I will still post on neat mathematics problems and critical thinking course discussion ideas.

Before I go temporarily silent on the tab program topic, though, I thought it would be good to summarize what I have written so far. My positions have evolved in three and a half years considerably.

I started out looking at whether high-low and high-high pairings created fair schedules for teams, specifically looking at opponent wins. I looked at the Harvard tournament, a set of big national tournaments, and what was possible in theory. In general, I think the case is pretty clear that even at big tournaments, where constraints should not be an issue, the traditional methods fail to deliver fair schedules -- even within a bracket. Teams break with easy schedules; teams don't break with hard schedules.

I looked for a way to pair within brackets that I felt was more fair. The idea I hit on was strength-of-schedule pairings: a team would get an opponent that would balance out its schedule difficulty compared to other teams in the same bracket. This is a method only a computer can do, since it requires simultaneously evaluating whether team A is a good opponent for team B (as defined above) AND whether team B is a good opponent for team A. Keeping track of both ratchets up the complexity beyond a human's hands. It is still an understandable method, just too many calculations for a person to do. I tried it on a small tournament and a large tournament and found that it does indeed work to even out the schedules.

That problem "solved," I started thinking about evaluating a team's strength (and therefore also a team's schedule strength) in a more sophisticated ways than just wins and losses. I looked at graph theory, but for all its promise for some tiebreakers in round robins, it is not a good method for regular tournaments. I looked at one weighted wins scheme, where the points per win decline for each subsequent round, but this system is not good. I next looked at a weighted wins scheme where a team receives extra "points" for its defeated opponents' wins and loses points for its defeating opponents' losses. This does seem to work well for a simple method, and it lead me to think about more complex ways of getting, from the data, a team's strength on the affirmative and strength on the negative. I have also been thinking about how reliable even the best methods are. How many "upsets" are there in debate?

Along the way, I have also thought about side assignment here. I have written about how many teams will break here, here, and here. And I have sparred with A Numbers Game on whether there is topic side bias (sorry, it looks to me like novices muck up the average; varsity debaters get closer to parity) or judge side bias (again, sorry, it looks alright to me).

So where have I landed? I started out thinking all I wanted to do was propose a different within brackets pairing algorithm, which then expanded to thinking about measuring a team's strength and opponent strength. But a very early post on round robins planted the seed: preliminary rounds don't have to be elim rounds. They don't need winners and losers brackets. Teams need to meet all their opponents, or failing that, a good cross-section. So the fairest statement of what I believe now is that tournaments should ditch the brackets and start making sure that teams get mixed up by skill level and geography in prelims; let a sophisticated algorithm do the mixing, not random chance; let the best teams break, and use elims like they always have been: to pick the champion. And I think N.F.L. Nationals ought to start.

No comments:

Post a Comment