Does CMA have a set of competition methods from which local organizers are encouraged or required to select for sanctioned events? Or does each local organizer get to "do his own thing" in selecting or concocting a scheme? If CMA has recommended or required methods, how do they go about deciding which methods should be included in the list? Is there a formal evaluation procedure for deciding if a method should be added to the list? ...
Rod Turnbull wrote:As for the one lane wonders and questions as to how things would have turned out if a competitor had been excluded from the event... this would be a kin to a top ranked rider or riders falling in the heat races or qualifying races and not making the finals... which ultimately comes down to 'that's just racing'...
Paragraph has good info, but answers a different question.
Rod Turnbull wrote:Have you looked into using a mean average or removing outliers when doing statistical analysis of the different methods of running a race? I don't have time to look at your paper today (work sucks) but I will look at it soon.
For analyzing actual competition results, I have worked with both arithmetic means and standard deviations with and without including the fliers. I usually only work actual race data to show whether or not there are abberations in the equipment or to develop "typical population data" for input to simulations. Actual race data tells you "one instance" of applying the method, usually for an unknown lane and racer population. Getting method accuracy measures requires a few thousand such races with lane and racer population distributions that reflect the real world (means and standard deviations). Even if I could get quality data, I don't have enough time to develop accuracy info that way.
Instead, the paper describes an evaluation concept involving simulation, using a statistical model of the racing environment (track and racers). It runs as many trials as are needed to stabilize the accuracy averages. For each trial a "racing environment" (including track and participants) is generated randomly, then the program applies the racing method to the environment. Each randomly generated sample contains "racers of known speeds on a perfect track" so the placement results from using the race method for a trial can be compared against the known rankings to tell the accuracy for the specific trial. After a few thousand such iterations, you can compute some accuracy averages for the method, including such things as the percent of time the Nth fastest racer finished in Nth place, and some other esoteric numbers.
Cory Young took the accuracy concepts from the paper and added/developed a good general case model of tracks and of racers and then built a nice simulation program dealing with charted points or timed racing and another that dealt with general N-loss elimination racing. My workbench has simulation programs for other racing methods in development, but I am lacking enough "round tuits" to finish 'em up!
Someone somewhere may have done something comparable, but I've not found 'em anywhere. So far as I've been able to tell, Cory's and my respective efforts are original and unique.