Race Schedules & Software

Debates and discussions on the various race scheduling methods that can be used and their fairness and accuracy in determining the winners.
User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Race Schedules & Software

Post by Stan Pope » Wed Oct 13, 2010 2:21 am

Continuing my discussion from my prior post ...

I plugged in generators for a 12 car CPN chart. Not unsurprisingly, the number of heats was 132 (12*11 from the formula that I showed above). At a typical rate of one heat per minute, that would take more than 2 hours! If you had 30 racers, this would be 870 = 30*29 heats (about 15 hours!)

There is another strategy that I suggest. Use a reasonable number of heats as a "screening race" to identify some number of finalists. From other analyses, I've found that a group of finalists numbering twice as many as I have place trophies to award usually assures that the deserving are among the finalists.

Then race the finalists in a PN or CPN chart.

So, if you are awarding trophies for places 1-3, then use a 4 round PPN chart for screening all the racers and then a 6 round chart for the 7 highest scoring racers.

For 30 racers this would be 120 heats for the prelim ... each racer makes 8 runs ... and 42 heats for the finals.

Maybe it is time for your pack to try a wider track! Or (much as I dislike 'em) run the competition based on average elapsed time for 6 or 8 heats. A timer for a 2 lane track is not that expensive!


Stan
"If it's not for the boys, it's for the birds!"

theDoc
Apprentice
Apprentice
Posts: 5
Joined: Sun Oct 10, 2010 12:45 pm
Location: Valdosta, GA

Re: Race Schedules & Software

Post by theDoc » Wed Oct 20, 2010 11:00 am

Stan Pope wrote:Continuing my discussion from my prior post ...

I plugged in generators for a 12 car CPN chart. Not unsurprisingly, the number of heats was 132 (12*11 from the formula that I showed above). At a typical rate of one heat per minute, that would take more than 2 hours! If you had 30 racers, this would be 870 = 30*29 heats (about 15 hours!)

There is another strategy that I suggest. Use a reasonable number of heats as a "screening race" to identify some number of finalists. From other analyses, I've found that a group of finalists numbering twice as many as I have place trophies to award usually assures that the deserving are among the finalists.

Then race the finalists in a PN or CPN chart.

So, if you are awarding trophies for places 1-3, then use a 4 round PPN chart for screening all the racers and then a 6 round chart for the 7 highest scoring racers.

For 30 racers this would be 120 heats for the prelim ... each racer makes 8 runs ... and 42 heats for the finals.

Maybe it is time for your pack to try a wider track! Or (much as I dislike 'em) run the competition based on average elapsed time for 6 or 8 heats. A timer for a 2 lane track is not that expensive!

So after reading and rereading your site (lots of info there!):
http://www.stanpope.net/gen2rr.html" target="_blank
I think I finally get it, but I would like your opinion Stan...
We have a two lane track and I think that to maximize the # of races (fun for the boys), yet minimize the time frame (most parents concern), I am thinking of doing a full round robin (using your charts)as a first set and then take the top 7 and double round robin (second set) for awards. Assuming that all dens have 12 or more cars qualified, if less I will just do the double round robin once.

Or would the PPN chart for the preliminary be that much faster and then DRR the top seven?

Either way I think I should let the boys graphite up the wheels before the start of each set (not round), right?

Thanks again for your patience AND knowledge,
Scott



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Race Schedules & Software

Post by Stan Pope » Wed Oct 20, 2010 2:58 pm

theDoc wrote:I think I finally get it, but I would like your opinion Stan...
We have a two lane track and I think that to maximize the # of races (fun for the boys), yet minimize the time frame (most parents concern), I am thinking of doing a full round robin (using your charts)as a first set and then take the top 7 and double round robin (second set) for awards. Assuming that all dens have 12 or more cars qualified, if less I will just do the double round robin once.

Or would the PPN chart for the preliminary be that much faster and then DRR the top seven?

Either way I think I should let the boys graphite up the wheels before the start of each set (not round), right?

Thanks again for your patience AND knowledge,
Scott
I'd start by noting the total time avaliable and how many heats could be run per hour. (To make the discussion concrete, let's assume 5 hours are available, and that you can run one heat per minute. That leaves time for 60 heats per age group.) This should tell you about how many heats you will have time for. Then divide those heats among the 5 age groups that will race separately. It is helpful to have each group run the same number of heats. The boys will see this as "fair". The downside is that probably some of the groups will run fewer heats than are needed for a full RR.

The value of the "preliminary screening + finals" method is that some inaccuracy in the screening does not hurt the overall accuracy of the racing. It is only needed that those deserving of a trophy reach the finals.

I suspect that you will be really pushed to do 7-car DDR's for each of 5 grades plus prelims for each grade as well in the time available. So, I'd look for a simplification that would cut the time with minimal reduction of accuracy. We need enough races in each prelim to get sufficient accuracy to give a high probability that the "deserving" will make the finals and the finals must have acceptable accuracy. Usually, if you send twice as many racers to the finals as you have trophies to award, the probability that all of the deserving get to finals is in the high 90%'s, even without exhaustive or totally balanced prelims. The prelim needs to be "kinda deep" but "not exhaustive." I've used Cory Young's simulation software to analyze accuracies and have seen that this two stage competition yields good accuracy while keeping the number of heats much smaller than single stage exhaustive methods of comparable accuracy.

Finally, a single round robin requires half as many heats as a DDR, but the accuracy is only slightly less. From the estimates above, I think that you will need those saved heats to fit in a satisfactorily deep preliminary.

The next step is to "get specific." How many trophies per grade? How many boys in each grade? How evenly matched are the lanes of the track? How many heats can you run per hour? Then, let's work up the details.


Stan
"If it's not for the boys, it's for the birds!"

theDoc
Apprentice
Apprentice
Posts: 5
Joined: Sun Oct 10, 2010 12:45 pm
Location: Valdosta, GA

Re: Race Schedules & Software

Post by theDoc » Wed Oct 20, 2010 5:09 pm

Lets see if I can answer all your questions:

1. 1st, 2nd, and 3rd will be given trophies

2. Last year we started at 9 am and finished around 2 to 3 pm, with a break for lunch...this year I have proposed that each den build a model rocket and we shoot them off after each den's race. That will give me time to look over all the results and double check everything before I make a mistake and announce the wrong winner. I think the parents wouldn't mind another hour this year, but that's about it.

3. All five dens have at least 20 boys, I think two dens have 30-35. Given that last year about 60-80% of the boys made cars, I don't expect more than 25 per den (some boys leave or are involved in sports a lot in the spring).

4. As far as the track lanes...I think they are about equally matched, with MAYBE a SLIGHT difference on the far lane

Thanks,
Scott



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Race Schedules & Software

Post by Stan Pope » Wed Oct 20, 2010 9:47 pm

Okay, here is how I would attack it:

Assumptions:
Pack has 120 Scouts.
You can run 60 heats per hour.
You have 5 1/2 hours available for racing.
Clerical work on one rank's racing is overlapped with heats for the next rank.
You have time to run about 330 heats... more if you can increase the heats/hour rate.
Analysis using Young's Simulation software (http://members.cox.net/pinehead/software.html" target="_blank) with parameters as given below.

Tentative assumptions:

80% of Scouts race => 120 * 0.8 = 96 racers competing.
7-racer finals for each rank
Den racers: 15, 16, 18, 21, 26 (total 96)

Prelims:

Sim parameters:
Car Speed StDev: 50 (2/3 of prelim racers are within 2*5.0=10 inches of each other at finish line)
Lane Speed StDev: 8 (2/3 of racers gain or lose 0.8 inches or less by changing lanes)
Tie Breaking: Head to Head
Random Element StDev: 5 (when prelim racers run again, 2/3 of their runs gain or lose 0.5 inches or less)
Reduction Parameter: 3 (3 trophies to award)
Scoring: Linear
Number of Trials: 10000 (consistency within about 0.5%)

Selection accuracy:

15 Racers
2 round: 3n7: 98.8%
3 round: 3n7: 99.2%
16 Racers
2 round: 3n7: 98.4%
3 round: 3n7: 99.4%
18 Racers
2 round: 3n7: 98.0%
3 round: 3n7: 99.2%
21 Racers
2 round: 3n7: 97.5%
3 round: 3n7: 98.5%
26 Racers
2 round: 3n7: 97.8%
3 round: 3n7: 97.2%
In all cases simulated, increasing the number of heats per racer produced at best a negligible improvement in selection accuracy.


Finals:

Sim Parameters:
Car Speed StDev: 20 (2/3 of finals racers are within 2*2.0=4.0 inches of each other at finish line)
Lane Speed StDev: 8
Tie Breaking: Head to Head
Random Element StDev: 2 (when prelim racers run again, 2/3 of their runs gain or lose 0.2 inches or less)
Scoring: Linear
Number of Trials: 10000 (consistency within about 0.5%)

Trophy assignment accuracy:
RR (PN): 3 Trophy: 65%, Top 3: 89%
i.e. in 65% of the trials, the correct trophy was awarded to the three objectively the fastest cars,
and in 89% of the trials, the some trophy was awarded to the three objectively the fastest cars.

DRR (CPN): 3 Trophy: 87%, Top 3: 93%
i.e. in 87% of the trials, the correct trophy was awarded to the three objectively the fastest cars,
and in 93% of the trials, the some trophy was awarded to the three objectively the fastest cars.

Comparison:

2 round prelim (192 heats, 4 per scout) + DRR (210 heats)
Total 402 heats + tie breakers. Much better accuracy
3 Trophy Accuracy: 0.87 * 0.98

3 round prelim (288 heats, 3 per scout) + RR (105 heats)
Total 393 heats + tie breakers (probably fewer).
3 Trophy Accuracy: 0.65 * 0.99

Both of these methods require more heats than you may have time for.


Charts used: Charts were generated using http://stanpope.net/ppngen.html" target="_blank (same charts as would be produced by Randy's GPRM software.
Last edited by Stan Pope on Thu Oct 21, 2010 2:09 am, edited 1 time in total.
Reason: Correct interpretation of StDev measures for Sim


Stan
"If it's not for the boys, it's for the birds!"

*5 J's*
Master Pine Head
Master Pine Head
Posts: 756
Joined: Fri Apr 03, 2009 8:55 am
Location: Norway, Maine

Re: Race Schedules & Software

Post by *5 J's* » Thu Oct 28, 2010 8:17 pm

Stan, I ran the simulations below for four different 40 car 2 lane PPN charts. One with 1 rounds, one with 2 rounds, one with 3 rounds, and one with 4 rounds. Why is the accuracy going DOWN as the number of rounds is increased?

Sim parameters:
Car Speed StDev: 50
Lane Speed StDev: 8
Tie Breaking: Timer
Random Element StDev: 5
Reduction Parameter: 3
Scoring: Linear
Number of Trials: 1000

Partial Perfect 40-2 (1 Round) Chart
1-Trophy Accuracy: 99.10% Top-1 Accuracy: 99.10%
2-Trophy Accuracy: 95.40% Top-2 Accuracy: 95.85%
3-Trophy Accuracy: 90.47% Top-3 Accuracy: 93.17% 3n3 Accuracy: 93.17%
4-Trophy Accuracy: 84.13% Top-4 Accuracy: 90.43% 3n4 Accuracy: 98.87%
5-Trophy Accuracy: 76.86% Top-5 Accuracy: 87.58% 3n5 Accuracy: 99.73%
6-Trophy Accuracy: 69.83% Top-6 Accuracy: 85.48% 3n6 Accuracy: 99.97%

Partial Perfect 40-2 (2 Round) Chart
1-Trophy Accuracy: 97.40% Top-1 Accuracy: 97.40%
2-Trophy Accuracy: 91.10% Top-2 Accuracy: 92.15%
3-Trophy Accuracy: 82.70% Top-3 Accuracy: 87.37% 3n3 Accuracy: 87.37%
4-Trophy Accuracy: 72.80% Top-4 Accuracy: 82.23% 3n4 Accuracy: 95.23%
5-Trophy Accuracy: 63.22% Top-5 Accuracy: 77.90% 3n5 Accuracy: 98.07%
6-Trophy Accuracy: 54.97% Top-6 Accuracy: 74.90% 3n6 Accuracy: 99.27%

Partial Perfect 40-2 (3 Round) Chart
1-Trophy Accuracy: 95.40% Top-1 Accuracy: 95.40%
2-Trophy Accuracy: 86.25% Top-2 Accuracy: 88.05%
3-Trophy Accuracy: 75.23% Top-3 Accuracy: 82.17% 3n3 Accuracy: 82.17%
4-Trophy Accuracy: 64.40% Top-4 Accuracy: 77.70% 3n4 Accuracy: 90.80%
5-Trophy Accuracy: 54.98% Top-5 Accuracy: 75.94% 3n5 Accuracy: 94.83%
6-Trophy Accuracy: 47.93% Top-6 Accuracy: 77.33% 3n6 Accuracy: 96.87%

Partial Perfect 40-2 (4 Round) Chart
1-Trophy Accuracy: 94.20% Top-1 Accuracy: 94.20%
2-Trophy Accuracy: 81.75% Top-2 Accuracy: 84.15%
3-Trophy Accuracy: 68.67% Top-3 Accuracy: 78.90% 3n3 Accuracy: 78.90%
4-Trophy Accuracy: 58.38% Top-4 Accuracy: 79.20% 3n4 Accuracy: 88.77%
5-Trophy Accuracy: 51.54% Top-5 Accuracy: 81.16% 3n5 Accuracy: 92.93%
6-Trophy Accuracy: 46.73% Top-6 Accuracy: 82.47% 3n6 Accuracy: 95.90%
Last edited by *5 J's* on Fri Oct 29, 2010 3:00 am, edited 1 time in total.



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Race Schedules & Software

Post by Stan Pope » Thu Oct 28, 2010 11:49 pm

Interesting results!

I'll give you a guess at what is at work in this case but I'll leave definitive response to Cory.

The cause of the anomoly is that tie break is invoked very frequently for a 2-round race on a 2-lane track! And the specified tie break procedure is "head to head".

With 2 lanes and a small number of rounds, there are few score groups for racers to occupy. Lots of ties, even among the top group. If you add rounds, there are more, smaller score groups, so that actual scores more closely depict the actual speeds.

But ... if many of the fastest racers are among the top group, then the specified tie break method will sort them out at decreasing expense as the number of rounds increases and, therefore, the number of ties decreases.

Try a different tie break method, especially random, since that will give a clearer picture of how many slower racers make the top group when there are only a few heats per racer. And, with a few heats, you almost always have to make the top group to win a trophy. The accuracy measure suffers in those cases that a would-be trophy winner races a slightly faster car, he gets bumped out of the top group. The effect of using random tie break is that lesser cars who did make the top score group get awarded trophies much more often, but that effect declines as the number of rounds increases to 3 and 4.

With head-to-head tie break, I think that you would see that after a few rounds, the accuracy measures begin to ramp up and soon surpass the initial poor accuracy measures.

I haven't actually run sims to demonstrate this analysis, but I think that it is sound.

Edit:
Well, I did run some sims for 1 thru 4 rounds, with all three modes of tie-break. Convinced myself by the extended run time for head-to-head tie break that there was a lot of tie-breaking going on, especially in the 1 and 2 round charts.

And accuracies went to pot when the slower cars had a 50-50 chance of beating the top car in a tie-break race! (the random or flip-a-coin tie break.) That tells me that the chart accuracy was helped ... a lot ... by the tiebreak process.

Thank you for raising the question. I learned a bunch from it! :)
Last edited by Stan Pope on Fri Oct 29, 2010 2:50 am, edited 1 time in total.
Reason: Add sim run info


Stan
"If it's not for the boys, it's for the birds!"

*5 J's*
Master Pine Head
Master Pine Head
Posts: 756
Joined: Fri Apr 03, 2009 8:55 am
Location: Norway, Maine

Re: Race Schedules & Software

Post by *5 J's* » Fri Oct 29, 2010 5:20 am

Stan - I reran the numbers at 4am at noted that I had hit the Timing Tie Breaker radio button. I have edited my post accordingly. I also added a 1 round 40-2 PPN schedule.

Below are the same simulations run as above but with 50-50 (flip of a coin) selected for tie breaking. I need to spend some more time with the simulator to understand the results, though the simulator is for a points based format vice timing which I plan to use. With timing the accuracy of the scheduling would increase (assuming the timer and timing process itself is accurate). I am on the Mac again - so I cannot run a Head-to-Head tiebreaker set - but will run when I'm back on the PC.

40-2 (1 round)
1-Trophy Accuracy: 7.30% Top-1 Accuracy: 7.30%
2-Trophy Accuracy: 7.30% Top-2 Accuracy: 15.05%
3-Trophy Accuracy: 7.10% Top-3 Accuracy: 22.23% 3n3 Accuracy: 22.23%
4-Trophy Accuracy: 6.93% Top-4 Accuracy: 27.63% 3n4 Accuracy: 27.70%
5-Trophy Accuracy: 6.86% Top-5 Accuracy: 33.96% 3n5 Accuracy: 33.70%
6-Trophy Accuracy: 6.67% Top-6 Accuracy: 39.42% 3n6 Accuracy: 39.33%

40-2 (2 round)
1-Trophy Accuracy: 13.50% Top-1 Accuracy: 13.50%
2-Trophy Accuracy: 12.35% Top-2 Accuracy: 24.30%
3-Trophy Accuracy: 11.90% Top-3 Accuracy: 34.60% 3n3 Accuracy: 34.60%
4-Trophy Accuracy: 11.03% Top-4 Accuracy: 43.90% 3n4 Accuracy: 43.80%
5-Trophy Accuracy: 10.32% Top-5 Accuracy: 50.80% 3n5 Accuracy: 51.67%
6-Trophy Accuracy: 9.73% Top-6 Accuracy: 57.52% 3n6 Accuracy: 58.87%

40-2 (3 round)
1-Trophy Accuracy: 17.40% Top-1 Accuracy: 17.40%
2-Trophy Accuracy: 16.40% Top-2 Accuracy: 34.10%
3-Trophy Accuracy: 14.90% Top-3 Accuracy: 47.37% 3n3 Accuracy: 47.37%
4-Trophy Accuracy: 13.75% Top-4 Accuracy: 56.43% 3n4 Accuracy: 57.87%
5-Trophy Accuracy: 12.72% Top-5 Accuracy: 62.30% 3n5 Accuracy: 66.20%
6-Trophy Accuracy: 11.92% Top-6 Accuracy: 66.08% 3n6 Accuracy: 73.23%

40-2 (4 round)
1-Trophy Accuracy: 23.40% Top-1 Accuracy: 23.40%
2-Trophy Accuracy: 19.95% Top-2 Accuracy: 42.80%
3-Trophy Accuracy: 17.87% Top-3 Accuracy: 55.90% 3n3 Accuracy: 55.90%
4-Trophy Accuracy: 16.28% Top-4 Accuracy: 62.35% 3n4 Accuracy: 66.10%
5-Trophy Accuracy: 15.42% Top-5 Accuracy: 67.38% 3n5 Accuracy: 74.80%
6-Trophy Accuracy: 14.52% Top-6 Accuracy: 70.47% 3n6 Accuracy: 81.27%



User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Race Schedules & Software

Post by Stan Pope » Fri Oct 29, 2010 11:05 am

These results show low accuracy.

The reason that the fastest car
40-2 (1 round)
1-Trophy Accuracy: 7.30% Top-1 Accuracy: 7.30%
shows so poorly is as follows:
1. There are 5 possible scores for each of the 40 racers.
2. The fastest car is in the highest scoring group most of the time by virtue of its speed.
3. Several other cars are in the highest scoring group by virtue of chance in the pairings. I'd guess that there are up to 14 of these present.
4. Suppose that there are only 8 racers in the highest scoring group. Most of the time, this group will include racers who did not race faster cars. There are lots of opportunities for the "not so fast" (the slower 30+ cars) to race only slower cars.
5. Some percentage of the time, cars among the top 5 or 10 fastest will race a faster car and, of those, most of the time, they will be in the next slower score group. The rules of scoring will place these cars behind ALL of the cars that managed by speed or chance to fall in the highest scoring group.
6. The final placement of cars in the highest scoring group is by chance, and with several slower cars in this group, the fastest will most of the time not "get the nod" in the coin flip tie break.

Once you understand the above, then it should become really clear why you need to have several runs in the prelims and why you need to use unbalanced opposition charts as screening rather than placement.

I tried some probability calculations (they get really nasty really quickly) this morning to see how the numbers play out in specific cases. Look at the probability that the Nth fastest racer will race a faster racer at least once in 2 heats, 4 heats, ... It is easier to compute if you look at the probability that he will not race a faster racer in those number of heats.
p(3rd fastest racer will race a faster racer at least once in N heats) =
1 - p(3rd fastest car will race only slower racers in N heats) =

N=2:
1 - (37/39)*(36/38)
N=4:
1 - (37/39)*(36/38) * (35/37)*(34/36)
N=6:
1 - (37/39)*(36/38) * (35/37)*(34/36) * (33/35)*(32/34)
The probability that a racer (even a high ranking racer) will race only slower racers decreases rapidly as the number of heats increases! The situation is compounded if most or all of the high ranking racers must race only slower cars in order to stay in the highest ranking group. Only when the number of groups grows so that the number of occupants of the highest ranking group(s) is forced to be low does the situation resolve.


Stan
"If it's not for the boys, it's for the birds!"

User avatar
Stan Pope
Pine Head Legend
Pine Head Legend
Posts: 6888
Joined: Sat Jul 05, 2003 7:01 pm
Location: Morton, Illinois
Contact:

Re: Race Schedules & Software

Post by Stan Pope » Fri Oct 29, 2010 5:58 pm

A bit of followup on the topic ... The cases discussed are at the extreme edge of the PPN operation. The combination of a small number of lanes and a small number of rounds work to cause each racer to face a small percentage of the opposition. In that light, it is understandable that details of operation, both of the racing and the simulation, are very important. In particular, simulation details such as how ties are resolved are extremely important and may (or may not) reflect how they would be resolved in a real race.

What this suggests to me is that simulations of such extreme cases must be handled with much more caution than one would handle simulations of less extreme cases.


Stan
"If it's not for the boys, it's for the birds!"

*5 J's*
Master Pine Head
Master Pine Head
Posts: 756
Joined: Fri Apr 03, 2009 8:55 am
Location: Norway, Maine

Re: Race Schedules & Software

Post by *5 J's* » Tue Nov 02, 2010 7:35 am

Stan Pope wrote:A bit of followup on the topic ... The cases discussed are at the extreme edge of the PPN operation. The combination of a small number of lanes and a small number of rounds work to cause each racer to face a small percentage of the opposition. In that light, it is understandable that details of operation, both of the racing and the simulation, are very important. In particular, simulation details such as how ties are resolved are extremely important and may (or may not) reflect how they would be resolved in a real race.

What this suggests to me is that simulations of such extreme cases must be handled with much more caution than one would handle simulations of less extreme cases.
So for somebody in this situation you have a few options
1) Increase the number of rounds at the expense of more time to conduct event
2) Increase the number of lanes at the expense of cost to purchase a new track or rent
3) Change from points based to time based scoring at the possible expense of accuracy due to timer issues

For us I don't think we can add enough rounds on our two lane track and complete in a reasonable time, nor do we have the capital this year to purchase or lease a track with more lanes, so we will use a timer but I will add a solenoid operated spring open gate to increase consistency of races.



Post Reply