Award Accuracy

[DMWOOD]

I had asked some questions in another thread viewtopic.php?t=374about how I could run a better race with the equipment I had. We currently use the Sterns method for scheduling and it was suggested that I look at this as one spot for improvement.

Finally this weekend I had a chance to look at the information Stan suggested. For my case (15 boys or less) it looks like a CPN 13-3 or 19-3 will be the most accurate race to run. But in looking at this information and thinking about how our District is run, I have some questions I need to ask.

Our District has 60 - 80 boys all racing at once and they are trying to award 1st - 3rd place to each group (Tiger, Wolf, Bear & Webelos). Last year the race was run on a three lane track using the Sterns method and each boy raced only 8 times. From what I have been reading, I question how accurately this method presented the awards for our District. What are your suggestions to improve our District race?

Our District usually does everything at the last minute but I have about two months before the race to get involved and see if I can help improve this years race. I have some ideas but I would like some thoughts from the experts here before I try to make any suggestions.

Thank you for your help.

[Stan Pope]

The accuracy of Stearns and PPN (final standings) charts for large numbers within a reasonable number of heats is discussed at
http://members.aol.com/standcmr/pwmevalc.html
A simple "final standings" race has poor accuracy for awarding trophies.

A Stearns final standings will do a creditable job of selecting finalists. To have a high probability that the N (number of trophies) fastest cars are in the group of finalists, the number of cars selected for the finals often needs to be about 2*N.

Since your district has a predisposition for Stearns, it would be easiest to stay with that format, but use it to select finalists rather than trophy recipients. This is described in Case 4 in the web page above.

[dknowles67]

My Pack's pinewood derby seems remarkably similar to yours.
This year we had 65 out of a possible ~100 cub scouts race.
We have always used DE, on a 3 lane track, and awarded 1st 2nd & 3rd place for each rank, and a 1st, 2nd, 3rd overall.
I have had concerns in the past about the fairness/accuracy of all this, and decided to do something about it after this years derby.
Stan and Cory have been a great help answering all my questions.

Stan seems reluctant to make a recommendation, but prefers to present all the facts and let you make up your own mind.

I'll come right out and say that I would recommend the PPN/PN charts to the Stearns, based on the accuracy results shown in the link from Stan's post.
From a cub scout/parents points of view, I'm not sure you'd be able to notice much difference between the Stearns, and PPN methods. You get lots of races, in every lane against a lot of opponents. You won't know who won until its all over. PPN just seems like it's more accurate.
After conducting my research (still in progress), it seems like doing a 2 round PPN first on large groups( > 8 scouts), and then doing a CPN on the 4-7 fastest cars, will produce the most accurate results. If you do a 2 round PPN (really PN or CPN) on the smaller groups, there is no need for finals, they are already accurate enough.
I'd be willing to provide more info if required, and possibly offer more help once I've completed my research.

[Cory]

From a cub scout/parents points of view, I'm not sure you'd be able to notice much difference between the Stearns, and PPN methods. You get lots of races, in every lane against a lot of opponents. You won't know who won until its all over. PPN just seems like it's more accurate.

Stearns charts, if you study them closely, will sometimes exhibit "spikes" in their opposition equity. For example, Car A races Car C one time, but Car B races Car C four times. If Car C is really fast, then Car B's owner and parents will often notice it. To quote Stan's site:

"Anomolies such as these contribute to inaccuracy. They are 'glaring' when 'critical' cars are involved."

PPN charts are a little easier to "defend", since they have "best-possible" opposition equity.

Also, because of the way they are constructed, Stearns charts will sometimes put more than one "bye" (i.e. empty lane) in the same heat. With a 3-lane track, this means a car might be "racing" all by itself.

With PPN, you have control over the number of byes. Sometimes it's actually better to have a single bye (e.g. with 6 cars, many prefer to use a PN or CPN 7-car chart instead of PPN 6-car chart). In most cases, it's preferable to have zero byes.

One nice thing about Stearns, though, is if you don't like a chart you can ask the program for another one and the second one will almost always be different from the first.

The PPN generator pgms, given the same input parameters, will always return the same chart.

[DMWOOD]

My pack is small (15 boys), it is our District that has 60 - 80 racers. But in either case we run them all as one group and then try to pick the top three from each of the 4 ranks as well as the top three overall.

Most of the accuracy information I have read has been based on picking the top X racers from the group of racers. We are trying to pick the top 3 racers from subsets within this large group. I am not sure how this effects the accuracy of the awards presented but my thought is that it's not good.

My thought for my small pack is that a CPN (with byes if necessary) could be used without a finals race because everyone is racing each other twice and lane usage is even. Please correct me if I am wrong.

I think what Stan was suggesting for our District was to run as we did but then pick the top 6 or 7 from each rank and have finals races for each rank to determine the awards. Again please correct me if I am wrong.

In the end we just want to run the most fair and accurate race we can so we do not affect who wins an award and who does not.

Thanks for your help and If anyone else has something to add, please do.

[Cory]

My thought for my small pack is that a CPN (with byes if necessary) could be used without a finals race because everyone is racing each other twice and lane usage is even. Please correct me if I am wrong.

CPN will do a good job of sorting your entire Pack and your Dens, all at the same time.

Most of the accuracy information I have read has been based on picking the top X racers from the group of racers. We are trying to pick the top 3 racers from subsets within this large group. I am not sure how this effects the accuracy of the awards presented but my thought is that it's not good.

I agree. Stan's analysis assumes you are picking the top 7 from the entire set, whereas you plan to take the top 7 from various subsets.

My chart simulator isn't smart enough to do this analysis, but I'm inclined to agree with you, the accuracy will suffer.

Given your requirements, I think a "cumulative time" method would be your best trimming tool, then you could run CPN charts for each rank if desired.

[dknowles67]

From a cub scout/parents points of view, I'm not sure you'd be able to notice much difference between the Stearns, and PPN methods

The point I was going for here, was that PPN won't be substantially more/less racing, or take much longer. I can't see why anyone would argue that we should have stuck with Stearns instead of going to a PPN type chart. PPN would result in higher accuracy, and therefore be an improvement that someone may notice.

(Cory made a good point).

[Darin McGrew]

My pack is small (15 boys), it is our District that has 60 - 80 racers. But in either case we run them all as one group and then try to pick the top three from each of the 4 ranks as well as the top three overall.

The home-brew software we've been using for the past few years schedules cars to race only within their own class, and then intermingles the races (race #1 for Builders, race #1 for Sentinels, race #1 for All Comers, race #2 for Builders, race #2 for Sentinels, race #2 for All Comers, and so on). There are several advantages:

• No one races against cars from another class.
• No one races in two consecutive races (which makes staging easier).
• Everyone is involved throughout the entire event.

At the end, we race the fastest cars from each class against each other to find the fastest overall. (The trophies go to the fastest in each class; the fastest overall only gets a ribbon.)