How many points?

[DMWOOD]

For a three lane track and using a point scoring system, what points should be assigned for places to give the best accuracy. I have seen two different sets of points used and wonder which one will give the most accurate results.

First = 3
Second = 2
Third = 1

First = 7
Second = 3
Third = 1

Any thoughts?

Thanks,

[Darin McGrew]

For a three lane track and using a point scoring system, what points should be assigned for places to give the best accuracy. I have seen two different sets of points used and wonder which one will give the most accurate results.

Is this for the purpose of determining overall results? Or is this for the purpose of qualifying finalists? Is this used with a PN schedule, where everyone races against everyone else? A CPN schedule, where everyone races against everyone else twice, with the lanes reversed the second time? Some other schedule?

First = 3
Second = 2
Third = 1

First = 7
Second = 3
Third = 1

What matters is the ratio of the differences. For example, there is no difference in the accuracy of a 3-2-1 spread, a 6-4-2 spread, a 5-3-1 spread, or a 4-2-0 spread.

Your two examples reduce to 2-1-0 and 3-1-0.

In my experience, the overall winner usually wins every race. So the accuracy comes into play only for second, third, etc. place.

So you've got two cars, one with 7 wins and 1 third place finish, and the other with 6 wins and 2 second place finishes. IMHO, you need to run a tie-breaker between these two cars. A point system with an even spread (2-1-0, 3-2-1, 5-3-1, whatever) will tell you that. A point system with an uneven spread will mislead you into thinking that you know which car is faster.

[Stan Pope]

In my mind, symmetry of PN and CPN charts dictates symmetry in scoring. The relative rewards for two cars in a heat should not be affected by the presence of faster cars or slower cars also on the track. The points difference between 1st and 2nd should be the same as the points difference between 2nd and 3rd, etc.

So, I think that Darin nailed it.

[MathGuy]

I will point out the ultimate point system. This system is beyond the computational abilities of the average derby committee, and definitely beyond explaination to a typical parent.

I will start with the simple question, does a victory over the 10th and 20th rank car be as impressive as a victory over the 4th and 5th fastest car? No. But they get the same points regardless.

A more dynamic scoring system, that takes into account strength of schedule, and give points based on the ultimate rank of the competitors. This would be done iteratively, and would require several iterations before the final results to converge.

The method is a statistical minimum bias procedure that is very similar to Sagrain ratings that go into the NCAA football BCS formulas. (Which are the best statistical method to evaluate a team based purely non-subjective final scores.)

Note that last year's flaw in the BCS, is that none of the non-subjective computer methods account for the fact Oklahoma lost it's last game. They only knew that Oklahoma lost one game, and crushed the rest of it's opponents.

I really don't expect anyone to do this. But it is the most accurate.


BTW, since we are approaching March Madness, I always look up Sagrain Power ratings in the USA Today website before I fill out my brackets.

[Stan Pope]

I will start with the simple question, does a victory over the 10th and 20th rank car be as impressive as a victory over the 4th and 5th fastest car? No. But they get the same points regardless.

When combined with balanced match-ups (which the NCAA doesn't accomplish, but PN and CPN do) the point system does produce meaningful comparisons ... and accurate results.

When the match-ups are not balanced (slightly unbalanced as for a PPN chart or wildly unbalanced as for NCAA games) then the rankings suffer and one must typically seek the top 6 to 10 point totals to give a high probability that the objectively top 4 are among the finalists.

[DMWOOD]

This year will be my first time running the race and I want to make things as fair as possible. It is nice to have a place to ask these questions before hand so you don't have to learn them the hard way.

I had already decided to run a CPN schedule for our small group and our software scores by points. Thanks guys for the explaination, now I understand which points are most fair. Our race is later this month so I am trying to think what I can before the race.

After the race I will be able to compare the total points and total times for each racer to see how the ranking compares. Will there be any benefit to this comparison?

[Stan Pope]

This year will be my first time running the race and I want to make things as fair as possible. It is nice to have a place to ask these questions before hand so you don't have to learn them the hard way.

Oops! Soapbox time!

Clearly distinguish in your mind between "fairness" and "accuracy".

You can have a very fair system of racing by deciding each heat by a coin flip. It won't be very "accurate", but it will be exquisitely "fair" (assuming no coin flip bias). When lane assignments are made arbitrarily by track staff, then fairness is in question. If you use a chart, and racer assignment into the chart is not determined by chance, then fairness is open to question.

"Fairness" is essential! Scouting ideals require it. "Accuracy" is very desirable.

I had already decided to run a CPN schedule for our small group and our software scores by points. Thanks guys for the explaination, now I understand which points are most fair. Our race is later this month so I am trying to think what I can before the race.

After the race I will be able to compare the total points and total times for each racer to see how the ranking compares. Will there be any benefit to this comparison?

Not all lane count - racer count combinations have PN or CPN charts of acceptable size. Be sure to produce charts ahead of time for the numbers of racers possible. Check to make sure that the charts meet your needs.

This is why, once the number of racers reaches a moderate size, you may be well served with a two stage method ... preliminaries that give high probability of including the 4 (or however many trophies you have) fastest cars among the finalists, and finals that can rank at least the top 4 finalists with high probability of accuracy.

On times, see Cory's comments elsewhere on lane differences... some good stuff there.

[MathGuy]

Yes the PPN, does an excellent job of equalizing the schedule. No doubt, and if you follow up with the top cars CPN, it is even better.

At our PWD, when we where setting up the night before, there was a paper output from a race 2 years ago with the our PWD stuff. It showed that the first place winner out of 20 kids had the 4th best average time. UNBELIEVABLE. I saw a result like this in a prior year (I wasn't running it), and thought it might have been an a quirk, but that quirk was minor compared to that.

In writing, this, I realized I still have that software on my laptop, and I used it to create a schedule for a 10 car, six round race on a 3 lane track. They added two dummy cars. I looked at a the standard race schedule, The #1 car faced:
#12 5x out 6 races (#12 is a dummy)
#11 1x out 6 races (#11 is a dummy)
#10 1x
#9 2x
#8 1x
#2 2x

Thus #1 raced against only 1 competitor in each six rounds, and faced only 4 cars. WOW. This software stinks. D$rbymast$r, not to name names. (older version).

This is an unbelieveably unbalanced schedule. We used a manual Tier or Sorting (see other post) method this year, and know I am even happier that we did.

Stan, I am so glad that you and Cory pushes the PPN and CPN methods.

[Stan Pope]

Thus #1 raced against only 1 competitor in each six rounds, and faced only 4 cars. WOW. This software stinks. D$rbymast$r, not to name names. (older version).

This is an unbelieveably unbalanced schedule. We used a manual Tier or Sorting (see other post) method this year, and know I am even happier that we did.

Unbelievable! Such ignorance should carry the ultimate penalty! Oops! That won't work ... they've already bred!

Until you establish the Tier accuracy parameters formally, consider applying it as a screening mechanism to put about 2T or 2.5T racers into finals of proven accuracy.

Stan, I am so glad that you and Cory pushes the PPN and CPN methods.

Have soapbox; Will rant!

[Cory]

A more dynamic scoring system, that takes into account strength of schedule, and give points based on the ultimate rank of the competitors. This would be done iteratively, and would require several iterations before the final results to converge.

I really don't expect anyone to do this. But it is the most accurate.

Am I mistaken in surmising that such a method could be implemented by any software package which does point scoring on final standing charts?

Something "crude" like the following, perhaps:

- Seed the algorithm with the raw point totals
- Multiple each point total by the average of its opponents' point totals
- Normalize by dividing by the average of all point totals
- Repeat until the delta is close to zero

I think this would converge pretty quickly. And it would get rid of ties.

[MathGuy]

[Something "crude" like the following, perhaps:

- Seed the algorithm with the raw point totals
- Multiple each point total by the average of its opponents' point totals
- Normalize by dividing by the average of all point totals
- Repeat until the delta is close to zero

I think this would converge pretty quickly. And it would get rid of ties.

Cory, that is exactly what I was describing. "Crude", but an effective model. (Generalized Linear Model)

Hmm. I am thinking if this is this really is the best approach.

[dknowles67]

This software stinks

If you are unhappy with the scheduling provided by that software, I posted a link to some FREE software that I developed using the Young & Pope PPN/CPN charting methods. The link/explanation is in the "Software" forum.
I would be happy if someone out there used it.
(I'm working on our Pack for next year)

[Cory]

Cory, that is exactly what I was describing. "Crude", but an effective model. (Generalized Linear Model)

I fiddled with it manually a little last night. It converged fast enough, but not quite the way I expected, and not in a useful way.

I will re-think the algorithm.

Hmm. I am thinking if this is this really is the best approach.

Are you thinking there's a better way or that there are issues with this way?

It occurred to me that for two cars with the same "raw" point total, the one with slightly more difficult schedule would come out ahead, even though it might be objectively slower. Perhaps cases like this are rare, but it still seems more "just" to let them break the tie head to head.

Thanks.

[MathGuy]

Cory:

It occurred to me that for two cars with the same "raw" point total, the one with slightly more difficult schedule would come out ahead, even though it might be objectively slower. Perhaps cases like this are rare, but it still seems more "just" to let them break the tie head to head.

Yes, this is what I was thinking.

The algorithm could also take a few different approaches. But the variables that I ussually study are for multiplicative or additive models. But this is not the case here. For an individual race.

Prob of x beating y = {0-1}, but most likely (0 or 1). This is really looks more like a Beta distribution with a U shape probability function to be maximized. (Add Lane bias, and the function gets even more complicated.)

I am sure there is an answer, but not as straight forward as I had though. I am going to guess a ELO Chess ranking system is going to work better. I haven't done that sort of calc before, but I am sure that it is something that is better suited for the job.

[Stan Pope]

Am I mistaken in surmising that such a method could be implemented by any software package which does point scoring on final standing charts?

Something "crude" like the following, perhaps:

- Seed the algorithm with the raw point totals
- Multiple each point total by the average of its opponents' point totals
- Normalize by dividing by the average of all point totals
- Repeat until the delta is close to zero

I think this would converge pretty quickly. And it would get rid of ties.

Just to see what Cory's "not very useful result' was, I plugged my Race of Champions results into tha algorithm. It was a 5-car PN on 4 lanes. (Values were truncated to 3 decimal places for display)

Sum 12.000 4.000 9.000 13.000 12.000
Adj 11.400 4.600 9.225 12.025 11.400
Adj 10.910 5.206 9.344 11.315 10.910
Adj 10.518 5.797 9.391 10.788 10.518
Adj 10.206 6.353 9.394 10.391 10.206
Adj 9.960 6.857 9.373 10.089 9.960
..
..
..
Adj 9.143 9.141 9.143 9.143 9.143
Adj 9.143 9.141 9.143 9.143 9.143
Adj 9.143 9.142 9.143 9.143 9.143

So, I looked at the algorithm again, and, sure enough ... I missed the fact that each iteration moves each total toward the central value.

Iteration doesn't really fit the problem, but the first adjustment does compensate for strength of schedule.

When the system is balanced, the adjustment does not resolve final standing ties, at least in this example.