Historical RPI vs KenPom on Selection Sunday

[ShockTalk]

OK, I'm being dense about this. Just what again is the "%" column suppose to represent?

If a team is listed #1 in one, but #2 in the other, they are 1 spot off. If a team is #99 in one, but #100 in the other, they are also 1 spot off. What difference does it make whether the team being looked at is near the top or near the bottom as far as the %?

[WUpigsooie]

Eh, after assembling some data my hypothesis was flimsy at best:

A good team (top 25 kenpom) and an average team (50-100) are affected by the strength of the conference differently. A good team will rack up wins almost regardless of schedule, therefore being in a good conference will bolster RPI by having those wins be against better competition. An average team's record will depend greatly on their competition, so an average team could be bolstered by a bad conference.

You can see this in the data above. in 2007 the Valley was a good league, the top teams (SIU, Creighton) had better RPIs than their KenPoms, whereas the lower ones (WSU, UNI) were worse.
Then in 2015 when the Valley was bad except UNI and WSU, their RPIs were worse than their KenPom.


Turns out its probably not that clear, mostly a random relation. Better teams (by KenPom) do tend to have much smaller differences in either direction for whatever thats worth

[Jamar Howard 4 President]

OK, I'm being dense about this. Just what again is the "%" column suppose to represent?

If a team is listed #1 in one, but #2 in the other, they are 1 spot off. If a team is #99 in one, but #100 in the other, they are also 1 spot off. What difference does it make whether the team being looked at is near the top or near the bottom as far as the %?

Ranks 81 & 100 are 19 spots apart, but appear fairly close. If the RPI said 81 and KP said 100, the rankings would be considered generally in agreement. Not a perfect match, but in the same ballpark. Percentage wise, 81 is 81% of 100.

Ranks 1 & 20 are also 19 spots apart, but appear to be wildly different measures of a team. If RPI said 1 and KP said 20, people would rightfully say that one of the rankings must be way off. 1 is only 5% of 20.

The closer you get to rank #1, the more important small differences in ranks become.

I think percentage difference helps a bit, but tends to overcompensate and has its own problems. Looking at both the pure difference in ranks and the % difference can be helpful.

Does this make more sense?

[proshox]

Should total number of teams within the dataset be accounted for? My first thought was to reverse the system so that the best team had the highest rank instead of the lowest.. The number one team would be recalibrated to be 350 (or whatever the total number of division one teams was for the year). Maybe it is all relative, but my hunch is that the difference in ranking is much smaller in scale then the first post suggests. I need to break out excel.

Feel free to point out why I am completely stupid with this post. I will not take it personally.

[Keyser Soze]

Should total number of teams within the dataset be accounted for? My first thought was to reverse the system so that the best team had the highest rank instead of the lowest.. The number one team would be recalibrated to be 350 (or whatever the total number of division one teams was for the year). Maybe it is all relative, but my hunch is that the difference in ranking is much smaller in scale then the first post suggests. I need to break out excel.

Feel free to point out why I am completely stupid with this post. I will not take it personally.

Idiot

[ShockTalk]

Ranks 81 & 100 are 19 spots apart, but appear fairly close. If the RPI said 81 and KP said 100, the rankings would be considered generally in agreement. Not a perfect match, but in the same ballpark. Percentage wise, 81 is 81% of 100.

Ranks 1 & 20 are also 19 spots apart, but appear to be wildly different measures of a team. If RPI said 1 and KP said 20, people would rightfully say that one of the rankings must be way off. 1 is only 5% of 20.

The closer you get to rank #1, the more important small differences in ranks become.

I think percentage difference helps a bit, but tends to overcompensate and has its own problems. Looking at both the pure difference in ranks and the % difference can be helpful.

Does this make more sense?

Yes......but, one at 4 and one at 5 does not seem far apart yet the "%" is 20 as compared to 19 and 20 which is 5%. Both appear as a toss up, so I agree with the overcompensation in certain cases as well. Thanks.

[ShockTalk]

Idiot

Why is it that when Keyser Soze calls someone "Idiot" I feel they should be afraid.....very afraid. :hororr:

[proshox]

Why is it that when Keyser Soze calls someone "Idiot" I feel they should be afraid.....very afraid. :hororr:

Not afraid. Not afraid. Not afraid.

Do you believe me?

[Cdizzle]

Interestingly, WSU has 1 more loss than KU, both teams are halfway through their league schedules, and WSU is rated higher than KU by kenPom.

I guess quality losses really are the way to go. Hopefully the committee uses the same rules.

[Good News]

The average distance isn't that telling. Calculate the average "error" per team.

Error = (RPI - KenPom) / KenPom

Then you can average that over the whole set.

I don't think it's particularly meaningful - either relatively or nominally - but why wouldn't you use the midpoint to calculate the error or percent variance? Dividing by KenPom only tells you how far off it is from KenPom, not how far apart they are, which is what I thought point of the exercise was.

[wufan]

I don't think it's particularly meaningful - either relatively or nominally - but why wouldn't you use the midpoint to calculate the error or percent variance? Dividing by KenPom only tells you how far off it is from KenPom, not how far apart they are, which is what I thought point of the exercise was.

What you are describing I believe is defined as percent difference (x-y/mean x, y). I believe what was performed was %error with KenPom being the standard by which the accuracy of the RPI is judged. In either case the % calculated for the mean of KenPom and RPI is incorrect. As such, I can easily conclude that the statistical model utilized in the OP is poor, however the conclusion seems just as valid.

[Good News]

Not related to your question, but man did we have a snootfull of good RPI's in 2006 or what? Wasn't that back in the day when the whiney P5 conferences accused the Valley of "gaming" the system?

One of my favorite seasons ever, definitely top 5. Shocks win the Valley with a MVC player of the year. The Valley had a conference RPI rank of 5 or 6 and ahead of the Pac-10. Six Valley teams with top 50 RPI rankings and at-large resumes, 5 deserved to be in. Four actually got in, and the other one had the best ever RPI to not get in (btw JH4P I always thought that number was 21).

After seeing 4 Valley teams in, Billy Packer flips out on national television, specifically referencing Bradley and the Valley and openly challenging the head of the NCAA selection process in an interview.

Then the Shocks lay a beatdown on a Big East team in a 7-10 game and beat a 2 seed SEC team to advance to the second weekend. Bradley beats Kansas to advance as well, making Billy Packer look like a horse's arse and paving the way for his exit from calling college basketball games.

And I destroyed 2 different office pools for about 800 bucks, but that wasn't as much fun as watching the Valley up and Kansas and Packer down.

Good times.

[Kung Wu]

I don't think it's particularly meaningful - either relatively or nominally - but why wouldn't you use the midpoint to calculate the error or percent variance? Dividing by KenPom only tells you how far off it is from KenPom, not how far apart they are, which is what I thought point of the exercise was.

No, for my purpose I wanted to assume KenPom was the "correct" answer and I wanted to see the deviation from that. That doesn't mean KenPom IS correct, but that's the test I was wanting to see.

[Good News]

No, for my purpose I wanted to assume KenPom was the "correct" answer and I wanted to see the deviation from that. That doesn't mean KenPom IS correct, but that's the test I was wanting to see.

Gotcha. That was exactly my point and question - I happen to think KenPom is the best in the biz and the most accurate but I thought the exercise was in a different context.