RPI vs KenPom

[wufan]

Wufan, you are missing a simple point. The fact that RPI shifted less has no bearing on my point. I am not arguing which system is better NOW. I'm arguing which system was better THEN.

KenPom and RPI give a range of current rankings. For VCU, it is 13-26. No matter where in the range you feel VCU's "real ranking" should be, the number you choose is closer to KenPom's ranking on 12/29 than it is to the RPI's ranking on 12/29.

The current RPI rank of 13 may be the better rank. The current KenPom rank of 26 may be the better rank. Either way, it doesn't matter. Both current ranks, and everything in between, are closer to 14 than they are to 3.

Heck, we could expand further. VCU is currently 25 in Sagarin. 25 in the AP poll. 26 in the coaches poll. Every single one of these rankings also agrees that KenPom did a better job on 12/29 than RPI did.

Here's a quote from you earlier in the thread:

"What I mean by saying KenPom "was better" with a specific team is that KenPom's ranking stayed fairly stable while the RPI ranking moved significantly in KenPom's direction."

Then you go on to ignore the fact that Kenpom shifted further away from its original ranking. As pointed out by TMH, the shift certainly isn't enough to show superiority, but at least it's something that could be followed.

IMO, comparing highly ranked teams in December to their eventual NCAA seed offers some insight, not only on accuracy, but on number of outliers. What you're doing looks like a preconceived notion and eyeballing the results to prove your own theory.

[Jamar Howard 4 President]

13 is closer to 14 than to 3
14 is closer to 14 than to 3
15 is closer to 14 than to 3
16 is closer to 14 than to 3
17 is closer to 14 than to 3
18 is closer to 14 than to 3
19 is closer to 14 than to 3
20 is closer to 14 than to 3
21 is closer to 14 than to 3
22 is closer to 14 than to 3
23 is closer to 14 than to 3
24 is closer to 14 than to 3
25 is closer to 14 than to 3
26 is closer to 14 than to 3

Can we agree that those 14 statements are all factually correct?

[wufan]

13 is closer to 14 than to 3
14 is closer to 14 than to 3
15 is closer to 14 than to 3
16 is closer to 14 than to 3
17 is closer to 14 than to 3
18 is closer to 14 than to 3
19 is closer to 14 than to 3
20 is closer to 14 than to 3
21 is closer to 14 than to 3
22 is closer to 14 than to 3
23 is closer to 14 than to 3
24 is closer to 14 than to 3
25 is closer to 14 than to 3
26 is closer to 14 than to 3

Can we agree that those 14 statements are all factually correct?

Yes, but 13 is closer to 3 than 14 is to 26. If your argument is that Kenpom was then was closer to RPI now, then while factually correct, it is doesn't prove your point. Perhaps I'm getting ahead of myself, so please continue.

[Jamar Howard 4 President]

Yes

So in regards to VCU:
13 (current RPI) is closer to 14 (KenPom's 12/29 rank) than to 3 (RPI's 12/29 rank).
26 (current KenPom) is closer to 14 than to 3.
Every value in between 13 and 26 (14, 15, 16, etc.) is closer to 14 than to 3.

So if you feel that the current RPI is the "true rank" for VCU, then KenPom proves to have been the better December ranking.
If you feel that the current KenPom rank is the "true rank" for VCU, then KenPom proves to have been the better December ranking.
If you feel that the "true rank" for VCU is somewhere in between what KenPom and RPI currently say, then KenPom proves to be the better December ranking.

Can you now agree that KenPom's December rank of VCU at 14 was a better ranking than the RPI's December rank of 3?

[Rocky Mountain Shock]

This thread has become the SOS thread's fat ugly red-headed step sister

[wufan]

So in regards to VCU:
13 (current RPI) is closer to 14 (KenPom's 12/29 rank) than to 3 (RPI's 12/29 rank).
26 (current KenPom) is closer to 14 than to 3.
Every value in between 13 and 26 (14, 15, 16, etc.) is closer to 14 than to 3.

So if you feel that the current RPI is the "true rank" for VCU, then KenPom proves to have been the better December ranking.
If you feel that the current KenPom rank is the "true rank" for VCU, then KenPom proves to have been the better December ranking.
If you feel that the "true rank" for VCU is somewhere in between what KenPom and RPI currently say, then KenPom proves to be the better December ranking.

Can you now agree that KenPom's December rank of VCU at 14 was a better ranking than the RPI's December rank of 3?

No. We need an independent standard to rank against. All that has been shown is that both rankings adjusted roughly the same amount. As an example what if their "true" ranking should be #5. At that point RPI is clearly superior.

Without a true standard (which I agree to be the case currently) the only thing left to judge is degree of change, which again shows nothing. My argument isn't that one is better than the other, but rather that we can't tell based on the data submitted and to present one as being better than other has no basis.

[Cdizzle]

No. We need an independent standard to rank against. All that has been shown is that both rankings adjusted roughly the same amount. As an example what if their "true" ranking should be #5. At that point RPI is clearly superior.

Without a true standard (which I agree to be the case currently) the only thing left to judge is degree of change, which again shows nothing. My argument isn't that one is better than the other, but rather that we can't tell based on the data submitted and to present one as being better than other has no basis.

This is where I've been hung up as well. I get the comparisons being made, and find that I generally agree with the conclusion, but without an independent standard to compare the two ratings to, it does get pretty hand-wavy.

[ShockerExpress]

13 is closer to 14 than to 3
14 is closer to 14 than to 3
15 is closer to 14 than to 3
16 is closer to 14 than to 3
17 is closer to 14 than to 3
18 is closer to 14 than to 3
19 is closer to 14 than to 3
20 is closer to 14 than to 3
21 is closer to 14 than to 3
22 is closer to 14 than to 3
23 is closer to 14 than to 3
24 is closer to 14 than to 3
25 is closer to 14 than to 3
26 is closer to 14 than to 3

Can we agree that those 14 statements are all factually correct?

Oh my god

[Shocker-maniac]

This is killing me. Being back the SOS thread.

[Jamar Howard 4 President]

As an example what if their "true" ranking should be #5. At that point RPI is clearly superior.

Of course a "true ranking" of 5 for VCU is possible, but it is highly unlikely. You are talking about an extreme outlier from what the computer formulas say.

Let me do a little quick research. I have a very simple counter point but want to get my numbers straight first.

[ShockerInKC]

This is where I've been hung up as well. I get the comparisons being made, and find that I generally agree with the conclusion, but without an independent standard to compare the two ratings to, it does get pretty hand-wavy.

I agree. If you look at the limits, say kenpom rating is based on an alphabetic score based on the team's name. Then kenpom will be perfectly consistent when compared against any other measure based on team performance. So you need a calibrated rank to compare against and there is none. Although an average rank of multiple measures could get close, you still have a problem.

[Jamar Howard 4 President]

Let’s use the selection committee as our “true” ranking provider. Of course they don’t do a perfect job, but they can act as a useful 3rd source of rankings that you have been asking for.

Last year, looking at the top 8 seeds (32 teams total), 21 teams were seeded within the window that KenPom and RPI created.

For example, BYU was:
#32 in RPI, #50 in KenPom
They received a 10 seed (which corresponds to teams 37-40)
The selection committee placed them inside the window that RPI and KenPom had created.

Furthermore, 26 of the 32 teams were within 1 seed line (maximum 4 spots in the rankings) away from the "window" created by RPI and KenPom.

That means 66% of teams were inside the window, and 81% of teams were within 4 spots of being in the window.

It just so happens that of the remaining 6 teams, 3 were significantly outside the window on the RPI side, 3 were outside the window on the KenPom side.

That means that 3 out of 32 teams fit your VCU example where they were significantly outside the window and favoring the RPI. That's 9%.

Sure I could go back to 2013, 2012, etc. and gain more data. Maybe your VCU outlier happens 7% of the time. Maybe it happens 10%. The point is, 90% of the time, give or take, the selection committee's version of the "true ranking" falls within just a few spots of the window created by RPI and KenPom.

So go ahead, tweak 10% of my green and red shaded teams. At last count, there were 24 green and 7 red. Is making it 22 green and 9 red going to change anything? Nope. The point is that outliers are just that... outliers. Most of the time a team's true rank will fall within the RPI / KenPom window, and most of the time, KenPom's ranking back in December will be looked on more favorably than KenPom's.

[Jamar Howard 4 President]

One more question for those who don’t like my logic. What do you think about Incarnate Word?

	Dec	29		Feb	19
Team	RPI	KenPom		RPI	KenPom
Incarnate Word	40	174		156	197

If their true rank is 197, then KenPom dominated in December.
If their true rank is 156, then KenPom dominated in December.
If their true rank is 131 (25 spots outside the RPI/KenPom window), then KenPom STILL dominated in December.

Can we at least agree on this one?

[wufan]

One more question for those who don’t like my logic. What do you think about Incarnate Word?

	Dec	29		Feb	19
Team	RPI	KenPom		RPI	KenPom
Incarnate Word	40	174		156	197

If their true rank is 197, then KenPom dominated in December.
If their true rank is 156, then KenPom dominated in December.
If their true rank is 131 (25 spots outside the RPI/KenPom window), then KenPom STILL dominated in December.

Can we at least agree on this one?

I would agree to that, but the point is that here is no true rank. I pulled 5 out of thin air as an example. I don't believe that is truly VCUs true rank, merely that without a true rank/standard, you have no basis for comparison.

I think the idea behind the ranks is to ultimately determine who will make the dance and at what seed. The only interesting value to this discussion is how accurate can the ranking systems in December predict the outcome of the seeding in March?

Thus far I've seen nothing that can support that either is accurate, but come selection Sunday, I would like to know where my outliers ultimately are and which one supplied the best data based on % correct and accuracy to seed.

[Jamar Howard 4 President]

wufan, did you completely skip my post about the selection committee? If you are so interested in "determining who will get what seed", then my post showing how the selection committee lines up with the KenPom/RPI window should have been just what you were looking for.

66% of the time the committee seeds teams inside the KenPom/RPI window.
81% of the time the committee seeds teams within 1 seed line of the window.
91% of the time the committee seeds teams within 1 seed line of the window, or they seed further away but in the direction that favors KenPom.

If that doesn't give you the proof you are searching for, I don't know what would. The window between KenPom's current rank and the RPI's current rank is a very good indicator of a team's "true rank".