RPI vs KenPom

[Jamar Howard 4 President]

Did you have a set criteria or just kindof eyeball it? This is an interesting exercise. Difficult to take in all the data, especially without knowing your cutoffs.

My thought process was to look at the 2/19 ratings and see what conclusions would be drawn based on each current ranking.

For example, North Dakota is currently 275 in RPI and 311 in KenPom. On 12/29, those rankings were 185 and 319. If we decide to trust the current RPI (275) as the "correct" number, then KenPom wins the battle of "better ranking on 12/29". His 12/29 rank was 44 spots away from 275, while the RPI's rank was 90 spots away from 275.

If we start to learn more toward KenPom and use 311 instead of 275, the comparison moves even further in favor of KenPom.

However you feel about North Dakota, the conclusion is the same. KenPom did better on 12/29.

This method is how I chose green and red. Anyone is free to challenge any of my color coding with a specific challenge. However, the count is currently 24 green, 7 red. I feel confident that even if we modified my choices slightly, KenPom wins in the end by a large margin.

[wu_shizzle]

1) Was anyone saying that RPI was the best tool earlier in the year?

2) Are you looking for the best predictive algorithm? At best you can get a good feel for how good a team has been, but it's another to predict how good a team will be. That's a lot to ask of math.

[Jamar Howard 4 President]

1) Was anyone saying that RPI was the best tool earlier in the year?

ripemupshocks pretty much poo-pooed the idea that we should ignore the RPI early and then feel free to use it later in the year.

I agree, the RPI is not a good tool in December.

ripemupshocks' response to my post:

I don't understand how waiting until January 1 makes the RPI any more/less credible than any other arbitrary date you may choose; other than waiting until January when conference play starts, ensures that the BCS teams that are just handed great RPI numbers prior to the start of the season do not have to worry about losses to low RPI teams in the non-con portion of their schedule having a negative impact on their RPI at the end of the season, thereby assuring their status as golden boys in the eyes of the NCAA Selection Committee. If the RPI numbers were static, I could see your argument. But they're not. The RPI numbers change with every game, even in December.

It's far to easy to say the RPI doesn't matter in December when a good non-con schedule gives WSU a good RPI number, but then say the RPI does matter in January when the MVC is dragging down WSU's RPI.

I have also seen many conversations where people effectly say "I know the RPI isn't very good in December", only to follow up that statement with "but take a look at so-and-so that is in the RPI top 50 on such-and-such day in December". I thought this thread would be helpful to emphasize just how bad the RPI is early in the year, even as late as the end of December.

[Jamar Howard 4 President]

	Dec	29		Feb	19
Team	RPI	KenPom		RPI	KenPom
Incarnate Word	40	174		156	197
Gardner Webb	63	167		162	211
McNeese State	144	302		281	269
New Orleans	158	305		328	310
Fairleigh Dickinson	168	294		308	323
North Dakota	185	319		275	311
Southern Illinois	321	202		280	230

This is a reduced table showing just the teams who were originally 100+ spots apart in the 12/29 rankings. These are the cases where RPI and KenPom were truly at a major disagreement. Notice the complete lack of a single example of RPI having been the better ranking.

[Jamar Howard 4 President]

2) Are you looking for the best predictive algorithm? At best you can get a good feel for how good a team has been, but it's another to predict how good a team will be. That's a lot to ask of math.

I'm just trying to look at which systems do a good job ranking teams with relatively few games of sample size (KenPom does well here) vs which rankings are very poor until a bunch of games have been played (this describes the RPI).

Take Incarnate Word for example. When Nebraska lost to them back in December, I remember a lot of posters trying to look on the bright side by mentioning that Incarnate Word might be good. WSU was looking at a possible matchup with Nebraska in Hawaii, and Shocker fans wanted Nebraska to be a quality opponent. I remember hearing "yes it is early, but Incarnate Word has a top 50 RPI". In reality, KenPom was very clear back then that Incarnate Word was a bad team. His rankings had them as a borderline top 200 team, not a borderline top 50 team like the RPI showed. I hope this thread helps people see why "ignore the RPI in December" literally means "IGNORE THE RPI IN DECEMBER!"

[Jamar Howard 4 President]

Please folks... no more of this:

As was mentioned in the other thread, Incarnate Word had an RPI in the 40's. So it might not be too bad a loss.

[wufan]

Regarding VCU, I agree their injury situation makes them a unique situation. Throw them out if you like. However, if we are going to make a judgement at all, here is why I say that KenPom was clearly better.

12/29, VCU 3, KenPom 14
02/19, VCU 13, KenPom 26

If you believe VCU is the #13 team in the country, then KenPom was much more accurate on 12/29.
If you believe VCU is the #20 team in the country, then KenPom was much more accurate on 12/29.
If you believe VCU is the #26 team in the country, then KenPom was much more accurate on 12/29.

Any reasonable rank of VCU today leads to the conclusion that KenPom's rank of 14 was more accurate than the RPI's rank of 3.

Um RPI shifted less. As I've mentioned before, you are totally eyeballing this with no criteria or standard. What makes one superior to the other? Less shift? Closest to some standard at the end of the season?

[wufan]

How is Wofford debatable? It shifted one spot in the RPI and 11 in KP.

[Jamar Howard 4 President]

Wufan, you are missing a simple point. The fact that RPI shifted less has no bearing on my point. I am not arguing which system is better NOW. I'm arguing which system was better THEN.

KenPom and RPI give a range of current rankings. For VCU, it is 13-26. No matter where in the range you feel VCU's "real ranking" should be, the number you choose is closer to KenPom's ranking on 12/29 than it is to the RPI's ranking on 12/29.

The current RPI rank of 13 may be the better rank. The current KenPom rank of 26 may be the better rank. Either way, it doesn't matter. Both current ranks, and everything in between, are closer to 14 than they are to 3.

Heck, we could expand further. VCU is currently 25 in Sagarin. 25 in the AP poll. 26 in the coaches poll. Every single one of these rankings also agrees that KenPom did a better job on 12/29 than RPI did.

[Jamar Howard 4 President]

How is Wofford debatable? It shifted one spot in the RPI and 11 in KP.

Wofford is virtually a stalemate. Basically hovering around 40 in RPI and around 90 in KenPom. Which ranking is more accurate is unclear. Both rankings have stayed very stagnant considering 2 months have passed.

Oh, and by the way, Wofford is 103 in Sagarin's ratings. That's one possible tie breaker that definitely favors what KenPom is saying.

[The Mad Hatter]

Wufan, you are missing a simple point. The fact that RPI shifted less has no bearing on my point. I am not arguing which system is better NOW. I'm arguing which system was better THEN.

KenPom and RPI give a range of current rankings. For VCU, it is 13-26. No matter where in the range you feel VCU's "real ranking" should be, the number you choose is closer to KenPom's ranking on 12/29 than it is to the RPI's ranking on 12/29.

The current RPI rank of 13 may be the better rank. The current KenPom rank of 26 may be the better rank. Either way, it doesn't matter. Both current ranks, and everything in between, are closer to 14 than they are to 3.

Heck, we could expand further. VCU is currently 25 in Sagarin. 25 in the AP poll. 26 in the coaches poll. Every single one of these rankings also agrees that KenPom did a better job on 12/29 than RPI did.

If your point had been that Kenpom's numbers become accurate earlier in the season than RPI's, a straight up comparison of how much each systems numbers shifted over time would accomplish that, but your opening sentence clearly rejects that as the aim of the exercise.

As you have stated your argument, it is a mere tautology. You are saying that Kenpom is better because it is more consistent with Kenpom. You are just asserting past Kenpom as accurate without any support for that notion.



If you were to compare both Kenpom and RPI to a third ranking system (such as Massey Composite) as you note you potentially could (but haven't), that would be an improvement because it would provide an objective frame of reference, which the current format lacks entirely. Even then, however, you would not have proven the superiority of the ranking system, because it would depend on asserting the 3rd option as superior (but at least it wouldn't be circular logic).

Any real defense of Kenpom being better than RPI would be based on an explanation of why the formula of Kenpom evaluates more relevant data in more sophisticated ways (which I believe it clearly does), but these comparisons don't do that.

[Jamar Howard 4 President]

You are saying that Kenpom is better because it is more consistent with Kenpom. You are just asserting past Kenpom as accurate without any support for that notion.

That is simply just not true. I said the December version of KenPom is more consistent with current KenPom, current RPI, and EVERYTHING IN BETWEEN!

If you are going to debate me, please argue what I said, not some twisted made up version of it.

[The Mad Hatter]

Not true. I said the December version of KenPom is more consistent with the entire range of current rankings spanning from current KenPom to current RPI. Why are you completely ignoring half my argument?

I have no idea what you mean by the phrase "from current KenPom to current RPI," and since the only data you have provided is from KenPom and RPI, I don't see how I am ignoring anything. How am I expected to interpret this phrase to mean rankings in addition to KenPom and RPI when you haven't listed any other ranking system as the basis of your comparison, in spite of being specifically asked about your methodology by numerous posters? Since you haven't specified any external rankings you are using for comparison nor given the data from said rankings, it would seem that you are the one ignoring that part of your argument. If you would like to provide the external comparison to other ranking systems, I'd be happy to see it.

[The Mad Hatter]

That is simply just not true. I said the December version of KenPom is more consistent with current KenPom, current RPI, and EVERYTHING IN BETWEEN!

If you are going to debate me, please argue what I said, not some twisted made up version of it.

Once again, what is everything in between and why is that data excluded from your comparison posts? If you don't like people misrepresenting your arguments, it would help if you provided all the data/methodology your are using rather than posting color coded results that are based on reference points that exist only in your head.

[The Mad Hatter]

What I mean by saying KenPom "was better" with a specific team is that KenPom's ranking stayed fairly stable while the RPI ranking moved significantly in KenPom's direction. The reverse would be true for NJIT, the only team on my list where the RPI seems to have had the more accurate ranking 4 weeks ago.

I'll update this again later in the season, but I think we can already see clearly why I always say that KenPom is significantly better than RPI early in the season.

Yes, I can already hear the complaints about my analysis of "better". I admit this isn't perfect, but I ask any skeptics to get past the small specifics and look at the bigger picture. There are numerous teams like North Dakota, where RPI changed by nearly 100 spots and KenPom stayed almost exactly the same. There are very few major examples of the opposite happening in the RPI's favor. Sure, we could argue if I should have made Creighton red instead of leaving them neutral. If someone thinks their "true ranking" should be around 150, then yes, RPI has been the better judge of Creighton so far.

My point is simply this. We can argue about any one team, but the sum of the parts gives a clear trend. In general, over the past 4 weeks, RPI rankings have moved toward KenPom much more often than the other way around. KenPom has stayed steady much more consistently than RPI, which has changed significantly in just 4 weeks time. I think this is why we can easily crown KenPom champ.

Here is your own post explaining your methodology, would you mind showing me where you mention comparing RPI and KenPom to external ranking systems or how I misrepresented your argument. I said it was a tautology because you are calling KenPom good for staying consistent with KenPom. That is exactly what your own posts are arguing for here.

Actually, your methodology is exactly the thing you say later is not your methodology (comparing amount of movement within each ranking system). How am I supposed to treat your methodology fairly, when you just keep changing what it is every time someone raises an objection?