RPI vs KenPom

[Jamar Howard 4 President]

Team	RPI	KenPom	AP	Coaches
VCU	3	14	27	26
Wichita State	8	13	16	16
Northern Iowa	10	26	23	23
Old Dominion	12	62	30	33
George Washington	17	38	31	34
Colorado State	19	55	24	24
LSU	20	54	30	35
Buffalo	21	82	-	-
Louisville	26	6	5	6
Texas	30	10	11	10
Oklahoma	36	12	18	21
Georgia Tech	38	89	-	-
Wofford	39	88	-	-
Incarnate Word	40	174	-	-
Penn State	41	84	33	42
Eastern Washington	47	104	-	-
High Point	58	148	-	-
Stony Brook	59	150	-	-
Minnesota	62	27	36	31
Gardner Webb	63	167	-	-
Ohio State	65	11	20	18
St. Francis PA	67	143	-	-
Lafayette	68	142	-	-
UConn	80	32	-	38
Syracuse	81	31	-	-
Texas Southern	91	178	-	-
Florida	94	14	36	-
Radford	97	173	-	-
Notre Dame	98	23	13	14
Texas Arlington	99	188	-	-
Indiana	104	44	35	37
Creighton	126	76	-	-
Wyoming	143	56	-	-
McNeese State	144	302	-	-
New Orleans	158	305	-	-
Fairleigh Dickinson	168	294	-	-
Delaware State	170	248	-	-
North Dakota	185	319	-	-
NJIT	197	258	-	-
Southern Illinois	321	202	-	-

As of 12/29, the above comparison shows teams where the RPI and KenPom disagree significantly. AP and Coaches polls have been added as an additional reference point. Let's track these teams as the season progresses and see how things shake out. I believe KenPom will be shown to be vastly superior.

Note: I may add to this table throughout today if I find more good teams for this comparison. After today, I'll leave it alone and we can see how things play out.

[MadaboutWu]

Want to add WSU as a point of interest (even if ours is closely aligned).

My question is what counts as superior? I assume which rating system's final evaluation matches the earlier ones more closely?

[Jamar Howard 4 President]

Want to add WSU as a point of interest (even if ours is closely aligned).

My question is what counts as superior? I assume which rating system's final evaluation matches the earlier ones more closely?

I went back and added WSU to the original post. Good suggestion.

I would say superior could be determined a couple ways. One of my expectations is that the computers (RPI & KenPom) will spit out numbers in March that are fairly close to each other, and that KenPom today will be closer to that future rank than the RPI of today.

[OregonShocker]

Team	RPI	KenPom	AP	Coaches
VCU	3	14	27	26
Wichita State	8	13	16	16
Northern Iowa	10	26	23	23
Old Dominion	12	62	30	33
George Washington	17	38	31	34
Colorado State	19	55	24	24
LSU	20	54	30	35
Buffalo	21	82	-	-
Louisville	26	6	5	6
Texas	30	10	11	10
Oklahoma	36	12	18	21
Georgia Tech	38	89	-	-
Wofford	39	88	-	-
Incarnate Word	40	174	-	-
Penn State	41	84	33	42
Eastern Washington	47	104	-	-
High Point	58	148	-	-
Stony Brook	59	150	-	-
Minnesota	62	27	36	31
Gardner Webb	63	167	-	-
Ohio State	65	11	20	18
St. Francis PA	67	143	-	-
Lafayette	68	142	-	-
UConn	80	32	-	38
Syracuse	81	31	-	-
Texas Southern	91	178	-	-
Radford	97	173	-	-
Notre Dame	98	23	13	14
Texas Arlington	99	188	-	-
Indiana	104	44	35	37
Creighton	126	76	-	-
Wyoming	143	56	-	-
McNeese State	144	302	-	-
New Orleans	158	305	-	-
Fairleigh Dickinson	168	294	-	-
Delaware State	170	248	-	-
North Dakota	185	319	-	-
NJIT	197	258	-	-
Southern Illinois	321	202	-	-

As of 12/29, the above comparison shows teams where the RPI and KenPom disagree significantly. AP and Coaches polls have been added as an additional reference point. Let's track these teams as the season progresses and see how things shake out. I believe KenPom will be shown to be vastly superior.

Note: I may add to this table throughout today if I find more good teams for this comparison. After today, I'll leave it alone and we can see how things play out.

It's CLEAR that this means the Shockers are... (talk amongst yourselves)

[wufan]

I went back and added WSU to the original post. Good suggestion.

I would say superior could be determined a couple ways. One of my expectations is that the computers (RPI & KenPom) will spit out numbers in March that are fairly close to each other, and that KenPom today will be closer to that future rank than the RPI of today.

Why not just look at a few years prior for comparison?

[ShockTalk]

I went back and added WSU to the original post. Good suggestion.

I would say superior could be determined a couple ways. One of my expectations is that the computers (RPI & KenPom) will spit out numbers in March that are fairly close to each other, and that KenPom today will be closer to that future rank than the RPI of today.

As a point of reference for me, maybe others, what are some of the major differences/factors between RPI and KemPom, and RPIForcasting and KemPom?

[Jamar Howard 4 President]

Why not just look at a few years prior for comparison?

wufan, I'm tired and may be completely overlooking something obvious, but my point with this thread was:

1 - Show how RPI and KenPom converge (partially, not fully) over the course of the year.
2 - Show that RPI tends to converge toward KenPom more than vice versa, showing that KenPom is much more useful early in the year.

How would we do those two things using previous years? The only data available (that I know of) for previous years is final, end of year data only. There is nothing from mid-year to compare to.

Like I said, I'm tired and may be missing something obvious.

[Jamar Howard 4 President]

As a point of reference for me, maybe others, what are some of the major differences/factors between RPI and KemPom, and RPIForcasting and KemPom?

RPI exclusively looks at W/L's and SOS. Margin of victory is not a factor at all.

KenPom looks at detailed stats (offensive and defensive efficiency, margin of victory, etc.) and ignores W/L's.

RPI Forecast simulates the games that have not yet been played and factors them into the current RPI to predict the end of season RPI in advance. There are some details of this process that I'm not real familiar with.

My main point of this thread is to show how the RPI gets fooled quite often by a small sample size, while KenPom evaluates significant amounts of detailed data from each game, allowing it to produce better results from fewer games.

For example, Louisville and Texas are both outside the top 25 in the RPI right now because they have both played a number of cupcakes that have hurt their SOS. The RPI just sees "easy opponent, win". "tough opponent, loss". It does not see what real people watching the games see. "Easy opponent, dominant win". "Tough opponent, competitive loss". Both UL and UT played close games against Kentucky. Real people watching those games saw Texas and Louisville play competitively with an excellent Kentucky team. Real people can then conclude that UL and UT are very good. KenPom is a computer program, but it is able to pick up on these things in ways that the RPI cannot. Its formula is 100x more complex than the RPI, and the result is significantly better results.

[wufan]

wufan, I'm tired and may be completely overlooking something obvious, but my point with this thread was:

1 - Show how RPI and KenPom converge (partially, not fully) over the course of the year.
2 - Show that RPI tends to converge toward KenPom more than vice versa, showing that KenPom is much more useful early in the year.

How would we do those two things using previous years? The only data available (that I know of) for previous years is final, end of year data only. There is nothing from mid-year to compare to.

Like I said, I'm tired and may be missing something obvious.

I was thinking there would be data available week-to-week in previous years.

[WSUwatcher]

Almost any serious power index, KenPom included, will be better -- in the sense of defining how "good" a team is (mainly, how likely it is to win any given game) -- than RPI, because it considers more factors of importance than RPI does. In the RPI, a win by one is the same as a win by 20, but in real life it isn't; usually, dominating a team says something about the relative quality of the two teams, but RPI doesn't consider that. RPI is flawed because it treats all wins or losses of a given type (home, road, neutral) as equal to each other, and they simply aren't. KenPom (for example) tries to differentiate, and even if you don't agree with exactly how Pomeroy quantifies the differences, at least he recognizes they exist.

[MadaboutWu]

IIRC RPI does discount home wins and road losses and reward road wins but agree with all the rest.

I believe RPI doesn't adjust for road and home for SOS, which is a serious flaw.

[Jamar Howard 4 President]

I believe RPI doesn't adjust for road and home for SOS, which is a serious flaw.

Over the course of a season, WSU may play 55% at home and 45% on the road, thus it is important that the RPI factors home/away into the first level of calculations.

However, it would be really rare for WSU to put together a schedule of teams that all tend to play mostly home or mostly away games. The sum total of all of WSU’s opponents’ schedules is likely to be much closer to a 50/50 home/away split. Unlike the mere 34 games (give or take) that WSU plays, WSU’s opponents will play a total of 1,000+ games. The larger sample size (1,000+ vs 34), the greater chance the ratio will be 50/50.

At the level of opponents’ opponents, we would be talking about almost 40,000 games, making home/away a virtual certainty to be nearly an exact 50/50 split.

I'm not saying taking home/road into account for SOS would be bad, just that it would make almost zero difference in the final rankings the RPI produces and would add additional complexity to the formula.

[Ricardo del Rio]

Almost any serious power index, KenPom included, will be better -- in the sense of defining how "good" a team is (mainly, how likely it is to win any given game) -- than RPI, because it considers more factors of importance than RPI does. In the RPI, a win by one is the same as a win by 20, but in real life it isn't; usually, dominating a team says something about the relative quality of the two teams, but RPI doesn't consider that. RPI is flawed because it treats all wins or losses of a given type (home, road, neutral) as equal to each other, and they simply aren't. KenPom (for example) tries to differentiate, and even if you don't agree with exactly how Pomeroy quantifies the differences, at least he recognizes they exist.

Thank you, Watcher. Everything that Jamar and you have said makes good sense.

As an example, a one point overtime loss on a road court does not equal a one point win on a home court. i.e. Utah v. Alabama.

[MadaboutWu]

Over the course of a season, WSU may play 55% at home and 45% on the road, thus it is important that the RPI factors home/away into the first level of calculations.

However, it would be really rare for WSU to put together a schedule of teams that all tend to play mostly home or mostly away games. The sum total of all of WSU’s opponents’ schedules is likely to be much closer to a 50/50 home/away split. Unlike the mere 34 games (give or take) that WSU plays, WSU’s opponents will play a total of 1,000+ games. The larger sample size (1,000+ vs 34), the greater chance the ratio will be 50/50.

At the level of opponents’ opponents, we would be talking about almost 40,000 games, making home/away a virtual certainty to be nearly an exact 50/50 split.

I'm not saying taking home/road into account for SOS would be bad, just that it would make almost zero difference in the final rankings the RPI produces and would add additional complexity to the formula.

One issue though is SOS is presented as a rank on its own. Most years we play more OOC away and neutral than other programs in the top 25. They also rank OOC SOS. We are being put at a disadvantage although probably slight in that measure.

[WSUwatcher]

IIRC RPI does discount home wins and road losses and reward road wins but agree with all the rest.

I believe RPI doesn't adjust for road and home for SOS, which is a serious flaw.

You're correct about the home and road adjustments, Mad, which is why I referred to wins and losses "of a given type (home, road, neutral)" -- because they all have different values in the RPI calculation. Of course, one can assume pretty confidently even without trying to do any formula research that Pomeroy and other rankers also assign different values to each type, too, so that isn't really a point in favor of RPI; it's just a respect in which RPI is probably no worse than the others.

The problem with RPI is that the home-road differential is about as sophisticated as it gets.