The Flaw in RPI

[Aargh]

I posted this on MVCFans, but many here don't go there, so I'll duplicate it here.

I think Valley fans pay too much attention to RPI. RPI is nothing more than an indicator and it can get way out of whack. I think the Selection Committee has recognized this problem many years ago and the Committee has probably been using RPI appropriately by mostly ignoring it.

Let's say that a team played half their season against end-of-the-season top-50 teams and went 8-8 in 16 games. That would establish them as probably the 20th-30th best team in the country. Their RPI would probably be below 10. There's the first problem.

Now, have that same team play another 16 games against bottom-50 teams and win all the games. The cupcakes kill the team's RPI. The team, that is now 24-8 in March, probably has an RPI around 70 or 80.

The team is established as a top-25 type team, yet they have an RPI that would lead to automatic elimination by the Committee if you just looked at RPI.

WSU's game against ChiSt demonstrates this. Compared to all other teams, WSU had been 28th based on their performance to this point. After playing ChiSt, WSU dropped to 50th because the SoS component was affected. WSU didn't actually get worse in comparison to other teams, but the RPI indicates they did.

This is probably what affected MSU a couple of times under Hinson. Other schools had played more games against higher-level competition (and won enough of those games), but also played more games against lower-level competition. MSU had a great W/L record and a solid SoS component, but beating RPI 100-150 teams bumped MSU's SoS higher than some other schools with a bunch of W's over RPI 250+ teams.

If Team A beats an RPI 100-150 opponent and Team B beats an RPI 250 opponent, that does nothing to indicate which team is better, but Team A will have a much higher RPI than Team B.

[SB Shock]

NCAA uses rpi when it useful for them to justify a decision. They ignore it when it doesnt help support their decisions.

As with any metric, it is useful if you understand how it is formulated, what it's limitations are, and what it really tells you.

[Kung Wu]

I posted this on MVCFans, but many here don't go there, so I'll duplicate it here.

I think Valley fans pay too much attention to RPI. RPI is nothing more than an indicator and it can get way out of whack. I think the Selection Committee has recognized this problem many years ago and the Committee has probably been using RPI appropriately by mostly ignoring it.

Let's say that a team played half their season against end-of-the-season top-50 teams and went 8-8 in 16 games. That would establish them as probably the 20th-30th best team in the country. Their RPI would probably be below 10. There's the first problem.

Now, have that same team play another 16 games against bottom-50 teams and win all the games. The cupcakes kill the team's RPI. The team, that is now 24-8 in March, probably has an RPI around 70 or 80.

The team is established as a top-25 type team, yet they have an RPI that would lead to automatic elimination by the Committee if you just looked at RPI.

WSU's game against ChiSt demonstrates this. Compared to all other teams, WSU had been 28th based on their performance to this point. After playing ChiSt, WSU dropped to 50th because the SoS component was affected. WSU didn't actually get worse in comparison to other teams, but the RPI indicates they did.

This is probably what affected MSU a couple of times under Hinson. Other schools had played more games against higher-level competition (and won enough of those games), but also played more games against lower-level competition. MSU had a great W/L record and a solid SoS component, but beating RPI 100-150 teams bumped MSU's SoS higher than some other schools with a bunch of W's over RPI 250+ teams.

If Team A beats an RPI 100-150 opponent and Team B beats an RPI 250 opponent, that does nothing to indicate which team is better, but Team A will have a much higher RPI than Team B.

You are finally catching up to @JamarHoward4President and my discussion last year:

http://shockernet.net/forum/showthread.php?9917-Imagine-a-conference-where...&p=241449&viewfull=1#post241449

I did some crunching on the real numbers at the end of the season and showed that 69% of RPI is based on the strength of schedule. Therefore RPI is a complete buzz saw to all but the incestuous big money conferences. I say incestuous because conferences like the Big East will have teams toward the bottom of the conference getting a substantial artificial boost by simply having automatic games against top tier programs.

The BCS conferences should avoid "RPI Killers" like WSU. Why shouldn't they? They will get 69% of their RPI in conference and it behooves them to not "share the wealth" with very strong teams that can't get the schedule that they can.

[KC Shox]

I'm no expert, but isn't the Sagarin rankings a lot more accurate than the RPI?

[The Mad Hatter]

I'm no expert, but isn't the Sagarin rankings a lot more accurate than the RPI?

I guess it depends on what you mean by accurate, but rankings like Sagarin and Pomeroy are certainly much more sophisticated models for team rankings than RPI is. In part this is because RPI isn't really supposed to be a complete statistical ranking model so much as way to weigh winning percentage and strength of schedule.

I don't know all of the statistical models out there, but Sagarin, Pomeroy, and Massey all take a lot more factors into consideration than RPI does and gives them weight in a more sophisticated manner.

[shox1989]

NCAA uses rpi when it useful for them to justify a decision. They ignore it when it doesnt help support their decisions.

As with any metric, it is useful if you understand how it is formulated, what it's limitations are, and what it really tells you.

+1 - The NCAA Selection committee will definitely hold it against you if you have a bad rpi. So you have to pay some attention to it.

Ignore it at your own risk!

[Aargh]

+1 - The NCAA Selection committee will definitely hold it against you if you have a bad rpi. So you have to pay some attention to it.

Ignore it at your own risk!

Then WSU has a real problem. The MVC contains 3 teams that look like RPI 250+ teams. UE, SIU, IlSU. That gives WSU 6 conference games that are RPI killers. Throw in ChiSt and that's 7 RPI killers on the schedule.

The MVC has 6 teams that look like top-100 teams and 4 that could be top-50 teams, but...after all those top-50 type teams play 6 games against the RPI killers, it could be difficult for the top end of the Valley to maintain top-50 RPI status. RPI penalizes a team for winning a game against an RPI 250 team when other comparable teams are winning games against RPI 150-200 type teams.

The way RPI is set up, WSU, CU, UNI, and InSU will all be penalized for playing in a league that has 3 RPI 250+ teams. Right now, the top 6 in the MVC have better RPI's than the top 6 in the B12. That is probably accurate for the level of play in the two leagues. Iowa State is the #4 team in the B12 and the Valley teams that have played ISU have had no problems.

The bottom 4 in the B12 are WAY better than the bottom 4 in the MVC. The B12 us #4 in conference RPI. The MVC is #8. The difference is based on the bottom teams. It appears it will be easier to beat the top 6 in the B12 than the top 6 in the Valley, but it will be more difficult to beat the bvottom 4 in the B12 than the bottom 4 in the MVC. The RPI says that's enough to make the B12 the #4 conference and the MVC the #8 conference.

It is my opinion that the difference in difficulty of a top-50 team beating a 150 team or a 250 team is negligible. RPI treats the win against a 250 team much worse than a win against a 150 team.

[im4wsu]

Then WSU has a real problem. The MVC contains 3 teams that look like RPI 250+ teams. UE, SIU, IlSU. That gives WSU 6 conference games that are RPI killers. Throw in ChiSt and that's 7 RPI killers on the schedule.

The MVC has 6 teams that look like top-100 teams and 4 that could be top-50 teams, but...after all those top-50 type teams play 6 games against the RPI killers, it could be difficult for the top end of the Valley to maintain top-50 RPI status. RPI penalizes a team for winning a game against an RPI 250 team when other comparable teams are winning games against RPI 150-200 type teams.

The way RPI is set up, WSU, CU, UNI, and InSU will all be penalized for playing in a league that has 3 RPI 250+ teams. Right now, the top 6 in the MVC have better RPI's than the top 6 in the B12. That is probably accurate for the level of play in the two leagues. Iowa State is the #4 team in the B12 and the Valley teams that have played ISU have had no problems.

The bottom 4 in the B12 are WAY better than the bottom 4 in the MVC. The B12 us #4 in conference RPI. The MVC is #8. The difference is based on the bottom teams. It appears it will be easier to beat the top 6 in the B12 than the top 6 in the Valley, but it will be more difficult to beat the bvottom 4 in the B12 than the bottom 4 in the MVC. The RPI says that's enough to make the B12 the #4 conference and the MVC the #8 conference.

It is my opinion that the difference in difficulty of a top-50 team beating a 150 team or a 250 team is negligible. RPI treats the win against a 250 team much worse than a win against a 150 team.

Good analysis. Perhaps an alternate RPI should be calculated capping the RPI of any team at, say, 200. Then compare the original and alternate RPI to determine how big an effect this "lower tier" has had on any team and let human judgement take over from there.

Another potential for "sensitizing" the RPI is to throw out the three wins over highest RPI teams and three losses to the best RPI teams, being that these are games you are SUPPOSED to win or lose. Then see how your team is doing against more comparable opponents.

[WuDrWu]

I think we all are aware, and if we aren't then we should be, that RPI has gone from a good tool as a part of an equation to being yet another BCS biatch. It will be summarily dismissed if it's in your favor and held over you like the sword of Damocles if it's too low.

That being said, the formula is what it is, math, and the results are what they are, mathematical results based on input. The Valley has "tricked" the RPI so far this year by being good and fortunate. UNI is 3. Anyone think UNI is even a top 50 team? For example, Providence is (I think) 9-2. I've seen the Friars play. They could go 3-13 in the Big East. St. Marys has a good record against good teams (record wise) but they've played few games. I suspect their #1 rating is about to crash. That's not to say UNI isn't good, or that they haven't performed well. They are good (better than I suspected) and they've been great. They just aren't going to be a top 5, or even top 10-40 unless they continue to win most if not all of their games.

I suspect the Valley will be fortunate to have 3 top 50 teams at the end of the day....and easily it could be 1.

[The Mad Hatter]

Then WSU has a real problem. The MVC contains 3 teams that look like RPI 250+ teams. UE, SIU, IlSU. That gives WSU 6 conference games that are RPI killers. Throw in ChiSt and that's 7 RPI killers on the schedule.

The MVC has 6 teams that look like top-100 teams and 4 that could be top-50 teams, but...after all those top-50 type teams play 6 games against the RPI killers, it could be difficult for the top end of the Valley to maintain top-50 RPI status. RPI penalizes a team for winning a game against an RPI 250 team when other comparable teams are winning games against RPI 150-200 type teams.

The way RPI is set up, WSU, CU, UNI, and InSU will all be penalized for playing in a league that has 3 RPI 250+ teams. Right now, the top 6 in the MVC have better RPI's than the top 6 in the B12. That is probably accurate for the level of play in the two leagues. Iowa State is the #4 team in the B12 and the Valley teams that have played ISU have had no problems.

The bottom 4 in the B12 are WAY better than the bottom 4 in the MVC. The B12 us #4 in conference RPI. The MVC is #8. The difference is based on the bottom teams. It appears it will be easier to beat the top 6 in the B12 than the top 6 in the Valley, but it will be more difficult to beat the bvottom 4 in the B12 than the bottom 4 in the MVC. The RPI says that's enough to make the B12 the #4 conference and the MVC the #8 conference.

It is my opinion that the difference in difficulty of a top-50 team beating a 150 team or a 250 team is negligible. RPI treats the win against a 250 team much worse than a win against a 150 team.

Actually, so long as we take care of business and win the games we are supposed to, we are in no problem whatsoever. RPI Forecast has us finishing 23rd in the RPI if we meet the win total suggested by the Sagarin Predictor (not counting the Bracket Buster game or MVC tourney).

http://www.rpiforecast.com/index2.html

[The Mad Hatter]

I think we all are aware, and if we aren't then we should be, that RPI has gone from a good tool as a part of an equation to being yet another BCS biatch. It will be summarily dismissed if it's in your favor and held over you like the sword of Damocles if it's too low.

That being said, the formula is what it is, math, and the results are what they are, mathematical results based on input. The Valley has "tricked" the RPI so far this year by being good and fortunate. UNI is 3. Anyone think UNI is even a top 50 team? For example, Providence is (I think) 9-2. I've seen the Friars play. They could go 3-13 in the Big East. St. Marys has a good record against good teams (record wise) but they've played few games. I suspect their #1 rating is about to crash. That's not to say UNI isn't good, or that they haven't performed well. They are good (better than I suspected) and they've been great. They just aren't going to be a top 5, or even top 10-40 unless they continue to win most if not all of their games.

I suspect the Valley will be fortunate to have 3 top 50 teams at the end of the day....and easily it could be 1.

So long as the top Valley teams don't drop games to the bottom feeders in the conference, we should have no problem having 3 top 50 teams. As I said above, WSU is currently projected to finish 23rd in the RPI (with a 14-4 conference record). UNI is predicted to finish 20th in the RPI (with a 13-5 conference record) and Creighton is predicted to finish 44th (with a 13-5 conference record). Additionally, Missouri State is predicted to finish 53rd (with a 13-5 conference record). Obviously, bad losses could change things, but it's not like these teams need to avoid losses altogether to be top 50, just losses to the very bottom. Bracket Busters could also change things, but none of these teams should get an RPI killing match up.

[ShockTalk]

Actually, so long as we take care of business and win the games we are supposed to, we are in no problem whatsoever. RPI Forecast has us finishing 23rd in the RPI if we meet the win total suggested by the Sagarin Predictor (not counting the Bracket Buster game or MVC tourney).

http://www.rpiforecast.com/index2.html

This! The "current" RPI will continue to go up and down, even if a team does what is expected of them to do. Example: The rpiforecast.com prediction had already taken into account a Chicago St win in its ending forecast. The things that will affect that future RPI is beating or losing to teams you were not expected to, as well as, this same affect upon those teams you played and they played.

[Kung Wu]

Actually, so long as we take care of business and win the games we are supposed to, we are in no problem whatsoever. RPI Forecast has us finishing 23rd in the RPI if we meet the win total suggested by the Sagarin Predictor (not counting the Bracket Buster game or MVC tourney).

http://www.rpiforecast.com/index2.html

I agree. This year is unique over the last few because the top three teams can beat each other but won't be "cannibalizing" each other. The past few years the scheduling across the conference (69% of RPI's value) was bad enough that it seemed like one, maybe 1.5 teams had to dominate ALL other opponents to keep a very good RPI. This year it's closer to three teams can all win their home games, and lose their away games to each other, and still fair well. It's just a different SOS ranking for the conference this year (finally).

[ShockerPhi]

This was an interesting read. I guess I didn't realized that beating the snot out of awful teams still makes you drop dramatically in RPI. From here on out, I'll just try to ignore it. Thanks for the info