SOS - What it does and doesn't tell us

[ciaomichael]

The problem with the RPI is that is is additive, where it should be multiplicative. A team should only get the benefits of an opponent's SOS if it actually wins the game against them. The formula should be: win, then add 1 x opponent's SOS. Lose, then add 0 x opponent's SOS. That would eliminate the ridiculous situation of a team improving it's RPI when it loses. You should't benefit just by virtue of playing someone.

[Jamar Howard 4 President]

ciaomichael, that is a whole other topic. If you are interested, feel free to start a thread and I'd be happy to discuss there, but I've been trying to steer this discussion away from the merits of the RPI because I know that it is controversial. I made my initial scenario RPI free for just that very reason.

[Jamar Howard 4 President]

Alright, back to the intended discussion.


Many of you have asked why I provided such limited information for the initial question, “who would you rank higher, team A or team B?” Also, many have asked why the thread had SOS in the title when the first post didn’t even mention SOS.

The answer is that I wanted to make a point regarding SOS and to show that, in my opinion, SOS rankings are not as concrete as many people think, meaning that upon detailed analysis, there are times when a team with a calculated SOS of 50 might actually have played a tougher schedule than a team with a calculated SOS of 40. It is my opinion that two teams could have identical W/L records, and the team with the “weaker” SOS ranking could have, in reality, played a significantly harder schedule than the other, and therefore, deserve to be ranked significantly higher. Put into numbers, this would mean a 22-9 team with an SOS of 50 could actually, in certain cases, deserve to be ranked higher than a 22-9 team with an SOS of 40. A first glance would tend to lead to the assumption that 22-9 with an SOS of 40 MUST be better. Same W/L record, better SOS ranking. What’s to argue, right? That assumption, which seems logical on the surface, is what I hoped to show to not always be the case.

Many of you responded that you couldn’t answer the original question with any level of confidence because there were so many unknowns about Team A and Team B. Many of you felt that additional information I might add later in this discussion would likely alter your answer to which team should be ranked higher, and I think that scared you off from making an initial assessment based on my limited info. Maybe I just didn’t explain myself well enough, but the answer to the original question “who would you rank higher” wasn’t all that important. It was the reasoning behind making that determination that I wanted to focus on. I didn’t so much need the answer of WHO, Team A or Team B. I needed to know WHY you selected Team A or Team B. I assumed most people would agree on Team A, so I tried to ask anyone who responded “B” to explain WHY. A whole bunch of posts followed questioning the validity of where I might be going with all this, or simply being impatient that I wouldn’t move on at the pace they demanded. All those posts tended to overshadow this one main item I was trying to establish. A few of you did a good job answering “why”. I just wish we could have had more answers and fewer rabbit trails.

Let’s focus a little more on Team A vs Team B for a moment. I think the majority of folks would say Team A should be ranked higher based on the very limited info. They would say that beating a top 10 team and a crappy team is more impressive than beating 2 mediocre teams. The key focus here is that the win over the crappy team shouldn’t be considered to be a negative on Team A’s resume. They won the game. Heck, let’s even add a piece of information. Let’s say they won it by 30. In that case, they did exactly what a good team should have done. I sometimes hear people criticize good teams by mentioning some of the crappy teams they beat. We can all complain about the high level BCS programs who schedule a bunch of cupcakes at home. This is a legitimate complaint for another discussion, but I don’t think the scheduling philosophy of a team is usually all that relevant to a pure discussion of how good (or bad) that teams is. To put it another way, I can sympathize with anyone who complains about Syracuse beating up on crappy competition at home in the non-conf. However, I do not agree that a Syracuse 30 point win at home over some crappy team proves anything negative about Syracuse’s quality of basketball. It may not be worth much as a resume enhancer, but that does not mean it should be a black eye on their resume either. It may be that it simply should be thrown out as virtually meaningless, but I think it is critical to establish that a blowout win over a weak opponent should never be used as a strike against a team. This can easily be seen by Saturday night’s performance against MSU. The Shox may or may not deserve much of a bump from that game, but I would sure hope that they would never take a hit in anyone’s rankings for it.

I’m out of time for now. Sorry for needing to break right in the middle of my comments, but I will have to finish posting more thoughts later.

[jdshock]

The answer is that I wanted to make a point regarding SOS and to show that, in my opinion, SOS rankings are not as concrete as many people think, meaning that upon detailed analysis, there are times when a team with a calculated SOS of 50 might actually have played a tougher schedule than a team with a calculated SOS of 40. It is my opinion that two teams could have identical W/L records, and the team with the “weaker” SOS ranking could have, in reality, played a significantly harder schedule than the other, and therefore, deserve to be ranked significantly higher. Put into numbers, this would mean a 22-9 team with an SOS of 50 could actually, in certain cases, deserve to be ranked higher than a 22-9 team with an SOS of 40. A first glance would tend to lead to the assumption that 22-9 with an SOS of 40 MUST be better. Same W/L record, better SOS ranking. What’s to argue, right? That assumption, which seems logical on the surface, is what I hoped to show to not always be the case.

Throwing losses into the mix seems like an unnecessary complicator.

Imagine with me, for a moment, a not-so-distant future: RPI #351 - Creighton. After the mass exodus to Nebrasketball, Creighton struggles to pay the rent for its deal with the Quiet Center. They begin to accept buy games at some of the elite programs in the country (Kentucky, Duke, Wichita State). With these buy games, Creighton manages to rack up the #1 overall SOS. They lose 9 games against the top 9 teams in the country. Then, in the Big East (merely a shell of its former glory) Creighton manages a few unanticipated upsets and ends up 22-9. They didn't beat a single team with an RPI higher than 175, though. Many, many, many teams with worse SOS's should be ranked higher than that Creighton team.

I say most of this in jest. I am serious when I ask why you would throw losses into the mix? It feels like it just muddies the waters because now you must ask who the teams lost to and who they beat.

[SB Shock]

First my statement was "gibberish", but now you say it was so obvious that you are questioning why I bothered to mention it at all?

Of course it was gibberish, it made no sense and required yourself to give an explanation to what you meant. Then you give an explanation and the only response could be is "No Duh".

You are just being difficult for the sake of being difficult. If you are only here to criticize me for adding 3 words to a post so that there would be less room for misunderstandings, then please just leave and go be a jerk elsewhere.

LOL, you must have been looking in a mirror when you wrote that.

[wufan]

Alright, back to the intended discussion.


Many of you have asked why I provided such limited information for the initial question, “who would you rank higher, team A or team B?” Also, many have asked why the thread had SOS in the title when the first post didn’t even mention SOS.

The answer is that I wanted to make a point regarding SOS and to show that, in my opinion, SOS rankings are not as concrete as many people think, meaning that upon detailed analysis, there are times when a team with a calculated SOS of 50 might actually have played a tougher schedule than a team with a calculated SOS of 40. It is my opinion that two teams could have identical W/L records, and the team with the “weaker” SOS ranking could have, in reality, played a significantly harder schedule than the other, and therefore, deserve to be ranked significantly higher. Put into numbers, this would mean a 22-9 team with an SOS of 50 could actually, in certain cases, deserve to be ranked higher than a 22-9 team with an SOS of 40. A first glance would tend to lead to the assumption that 22-9 with an SOS of 40 MUST be better. Same W/L record, better SOS ranking. What’s to argue, right? That assumption, which seems logical on the surface, is what I hoped to show to not always be the case.

Many of you responded that you couldn’t answer the original question with any level of confidence because there were so many unknowns about Team A and Team B. Many of you felt that additional information I might add later in this discussion would likely alter your answer to which team should be ranked higher, and I think that scared you off from making an initial assessment based on my limited info. Maybe I just didn’t explain myself well enough, but the answer to the original question “who would you rank higher” wasn’t all that important. It was the reasoning behind making that determination that I wanted to focus on. I didn’t so much need the answer of WHO, Team A or Team B. I needed to know WHY you selected Team A or Team B. I assumed most people would agree on Team A, so I tried to ask anyone who responded “B” to explain WHY. A whole bunch of posts followed questioning the validity of where I might be going with all this, or simply being impatient that I wouldn’t move on at the pace they demanded. All those posts tended to overshadow this one main item I was trying to establish. A few of you did a good job answering “why”. I just wish we could have had more answers and fewer rabbit trails.

Let’s focus a little more on Team A vs Team B for a moment. I think the majority of folks would say Team A should be ranked higher based on the very limited info. They would say that beating a top 10 team and a crappy team is more impressive than beating 2 mediocre teams. The key focus here is that the win over the crappy team shouldn’t be considered to be a negative on Team A’s resume. They won the game. Heck, let’s even add a piece of information. Let’s say they won it by 30. In that case, they did exactly what a good team should have done. I sometimes hear people criticize good teams by mentioning some of the crappy teams they beat. We can all complain about the high level BCS programs who schedule a bunch of cupcakes at home. This is a legitimate complaint for another discussion, but I don’t think the scheduling philosophy of a team is usually all that relevant to a pure discussion of how good (or bad) that teams is. To put it another way, I can sympathize with anyone who complains about Syracuse beating up on crappy competition at home in the non-conf. However, I do not agree that a Syracuse 30 point win at home over some crappy team proves anything negative about Syracuse’s quality of basketball. It may not be worth much as a resume enhancer, but that does not mean it should be a black eye on their resume either. It may be that it simply should be thrown out as virtually meaningless, but I think it is critical to establish that a blowout win over a weak opponent should never be used as a strike against a team. This can easily be seen by Saturday night’s performance against MSU. The Shox may or may not deserve much of a bump from that game, but I would sure hope that they would never take a hit in anyone’s rankings for it.

I’m out of time for now. Sorry for needing to break right in the middle of my comments, but I will have to finish posting more thoughts later.

Any chance you have an example that proves your point? Not sure if I agree or disagree.

The problem with the "2-0" scenario is that I would look at home/away and margin of victory to determine who I liked better. The SOS has too little data to be taken on its own as a ranking other than to say they are equal.

[Jamar Howard 4 President]

Throwing losses into the mix seems like an unnecessary complicator.

Imagine with me, for a moment, a not-so-distant future: RPI #351 - Creighton. After the mass exodus to Nebrasketball, Creighton struggles to pay the rent for its deal with the Quiet Center. They begin to accept buy games at some of the elite programs in the country (Kentucky, Duke, Wichita State). With these buy games, Creighton manages to rack up the #1 overall SOS. They lose 9 games against the top 9 teams in the country. Then, in the Big East (merely a shell of its former glory) Creighton manages a few unanticipated upsets and ends up 22-9. They didn't beat a single team with an RPI higher than 175, though. Many, many, many teams with worse SOS's should be ranked higher than that Creighton team.

I say most of this in jest. I am serious when I ask why you would throw losses into the mix? It feels like it just muddies the waters because now you must ask who the teams lost to and who they beat.

jdshock, good post. Your analogy actually works very well to show one of the things I've wanted to show. SOS can be deceptive, and we need to be willing to look beyond the numbers. Your Creighton example may be extreme, but there are real world examples of that exact same thing happening, just on a less extreme scale.

Indeed, losses do "muddy the waters". That is why I said that what might seem obvious at first (same record and better SOS = better team) is not always the case.

[Jamar Howard 4 President]

Now you are talking about whether a playing different RPI teams will lead to the same SOS... your curve is a mess...and now you will have to cherry pick team to make your argument.

I already told you that this is the wrong graph. It is a virtually meaningless piece of information relative to our discussion. Why did you bring it up again?

[Jamar Howard 4 President]

Any chance you have an example that proves your point? Not sure if I agree or disagree.

The problem with the "2-0" scenario is that I would look at home/away and margin of victory to determine who I liked better. The SOS has too little data to be taken on its own as a ranking other than to say they are equal.

Home/away is not given, thus must be ignored. I guess I could have just as easily said "all games for Team A and Team B were at home". Wouldn't have made a difference. Either way, the category should be ignored.

Same for margin of victory. If you don't know the answer, you must ignore the category altogether. Once again, I could have said "all wins by Team A and Team B were by 10 points". Wouldn't have a difference.

Home/Away, margin of victory, Etc were all meant to be constants, not variables.

[SB Shock]

So I have a hypothetical team that beats a Top 5 team and loses to a top 5 team and has 3 additional wins against Mid-200 teams? Where do you rank this hypothetical team?

[wufan]

Home/away is not given, thus must be ignored. I guess I could have just as easily said "all games for Team A and Team B were at home". Wouldn't have made a difference. Either way, the category should be ignored.

Same for margin of victory. If you don't know the answer, you must ignore the category altogether. Once again, I could have said "all wins by Team A and Team B were by 10 points". Wouldn't have a difference.

Home/Away, margin of victory, Etc were all meant to be constants, not variables.

Again, with no additional info, I'd say they are the same. Any examples of this in real life?

[Jamar Howard 4 President]

Again, with no additional info, I'd say they are the same. Any examples of this in real life?

Thanks for the honest, straightforward answer. I'll get to real life examples pretty soon.

[Jamar Howard 4 President]

So I have a hypothetical team that beats a Top 5 team and loses to a top 5 team and has 3 additional wins against Mid-200 teams? Where do you rank this hypothetical team?

Compared to whom? All my questions gave 2 teams to be compared.

[jdshock]

jdshock, good post. Your analogy actually works very well to show one of the things I've wanted to show. SOS can be deceptive, and we need to be willing to look beyond the numbers. Your Creighton example may be extreme, but there are real world examples of that exact same thing happening, just on a less extreme scale.

Indeed, losses do "muddy the waters". That is why I said that what might seem obvious at first (same record and better SOS = better team) is not always the case.

Fair enough

This article makes a similar argument

When we're talking about high major conference teams who play a few good teams and maybe some really terrible teams, the best way to evaluate the schedule is to determine what an elite team's winning percentage would likely be against that schedule. This is superior to merely averaging the rank or Pythagorean expectancy of opponents, because a high-major team that plays the No. 1 and No. 351 teams has played a much tougher schedule -- in terms of the likelihood of suffering a loss -- than a team that plays No. 175 and No. 176. In other words, that method over-emphasizes the difference between bad (No. 175) and terrible (No. 351) teams -- both of which a high-major team is very likely to beat.

[SB Shock]

Compared to whom? All my questions gave 2 teams to be compared.

against your other hypothetical team that is 5-0 that with wins against say mix of low 100 and with a mid-200 thrown in