Boyd's World-> Breadcrumbs Back to Omaha-> "Gaming" the RPI

About the author, Boyd Nation

This week's column comes with not one but two disclaimers (yeah, yeah, he takes three weeks off to recover and then we get disclaimers; I'll gladly refund your money). First off, there's probably going to be a lot of math in this one. That's because I'm writing about a mathematical formula, and I'm going to try to pick it apart pretty thoroughly. The good news is that there's nothing in here more complicated than simple arithmetic and logic, so anybody that made it to high school algebra shouldn't have much trouble following.

More importantly, I'm on somewhat shaky ground ethically this week. What I want to describe is the means by which a coach or administrator could manipulate the RPI through scheduling so that their team would be rated higher by the RPI than they should be. This would be, essentially, a distortion of the intended purpose of the RPI's -- it's intended to measure what happens, not to be a goal in and of itself.

Now, following the steps I point out here would provide an unfair benefit to any program that used them, and, if enough people did so, there are some very good programs that would actually have trouble scheduling games. I'm hoping that no one actually does anything I'm writing about this week.

So, why am I writing about it?

I'm not the only one who made it to high school algebra.

I have no idea if there are any coaches who are already scheduling with one eye on the RPI, or if they've figured out exactly how to do it, but I think it's quite possible -- a later project might be to look back at past schedules and see if I can spot that. The problem, though, in the end, is not that someone (or everyone) knows that there's a problem with the RPI's that can be exploited. The problem is that there is a problem with the RPI's. I'm hoping that publishing that problem, with a hopefully good explanation of what it is and how to exploit it, will be one more step toward pushing the RPI out the door and having it replaced by something better. (Those who spot this as being similar to the argument for full public disclosure of computer security problems can congratulate themselves now.)

The RPI is, fundamentally, a very simple formula. Your RPI is 25% of your winning percentage (WP), 50% of the average of your opponents' winning percentages with games against you removed (OWP), and 25% of your opponents' opponents' winning percentages (OOWP). There are some bonuses which get added for winning on the road against good teams, but they're small and don't affect much. Because of the way the percentages are set up, the RPI runs on a scale from 0 to 1.

In actual practice, around 85-90% of the scores end up between .400 and .600. That's an extremely narrow range. Since the whole purpose of the RPI is to aid the selection committee in choosing and seeding the tournament field, it's primarily used to create an ordinal ranking of teams, and only teams that end up in the top 75 or so really care about their RPI. Since they tend to use other factors more in seeding, the places where it matters most is between about places #35 and #65, where the at large bids tend to come from.

I don't have the actual numbers for last year's RPI, but here are some entries from the pseudo-RPI's, which are my best imitation of the RPI:

Rank    pRPI   Team

 35     .586   South Florida
 45     .574   UCLA
 55     .558   Georgia Southern
 65     .550   Ball State

Looking at the gaps here, it looks like picking up around .012 in the RPI will move you up around 10 spaces in the range we're interested in. Ten spaces in the RPI can easily be the difference in getting in and not, all other things being equal. Now, how are we going to get that .012?

Let's pretend, for the rest of this discussion, that everyone plays fifty games each year through the end of the conference tournaments. It's not fully accurate, of course, but it's close enough to the average to not throw off our numbers by any notable amount, and it makes things easier to follow from here on out.

Now, winning percentage is worth up to .250 of your RPI. If we divide that by 50 games, we see that each win is worth .250 / 50 = .005 of RPI. The second factor, though, OWP, is worth up to .500. Dividing that, we see that each opponent there is potentially worth up to .500 / 50 = .010. In other words, an opponent's winning percentage is potentially worth twice as much as whether or not you beat them. On the other hand, winning the game is an all-or-nothing proposition, while their winning percentage is a more fluid value. There are some interesting extremes worth noting, though -- for example, losing to a .780 WP team is better for you than beating a .260 WP team.

The third factor, OOWP, by the way, gets complicated enough that we'll ignore it for now, although I'll include it later when I discuss a numeric schedulability factor.

The thing about baseball, though, is that it's not that predictable. If you're one of those teams that's likely to end up on the bubble for the tournament (and anyone who thinks they're likely to be in the top 125 or so at the time the schedules are made is a candidate for the bubble if things break right), then you have a chance to beat anybody, and you just have to worry about how big that chance is, because you're going to be running a basic risk -- do you schedule the .500 team that you're almost sure that you can beat, which will total up to .010 in RPI with a small chance of only .005, or do you schedule the .700 team you're evenly matched with and have an even chance of picking up .012 or .007? Those questions are close enough that you're probably better off ignoring them and scheduling for the usual reasons like tradition, trying to improve your team, meeting obligations, and the like. Those are good things, and that's how a ratings system should work.

Unfortunately, there's a flaw in this system when it comes to the RPI's, and it comes from overemphasizing the use of winning percentage. Not all .700 teams (or .500 teams, or .300 teams) are created equal. What if your choice, instead of being between scheduling a weak .500 team or an evenly-matched .700 team, was between a weak .500 team and a weak .700 team? Worse yet, what if it was between a strong .500 team and a weak .700 team? A quick look at last year's standings will show several examples of all of those.

And that, in the end, is how you can "game" the RPI. Do schedule teams from the tops of weak conferences and the second tier of mid-level conferences who have gaudy winning percentages but are relatively weak (I'm going to use ISR, one of my rating systems, as a measure of team strength from this point on; you're free to substitute your own judgment of quality if you think it's better). Don't schedule teams from the bottom of strong conferences or regions who have low winning percentages but are still strong teams (these teams, by the way, are the reason that I hope no one takes these steps).

Remember that you only need .012 of RPI to make a decent-sized move up the list, and that .025 will move you quite a long ways. Let's look at a concrete example of a five-game segment from two possible schedules, with each team having their 2001 WP included:

Schedule A:                     Schedule B:

Delaware (2 games, .746)        Oklahoma (2 games, .418)
Delaware State (2 games, .767)  Indiana (2 games, .380)
Ohio State (1 game, .712)       Pacific (1 game, .429)

Now, that's five games. Only one-tenth of the schedule, so subtle that it's barely even noticeable (and frankly, it doesn't even look all that bad; from a prestige point of view, the first schedule probably looks about as good as the second). Each of those pairs of teams are very close to each other in ISR, so you're most likely going to put up the same winning percentage in each case. But, due to the differences in the winning percentages, Schedule A comes to a total of .017 more RPI points, enough for a move of about fifteen spaces in the rankings.

So, is there a way to measure how "schedulable" a team is? Sure, although you have to do some playing with the odds to get it to work out. Essentially, we can just use those numbers that we've used above and add it the work I've done elsewhere on the odds of a given team winning a matchup given the gap between their ISR and an opponent's and stir properly. Since I had to pick a baseline for the odds, I did it as if I were designing a schedule for a team with an ISR of 110, since that's the range where the bubble teams are who would be most helped by jiggling the schedule.

SF = (WP * .010) + (OWP * .005) - (PROB * .005)

where PROB comes from a function that simulates the odds curve for beating a team when your ISR is 110:

PROB = (110 - ISR) * .02 + .5

Since, in this case, we're mostly just interested in the numbers for comparison purposes, we'll just leave them on the scale where that ends up.

Using this formula, I get the following teams as the ten most schedulable for the 2001 season:

  SF      WP    OWP    ISR   Team

0.0138  0.767  0.337   90.9  Delaware State
0.0134  0.774  0.439  100.1  Southern
0.0129  0.746  0.523  106.4  Delaware
0.0129  0.694  0.431   97.1  Maine
0.0128  0.688  0.426   97.1  Stony Brook
0.0128  0.581  0.423   86.6  Alcorn State
0.0127  0.596  0.432   89.6  Maryland-Baltimore County
0.0126  0.787  0.531  114.6  Notre Dame
0.0126  0.628  0.434   93.1  Bucknell
0.0126  0.618  0.461   93.8  Marist

You'll notice included here are the champions of the America East, the SWAC, the MEAC, the NYSBC, the NEC, and the Big East, as well as a couple of other high finishers.

The bottom of the list is made up of teams with really low winning percentages; it's hard to overcome that no matter how easy you are to beat. I've included the entire list on a separate page since this is way too big already.

In conclusion, let me once again stress that I hope no one's actually doing what I'm suggesting here, and I hope no one starts. Let's get rid of the RPI once and for all, and if you get a chance to use this as evidence against it, go ahead; let me know if I can help.

Boyd's World-> Breadcrumbs Back to Omaha-> "Gaming" the RPI

About the author, Boyd Nation