| Boyd's World-> Breadcrumbs Back to Omaha-> Pre-season Polls and What They're Worth | About the author, Boyd Nation |
Publication Date: January 23, 2001
Ooh, We're Ranked
Periodically, I'm asked why I don't do some sort of pre-seeding in the ISR's. Usually, I'm asked this by someone who's offended by whoever's ranked first after ten games. My answer, when I bother to be more verbose than, "That way lies madness," or, "I'm not old enough to be a poll voter," is that there are several conflicting goals that a ranking system can have, and the one that I've chosen is to try to measure as accurately as possible the quality of each team's season to date. I'm not trying to measure the strength of the program; I'm not trying to predict how well a team will do in the postseason (although there's certainly a correlation between season quality to date and postseason success); I'm not trying to account for why a team has done well or poorly. I just want to know how it's gone so far.
This means that early in the season, the ISR's can look a little less logical than the conventional polls. However, what I'm beginning to think is that the conventional polls only tend to look logical because they're heavily-laden with traditionally-strong schools with a few newcomers thrown in to sell copies (not that the voters don't necessarily believe in what they're doing, but they have to maintain their own interest as well as the readers, and including unusual picks helps in that).
A Quick Study
To try to look at the issue of how accurate pre-season (and, by extension, early season) polls can be, I put together a small study, which was somewhat limited by available data. If you have pre-season polls going back more than a couple of years, let me know, and I'll take another look and see if there are any discrepancies over the long run.
Presumably, any accuracy that the pre-season polls had would come from the voters' ability to predict player and team performances based on their knowledge of the team roster and schedule. Since predictions on 18-21 year-old's are notably shaky, that means there should be a good bit of variety in the votes. That doesn't happen, though; the same folks end up awfully close to the same spots in each of the three major polls (my favorite example of this phenomenon is the "honorary" spot given to the best team from a mediocre conference -- Minnesota is in the next-to-last spot in two of the three polls this pre-season, which is interesting since that puts them at #24 and #39, respectively).
If there was some value to the predictions being made, you would expect the pre-season polls to be much more accurate than those from the previous year. That's only slightly true, though:
1999 CWS: Year Poll Top 10 Total Ranking 1998 PRS 5 84 1998 PoS 3 111 1999 PrS 5 84 2000 CWS: Year Poll Top 10 Total Ranking 1999 PRS 2 148 1999 PoS 2 129 2000 PrS 3 115 PRS -- Post-Regular Season PoS -- Post-Season PrS -- Pre-Season
The column headings indicate the number of teams from that year's CWS to appear in the given poll's top 10 -- there were five teams from the 1999 CWS in the 1998 post-regular season poll, for example -- and the total ranking of the field in that poll (I used 31 points for teams not listed at all). I used the Collegiate Baseball poll for this, but I have no reason to think that any of the other polls would give different results.
What we see from this is that the pre-season poll appears to be slightly more accurate than the previous season end-of-season polls, but only slightly. Given the turnover on rosters from season to season, that means that the voters are not doing particularly well at predicting performance. That's not their fault; it's just an impossible job until they actually play the games. And that's why I don't pre-seed the ISR's.
| Boyd's World-> Breadcrumbs Back to Omaha-> Pre-season Polls and What They're Worth | About the author, Boyd Nation |