Boyd's World-> Breadcrumbs Back to Omaha-> Coaching Metrics

About the author, Boyd Nation

Coaching Metrics

How do you tell how good a coach is? Obviously, there's the anecdotal approach, where every coach who's ever lived, no matter how inept or cruel, will have someone to testify that he was the greatest ever, or at least pretty darn good, and someone else will testify that he wasn't all that since they didn't win as many titles as they should have while he was there, however many that may be. If you've read me much, you'll know I don't have much use for that approach. On the other hand, there aren't any really good metrics for coaches yet (and, in a lot of cases, even if there were, we wouldn't have the historical data to make good comparisons). What I want to do this week is not to present any actual measurements, but just to begin the process of thinking about how you would answer that question of how good a coach is with any real certainty.

Now, there is one ever-present metric for a coach -- winning percentage -- and coaches get fired based on it all the time. It's not a particularly good metric, though, because the coach only has a partial impact on it, as all those external factors that are out of the control of anyone in the baseball program come into play. The situation is very similar to the case of measuring management in corporate America -- the only real metric that matters most of the time is how much money the company makes, and management has only a limited amount of control over that outcome. The analogy holds up in lots of other ways -- lots of really bad management teams have made lots of money in the right market while some good folks have gotten run out of town after things like weather-related business failures, and some fairly awful coaches have won a bunch of games, and vice versa.

The cynical Heller quote above notwithstanding, there are good managers and bad managers, and there are good coaches and bad coaches, but there are almost no measures available to tell the difference, and evaluation is often done badly. Corporate middle management, who is generally even protected from the effects of the bottom line, is essentially unmeasured except on things like safety measures or the irrelevant popularity contests that pass for employee surveys. That's someone else's problem (actually, it's mine, too, but it's not yours if you're lucky), but let's take a look at that baseball thing.

In order to measure, we'd have to figure out what coaches do that actually matters, so what follows is an incomplete list (suggestions for completion are welcomed, as long as they don't include the words inspiration or intangible), along with any ideas I have on how to measure them and how hard that would be.

Recruiting is probably the most important job that any coaching staff has. Each high school player is a free agent, and the single easiest way to win is to have the best players come to play for you. It's also one of the hardest to measure, as the worthlessness of the annual recruiting rankings shows. To be honest, I have no idea at all about how to construct a metric for recruiting that's not too bound up in the professional scouting constructs, which gets us off to a bad start here.

The second piece of the puzzle, as far as I can tell, is player development. Once you've got your guys in house, you've got them for three years, and how much better they are at the end of that time than at the beginning, especially given the wide range of development from 18 to 21 in general, can make a huge difference in your bottom line. This one is a lot easier to measure, since we know how much the average player's stats change from year to year and can compare that with the improvement for each team. There are some complicating factors involving how much you have to play freshman, but I think those can be controlled for.

Another big factor that falls on the shoulders of the coaching staff is talent evaluation. Deciding who gets how many at bats and how many innings pitched can make a big difference. Theoretically, it should be possible to work on this one by comparing stats for bench players and starters, but for an awful lot of schools the bench doesn't get enough playing time to be useful. This one will require some thought; it might also be possible to look at later numbers for underclassmen to see if they should have gotten more playing time earlier.

The final big piece that I can identify is pitching staff management, which can be divided into two major pieces -- pitcher fatigue management and injury rate. Fatigue management should be measurable: How much does the staff ERA rise during the season? Injury rates are a bit harder to measure in real life without hand-tracking each team (and teams often don't release that information), but we might be able to get to it indirectly by looking at IP patterns.

Plate discipline seems to be something that flows from the coaching staff, whether it's by being valued during recruiting or by being taught on campus, and that should be easy to measure by looking at walk rates on both sides of the ball.

One big difference between professional ball and college ball is that college coaches get to select their strength of schedule. Now, measuring strength of schedule is not an exact science, but we can get a reasonable feel for it, so it's worth putting it on the list to see what it gets us.

Finally, there's the least important but most visible things that coaches do, the in-game tactics. The ones that really matter are already reflected in the pitching management metrics, and things like stolen base rates and bunt rates really don't have enough effect to bother with, but offensive substitution patterns might be worth a look.

What we have there, then, is a reasonable starting point for a discussion of how to measure how good a coach is. Over the course of the season, I'll try to take some of these ideas and run with them, solidifying the measurements and testing to see how good a predictor they are of winning.

Boyd's World-> Breadcrumbs Back to Omaha-> Coaching Metrics

About the author, Boyd Nation