This wasn't really the way I planned it, but it might just work better.
Boyd's World tries to look big, but in the end it's just me working with
what I've got, and what I've got at the moment is a home broadband
connection that's really dodgy, left me off the air completely for a few
days last weekend and could go out at any time again. I'll try to keep the
ratings updated as I can, since I know some of you rely on them, but in
the mean time that leaves with the question of what to do with some
columns that I had planned.

In trying to think about what was actually needed, I realized that you
don't particularly need me to put together a list of what the field
would look like if seeded by the ISR's; you can do that yourself pretty
easily if you want to, and the results are predictable, with more West
Coast teams in and seeded higher but not many extra major conference
teams this year. You certainly don't need to know who I think is
actually going to be in the field; that's a whole cottage industry by
now, and everyone's sure about the same 60 teams and thoroughly
guessing on the rest.

What you do need to know, though, is why the current system doesn't work,
so let's talk about that for a while. First of all, there's the
question of the parts that don't involve the selection criteria at all.
Anyone who's ever spent much time working on committees will recognize
the output of the NCAA baseball committee if they look at it for a while,
because it's got all the hallmarks of any other dysfunctional committee --
everyone comes in with their own agenda, and they bargain pieces of
those agendas until everyone's slightly unhappy and then come up with
rationales for public consumption for the results. No one's evil here,
especially since the system is designed for committee members to serve
as regional advocates currently, but in the absence of clearcut rules,
these are the results you get.

That said, those rationales do have to make some sense, so there are some
fences put around their behavior. These are set up in the committee manual
when it talks about the factors that the committee is to consider. They
theoretically have some leeway on this, and they use that for
considerations like discounting the RPI or trying to grow the game, but
there's no evidence that they look at any statistical evidence other than
what's on that list, so let's look at the factors that are included with a
series of studies.

First of all, there's the RPI. It's not the source of all evil, but it
really doesn't work accurately for baseball. Just as a quick proof that
there are better systems out there, here's an example of one of the two
types of studies that I'm going to present; in this case, it's identifying
games between teams where the methods disagree on placing them. From 1999
to 2007, there were 160 games in the postseason where one team had a higher
ISR and the other had a higher RPI:

Better ISR, Worse RPI: 88-72, .550

I'm just going to do the ISR for simplicity (and I'm not going to look at
deeper issues, like the fact that the gap widens if you just look at cases
where the two systems disagree by at least 10 spots), but I'm sure you'd
get similar results with any of a half-dozen or so Internet-available
systems, notably Ken Massey's, since the NCAA already has experience with
him through the BCS. The RPI just doesn't work well.

But you know what works even less well? Everything else they're using.
We'll start with that starter of bar fights everywhere, conference
standing (or conference winning percentage, since that accomplishes the
same thing and is easier to compute). For this type of study, since
there aren't enough pairs for a head-to-head comparison, we'll compare
the full postseason records of the two teams from each set of pairs with
opposing results:

Conference winning percentage is a poorer predictor of postseason success
than the RPI and is much worse than the ISR, beginning a trend that we'll
continue here.

Next up is the darling of those who believe in the hot hand, the reason
that this week gets overemphasized so much, record in the last 15 games.
This time we've got enough matchups to do the head-to-head comparison:

I could go through the rest of the list, but it doesn't get any different.

When comparing two teams directly, they can also look at head-to-head or
common opponents, but those are harder to test (although they're unlikely
to be better predictors). There's also an ill-defined "record against
teams under consideration", which is probably something like the RPI top
75 and autobid winners, but who knows?

I'm well aware of the value of multivariate analysis, having done a good
bit of it myself, but that's not what the committee does -- they're just
taking the potentially useful pieces and then using the ones that they
need to justify their decisions, which is backwards from the way it should
work. Part of the reason for that is that they don't have the analysis
that they need going in; they have a very hard job to do this weekend,
and they're not going to be given the tools to do it with.

I'll see you Tuesday, hopefully, to talk about the results.

If you're interested in reprinting this or any other Boyd's World material
for your publication or Web site, please read the
reprint policy and
contact me