RealGM Analysis

If you asked me to vote for a ranking every week, my ballot would be incredibly boring. Teams sort of fall in a certain range, (as I illustrated the day after New Years), and it takes multiple games for me to change my mind. I’ve liked Kentucky, North Carolina, and Ohio St. from the start of the year, and nothing has changed that. Thanks to Baylor’s sudden ability to play solid man-to-man defense, after years of playing incredibly poor 2-3 zone, I am now a big believer in Baylor. And thanks to Fab Melo’s development, I am now a big believer in Syracuse. But if you asked me to rank teams every week, my poll would almost never change.

But fans want the polls to change. When Illinois upsets Ohio St., Illinois fans want to see the team pick up more AP votes. Illinois fans want to log onto kenpom.com and discover that the season expectation has changed from 8-10 in the Big Ten to 10-8 in the Big Ten. Fans want to see that the games are meaningful, even if each game is only a small part of an overall evaluation.

And that’s why fans find it so frustrating when something they love (the Pomeroy Rankings) seems detached from their perception. Wisconsin is currently 3rd in Ken Pomeroy’s ratings, a spot that is so high that Ken felt the need to defend his ratings on Monday. Kansas’ second place spot might puzzle some people too.

Today I had planned to run some splits showing various teams’ records against teams ranked 1-100 and 100+. And given the discussion on Wisconsin, it seems particularly relevant.

Crushing Bad Teams

Wisconsin has absolutely been pummeling bad teams this year. Even counting the home loss to Iowa, Wisconsin has absolutely been running up the score on the little guys. (I’d say it is bad sportsmanship, except that at Wisconsin’s slow pace, it doesn’t look like it.) And since Ken’s rankings are primarily derived from margin-of-victory, that causes his rankings to love Wisconsin. Here are the teams with the best ratings against bad teams:

Against 101+

Adj Off

Adj Def

W

L

Pyth

Wisconsin

115.3

73.5

10

1

0.9903

Kansas

116.2

75.9

7

0

0.9874

Ohio St.

109.8

72.5

12

0

0.9860

Syracuse

125.7

89.2

10

0

0.9714

Kentucky

118.4

84.3

13

0

0.9701

Indiana

121.8

87.7

10

0

0.9669

Michigan St.

115.0

83.4

10

0

0.9643

Duke

124.1

91.0

6

0

0.9600

Georgetown

108.5

79.6

7

0

0.9598

North Carolina

114.6

85.2

10

0

0.9538

Now, it is tempting to argue that we should throw out the results against teams rated 100+. But running up the score on the little guys is generally an important indicator that you have a good team. In particular, Syracuse, Kentucky, North Carolina, and Ohio St. are all on this list, and they tend to be considered among the top teams by most experts. The key is that all of these teams have also been good against quality competition. Meanwhile Wisconsin has been much worse against quality teams:

Against 101+

Against 1-100

Team

W

L

Pyth

W

L

Pyth

North Carolina

10

0

0.9538

5

2

0.9644

Kentucky

13

0

0.9701

3

1

0.9625

Syracuse

10

0

0.9714

8

0

0.9638

Michigan St.

10

0

0.9643

5

2

0.9550

Ohio St.

12

0

0.9860

3

3

0.9645

Indiana

10

0

0.9669

5

1

0.9323

Duke

6

0

0.9600

7

2

0.9235

Kansas

7

0

0.9874

6

3

0.9454

Georgetown

7

0

0.9598

5

3

0.8921

Wisconsin

10

1

0.9903

2

4

0.9039

The Badgers haven’t been horrible against Top 100 teams. Remember, they lost a close game at North Carolina, lost in OT to Michigan St., and beat BYU and UNLV pretty convincingly. But the Badgers have been nowhere near as good against elite teams as the have been against the Nebraskas of the world.

Interestingly, Wisconsin is far from the worst culprit in this rating inflation. Until Florida St.’s win against Virginia Tech this weekend, the Seminoles hadn’t done anything against a quality team:

Against 101+

Against 1-100

Team

W

L

Pyth

W

L

Pyth

Florida St.

9

1

0.9198

1

5

0.6254

Northern Iowa

9

1

0.9058

2

4

0.6611

Northwestern

9

0

0.8700

2

5

0.6958

Marquette

10

0

0.9535

3

4

0.8219

California

10

0

0.9533

3

4

0.8261

Arizona

10

0

0.8617

1

5

0.7403

Virginia Tech

9

1

0.8905

2

4

0.7704

BYU

9

0

0.9462

3

4

0.8450

Wichita St.

9

0

0.9445

3

3

0.8491

Creighton

10

0

0.8980

4

2

0.8057

On the flip side, here are some teams that have elevated their play against elite competition:

Against 101+

Against 1-100

Team

W

L

Pyth

W

L

Pyth

Seton Hall

10

0

0.8468

5

2

0.9127

Baylor

7

0

0.8880

8

0

0.9550

Temple

6

1

0.7691

5

3

0.8447

New Mexico

10

2

0.8721

3

0

0.9501

Virginia

9

1

0.8616

5

0

0.9579

Murray St.

10

0

0.8467

3

0

0.9462

St. Joseph's

9

2

0.7783

3

3

0.8778

Oral Roberts

10

1

0.7087

4

3

0.8864

Michigan

7

0

0.7370

6

3

0.9355

Middle Tennessee

11

1

0.7487

3

1

0.9508

In Baylor’s case, they were without Perry Jones for the early part of the season, and that explains why they were worse against team 100+. But in other cases, I honestly think it can come down to style of play. Michigan doesn’t have the type of athletes that can just pummel a team from the Patriot League. But their offensive system is so difficult to defend, that elite teams usually cannot stop them from scoring.

As followers of college basketball, we tend to judge teams mostly on how they do against quality competition. To some extent I agree with this. Everyone in the Big 12 should beat Texas Tech, so I don’t think we should read too much into a win against them. But there is a cost to ignoring this information. Baylor beat Texas Tech by 13, while Kansas beat Texas Tech by 35. And when trying to set a line for the Baylor vs Kansas game, I think we should be concerned about Baylor’s struggles against a mediocre team. But for those of you who believe games against teams 100+ are useless, the next table shows how the Pomeroy Rankings would look if we only looked at games against teams rated 1-100.

What you see are some pretty unrealistic rankings. Just because Middle Tennessee has looked great against elite competition, pounding UCLA, crushing Akron, and splitting with Belmont (with the loss coming in 2 OTs), I don’t think anyone believes for a second Middle Tennessee is a Top 10 team. The fact that their Sun Belt games are “competitive” tells us that they are not elite. And it should convince us that ignoring games against weaker teams is not a good way to build a ranking system.

Still, it is interesting to note that Missouri has actually performed better against elite competition than any of the other contenders, and that Virginia has also been surprisingly dominant against good teams this year.

Rank

Against 1-100 only

Adj Off

Adj Def

W

L

Pyth

1

Missouri

126.8

87.1

6

1

0.9790

2

Ohio St.

123.1

89.2

3

3

0.9645

3

North Carolina

117.6

85.2

5

2

0.9644

4

Syracuse

114.4

83.1

8

0

0.9638

5

Kentucky

115.9

84.4

3

1

0.9625

6

Virginia

107.8

79.5

5

0

0.9579

7

Baylor

114.6

85.1

8

0

0.9550

8

Michigan St.

115.1

85.4

5

2

0.9550

9

Middle Tennessee

119.6

89.6

3

1

0.9508

10

New Mexico

113.5

85.1

3

0

0.9501

11

Kansas St.

112.7

85.2

4

3

0.9464

12

Murray St.

120.6

91.2

3

0

0.9462

13

Kansas

113.6

86.0

6

3

0.9454

14

Michigan

118.2

91.0

6

3

0.9355

15

Indiana

119.9

92.9

5

1

0.9323

16

Florida

123.7

96.3

2

3

0.9286

17

UNLV

116.6

90.8

6

2

0.9285

18

Duke

118.2

92.7

7

2

0.9235

19

Gonzaga

114.4

90.4

4

2

0.9183

20

Seton Hall

111.9

89.0

5

2

0.9127

21

Purdue

117.9

94.1

5

2

0.9098

22

Alabama

110.9

88.8

5

3

0.9072

23

Wisconsin

106.2

85.4

2

4

0.9039

24

St. Mary's

111.9

90.0

4

2

0.9037

25

Georgetown

117.4

95.6

5

3

0.8921

26

Stanford

104.2

85.1

5

1

0.8892

27

Oral Roberts

116.7

95.5

4

3

0.8864

28

St. Louis

116.6

95.4

3

3

0.8860

29

St. Joseph's

111.2

91.8

3

3

0.8778

30

Connecticut

113.9

94.1

5

2

0.8761

31

Illinois

110.0

91.0

6

3

0.8748

32

Louisville

106.8

88.5

4

3

0.8734

33

West Virginia

112.2

93.2

8

5

0.8699

34

Texas

113.0

94.6

4

4

0.8611

35

Wichita St.

113.4

95.8

3

3

0.8491

36

BYU

114.2

96.8

3

4

0.8450

37

Temple

114.4

97.0

5

3

0.8447

38

Memphis

109.4

92.9

3

5

0.8428

39

Long Beach St.

113.7

97.1

3

5

0.8345

40

California

114.8

98.6

3

4

0.8261

41

Marquette

108.4

93.3

3

4

0.8219

42

Belmont

116.0

100.0

2

4

0.8218

43

Mississippi St.

108.5

93.7

3

3

0.8174

44

Dayton

114.4

98.9

4

3

0.8174

45

Vanderbilt

104.6

90.5

4

3

0.8160

46

Marshall

111.4

96.5

3

4

0.8140

47

Creighton

115.0

100.1

4

2

0.8057

48

Iowa St.

112.6

98.4

2

3

0.7980

49

Xavier

103.2

90.4

3

4

0.7952

50

Cincinnati

105.3

92.5

4

2

0.7898

But while it is ridiculous to throw out games, let’s take it one step further and apply the Jay Bilas test. Who has gone on the road and beaten quality teams in true away games (i.e. neutral sites don’t count)? At this point almost no one has multiple true road wins against quality competition:

True Road Wins vs Top 100

Adj Off

Adj Def

W

L

Pyth

Baylor

123.7

84.2

3

0

0.9809

Michigan St.

121.0

83.3

2

0

0.9787

Syracuse

124.9

90.0

2

0

0.9662

Vanderbilt

113.9

86.5

2

1

0.9437

Missouri St.

109.2

86.6

2

2

0.9152

Georgetown

115.7

95.1

2

1

0.8823

Creighton

114.1

95.5

2

1

0.8610

Cincinnati

108.4

97.6

2

1

0.7456

Iona

114.1

104.8

2

1

0.7049

By the end of the season, true road wins might be a nice metric, but right now there haven’t been enough of these games to learn anything. (I mainly bring this up because a lot of people have been criticizing Indiana for doing all their damage on their home floor, but that’s just silly. Let’s see how the Hoosiers are doing in February before we make that pronouncement.)

Realistically, there are a lot of ways to solve the “Wisconsin problem” without throwing away information. Systems that put less weight on margin-of-victory get around this problem. If you want more weight on outcomes and less weight on margin-of-victory, look no further than the LRMC system Ken linked to, or the Sagarin ELO chess. Another solution is to cap margin-of-victory at a certain level so that Middle Tennessee’s close games count, but Wisconsin's blowouts get less weight.

But if you want to calculate Offensive and Defensive ratings and truly understand “why” teams are winning or losing, then you can’t cap margin of victory. And that’s why Ken’s model won’t and shouldn’t change. His numbers are as much about understanding why teams are good or bad as finding the perfect ranking.

Overall, remember that margin-of-victory is just one piece of information that should be used to evaluate teams. I happen to think it is among the most important pieces of information, but it isn’t the only piece of information. I’ve said this for years and I’ll continue to say it. The beauty of the NCAA tournament is that the selection committee is made up of people. And unlike a formula, people can adapt and consider different pieces of information that may be important to evaluating a team. The NCAA committee may be less consistent than a formula, but it is more flexible. And since odd results crop up every season, that flexibility is incredibly valuable.