Ranking == Classification

Ranking problem can be transformed into a two-class classification problem via the pairwise transform.

Goal: Learn \( \; f : X \times X \rightarrow \{-1,1\} \).

Ranking == Regression

All we have to do is use: \( (\mathbb{R}, \geq) \)!

Obviously satisfies total ordering criteria.

But has a bit more: we have/need a metric.

Shows ranking can be a regression problem.

Goal: Learn scoring function \( q : X \rightarrow \mathbb{R} \) .

Getting it wrong: Loss Functions

Descriptive loss

Predictive loss

Ranking agreements

Zero-One Loss

Count the games where the ranking-predicted winner is upset.

Misrank Loss

If the ranking-predicted winner loses, add the difference in ranks to the loss.

Win Matrix

Season: 2012Week: 17

Ranking Agreement

Kendall tau distance is a metric that counts the number of pairwise disagreements between two rankings.

Two rankings \( R_1, R_2 \) disagree on pair \( i, j \) if:

\[ R_1(i) > R_1(j) \wedge R_2(i) < R_2(j) \]

Useful if we want to match some existing ranking, say ESPN.com Power Rankings.

Methods

Elo

Pythagorean wins

Eigenvector

Bradley-Terry-Luce

Feature engineering

Optimal Rankings

Elo

Elo

Elo ranking

Ratings of players A and B: \( R_A, R_B \)

Expected score of A while playing B:

\[ E(R_A, R_B) = \frac{1}{1 + 10^{(R_B - R_A)/400}} \]

Note that \( E(R_A, R_B) = 1 - E(R_B, R_A) \)

Elo score updates

Outcome when A plays B is \( S_{AB} \in {0, 0.5, 1} \)

\[ R_A' \leftarrow R_A + K ( E(R_A, R_B) - S_{AB} ) \]

\[ R_B' \leftarrow R_B + K ( E(R_B, R_A) - S_{BA} ) \]

\[ R_B' \leftarrow R_B - K ( E(R_A, R_A) - S_{AB} ) \]

Problems

How to set parameters?

How often to update?

Pythagorean Wins

From 1988 through 2004, 11 of 16 Super Bowls were won by the team that led the NFL in Pythagorean wins, while only seven were won by the team with the most actual victories. Super Bowl champions that led the league in Pythagorean wins but not actual wins include the 2004 Patriots, 2000 Ravens, 1999 Rams and 1997 Broncos.