League Ratings

The following is an attempt to attach a numerical value ot the relative strength of the leagues in the pyramid, based on their relative performances in the FA Cup, FA Trophy and FA Vase in the 2003-4 season.
The first table shows each league's rating, along with its level, and the number of matches used to calculate that rating.
The rating of each league can be used to work out the chance of a team from that league beating a team from another league. This works as follows:

Subtract the higher rating from the lower.

Look up this number in the upper row of the second hand table (or the nearest number to it)

The percentage given in the second lower row is the liklihood of the team from the stronger league beating the team form the weaker league.

Subtract this number from 100% to get the chance of the weaker league's team beating the stronger league's team.

The rather complicated details behind how these ratings are calculated are given after the two tables below. It is also woth noting that the rating of a league with fewer than 10 matches is almost always the result of the performances of only one or teams form that league, and so the ratings of such leagues should not be considered to be precise.
Key: P - Matches played; R - Rating.

League

Levelto 03/04

2000/1

2001/2 *

2003/4

2004/5

P

R

P

R

P

R

Level

P

R

Premiership

1

63

100

56

100

48

100

1

53

100

Division 1 / FLC

2

53

94.47

41

90.79

46

92.90

2

39

90.98

Division 2 / League 1

3

72

93.84

57

85.57

54

92.27

3

56

85.11

Division 3 / League 2

4

55

91.18

51

87.84

50

93.10

4

46

85.32

Conference

5

130

89.64

77

81.71

96

90.54

5

90

81.84

Conference N

6

107

78.16

Conference S

6

104

78.65

Isthmian Prem

6

126

85.62

60

79.02

111

86.14

7

108

79.04

NPL Prem

6

127

84.61

54

75.03

99

85.63

7

82

73.49

Southern Prem

6

116

85.85

53

75.09

91

83.24

7

95

76.47

Isthmian 1

7

114

80.99

38

70.88

8

90

74.29

Isthmian 1N

7

106

79.53

Isthmian 1S

7

97

82.43

NPL 1

7

111

83.59

34

69.57

89

83.30

8

95

75.39

Southern East

7

103

81.51

38

72.31

94

80.39

8

85

73.65

Southern West

7

112

84.96

37

69.71

93

82.20

8

82

72.39

Comb Co Prem

8

65

75.78

32

70.52

68

73.36

9

56

65.14

Eastern Prem

8

112

80.40

57

74.54

85

77.01

9

81

72.01

Essex Senior

8

27

73.08

15

65.76

40

79.83

9

44

66.80

Hellenic Prem

8

54

78.96

30

67.85

62

79.29

9

66

69.11

Isthmian 2

8

119

80.30

56

74.26

60

80.78

9

65

71.18

Kent

8

64

76.14

24

67.82

61

78.04

9

68

72.06

Midland Alliance

8

111

79.61

43

68.52

71

78.60

9

93

71.31

Northern 1

8

127

85.13

63

77.22

90

80.45

9

92

69.58

Northern Co E Prem

8

95

81.96

45

72.43

82

77.16

9

79

73.24

NW Counties 1

8

110

80.72

54

74.97

105

76.09

9

95

69.58

Spartan SM P

8

74

75.54

39

71.84

58

81.11

9

61

69.28

Sussex 1

8

75

74.82

41

71.06

50

75.80

9

57

65.04

United Counites P

8

79

75.50

44

71.94

73

75.77

9

74

68.44

Wessex 1

8

89

77.30

46

70.81

98

76.34

9

89

72.28

Western P

8

84

81.05

44

70.91

72

73.16

9

87

73.56

Dorset

9

1

76.53

Eastern 1

9

19

63.73

32

66.09

10

52

67.97

Hampshire League

9

4

66.66

25

83.92

Hellenic East

9

2

59.85

16

78.16

Hellenic West

9

1

61.84

4

81.79

10

2

57.80

Isthmian 3

9

103

78.92

52

70.03

Kent County

9

4

79.99

Leicester Senior

9

18

67.44

3

77.04

10

20

63.64

Northern 2

9

39

77.01

23

66.45

54

76.36

10

55

62.87

Northern Counties East 1

9

53

79.48

28

68.92

62

77.21

10

48

64.96

NW Counties 2

9

43

79.00

23

72.68

44

70.40

10

50

64.39

Spartan SM 1

9

14

67.97

6

74.17

10

7

60.04

Sussex 2

9

8

68.29

8

67.07

26

71.61

10

25

63.21

United Counties 1

9

5

60.77

6

67.41

10

5

59.02

Wessex 2

9

10

13

65.71

West Midlands

9

2

61.94

18

79.06

10

23

66.76

Western 1

9

22

71.91

19

65.36

48

69.83

10

39

66.49

Central Midlands S

10

17

68.83

8

72.80

11

16

63.71

Devon

10

1

60.81

5

72.14

11

5

64.98

Manchester

10

1

68.08

Midland Combo

10

17

64.66

4

72.52

11

22

64.77

Midland League P

10

21

67.58

Northern Alliance

10

6

72.22

11

4

65.27

South West

10

5

79.11

10

73.97

23

78.08

11

28

69.01

Sussex 3

10

11

1

55.04

Wearside

10

1

58.92

5

73.15

11

1

59.58

West Cheshire

10

3

69.52

20

77.41

11

1

54.39

Central Midlands P/1

10

1

77.75

12

4

63.10

London Intermediate

10

5

83.81

Notts Alliance

12

3

68.52

* The results for the 2001/2 season do not include the qualifying rounds for the FA Cup, though I hope to be able to include thse soon.

Difference

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

6

7

8

% Win

50%

52%

54%

56%

58%

60%

62%

64%

66%

67%

69%

73%

76%

79%

Difference

9

10

11

12

13

14

15

16

17

18

19

20-21

22-24

over 25

% Win

82%

84%

86%

88%

90%

92%

93%

95%

96%

96%

97%

98%

99%

~ 100%

How the ratings are calculated

The results for the FA Cup, FA Trophy and FA Vase for each season were used to how the ratio of wins and loses between teams from any given pair of leagues. It was then assumed that the space of all matches follows a normal distribution with mean zero, and a given standard deviation (see below). The number of standard deviations away from the mean that the win-loss ratio reprsented was calculated. This was then declared to be the difference in rating between those two leagues, so the rating of a league with respect another league is known. The Premiership was given a rating of 100, and each league's rating is the average of its rating with respect to all other leagues. This leads to an iterative process, which fortunately converges.

There are three arbituary parameters which go into these ratings. The first is the rating of the Premiership. Changing this causes all other ratings to change by exactly the same amount, so the difference between the ratings are unchanged, and so probabilities of a team from one league beating a team from another is unchanged.
The second is the standard devitaion. It works out that when the probabilites are calculated from the differences in ratings, they remain unchanged. (You need the standard deviation to calculate the differences / probabiliites table)
The third relates to what happens when a league has perfect win (or loss) record against another league. According to the process described above, the ratings should be infinitely different. This is obviously impractical, and so a certain number of standard deviations is 'awarded' for a prefect win record (or subtracted for perfect loss record). This number was chosen to be equvilant to winning 95% of the time.