Scientific Rankings

Monday, April 4, 2011

Shifting the Business School Hierarchy Part II:

The Real MBA Rankings behind the Financial Times and

US News & World Report Figures

By Philippe RUIZ, PhD and Benjamin GOURMEL, PhDc

The present article is an extension of the one published on January 21st, 2011 and titled: Shifting the Business School Hierarchy: The Real Rankings behind the Financial Times Figures. We strongly recommend our readers to read the previous article before reading this one, in particular the first four parts.

For this second issue, we have decided to scrutinize the rankings of MBA courses. Indeed, the MBA, despite recent criticism regarding its lack of specialization and relevance to the actual conduct of business, remains unquestionably the hallmark of higher business education. Besides, it is also one of the first degrees that went through the process of ranking in higher education. Therefore it seems natural to review the current ranking methodologies and to apply some sound statistics to the existing publications.

To compute the present rankings, we use the Financial Times’ (FT) MBA 2011 figures (collected yearly and published in January) as it provides the most commonly agreed upon criteria on the international scene. Therefore, no new underlying data are provided here, only differences in statistical analysis distinguish our work. We also compare our results with the latest US News & World Report (USN) findings (published in March 2011).

1- Method

The FT and USN rankings are based on the weighted average of different criteria such as salary after graduation or the percentage of international students. A weighted average is simply an average in which each quantity to be averaged is assigned a weight. These weightings determine the relative importance of each quantity on the average.

There are some problems with the weighted average that trained statisticians are aware of. The first is whether it is possible or not to add some quantities to obtain a meaningful sum. An example of a meaningless sum would be the sum obtained from the addition of 1) the number of hair on my neighbor’s head, 2) the number of trains leaving my local train station on a Sunday and, 3) the number of stars visible from my bedroom window on a clear summer night. The obtained sum is meaningless because the three numbers have nothing in common. Only things that are more or less related can be added to obtain a meaningful sum or an average. An example of an average of similar quantities is at school, when the overall average of all the courses followed by the students is obtained. In a typical business school, a Grade Point Average (GPA) is thus calculated. The addition of all the grades from different disciplines such as Marketing, Finance or Organizational Behavior is possible because those subjects somehow measure something in common. If they were not related at all, it would be impossible to compute a meaningful average.

Statisticians use the word “correlated” to indicate that two or more variables have tendency to “move together.” An example of two correlated variables in the human population is height and weight. Taller people have a tendency to be heavier (children are smaller and thus much lighter than adults) even if this relationship is not always true (there are some tall people who are lighter than some shorter people, but the bigger the difference in height, the less likely this is to be true). It is said that height and weight are positively correlated (i.e. people who are taller are usually heavier). When two variables are correlated, it is often possible to add them. However, when two variables are not correlated, it is never possible to add them to obtain a meaningful sum or an average. So the very first thing statisticians do before computing an average is to calculate the “coefficients of correlation” between the variables to find out whether it is possible or not to add them. For example, the Employed at 3 months (%) variable used in the FT ranking is absolutely not correlated with the Weighted salary (US$) variable and therefore the two quantities should never be added, even it appears “intuitively” that the two variables should be related.

Once it has been established that the variables are sufficiently correlated and that their average can be computed, another problem to address is what weights should be attributed to each variable. The FT and USN journalists seem to believe that their educated opinion can do the job, but once again, this is at the expense of sound statistical practice. In order to establish the weights when many variables are available, the correlations with all the other variables in the analysis should be obtained first by using a technique called Factor Analysis (FA). The purpose of FA is to replace a given number of correlated variables by a smaller number of uncorrelated factors. FA is an advanced technique but the basic principles are easy to understand. This technique allows to determine “trends” in variables that are interrelated, and to aggregate them into factors. The strength of the links of each variable with all the others will indicate its weight and importance in the study.

Going back to our school example above, when the GPA is obtained for the students, the GPA can be considered to be a factor because it integrates all the original variables; it gives an overall indication of the scholastic level of the students. This indicator or factor is good only if it has strong links to the original variables (strong correlations). Some courses may be more highly related to it (they may have strong correlations with the GPA) while others may be only vaguely related to it (they may have low correlations with the GPA). If there are many courses highly related to it then the factor is strong, if not, the value of the factor is dubious.

Let’s now suppose that Marketing has a strong relationship with the GPA (we are just making this up) and Finance has a weak relationship with the GPA, then marketing should be considered more important than finance and therefore it should be weighted more in the average than finance.

To sum up, if the original data of the FT or USN can be considered relevant and reliable, there are two main problems with these journals’ number crunching that make their study statistically mistaken and their rankings incorrect: 1) they did not check whether the numbers obtained could be added (we found that most of the variables do not share enough in common to compute an overall mean) and,2) the weights they have picked are arbitrary and subjective, not the outcome of a mathematically objective optimization procedure.

2- Results

We have thus applied correlation and factor analysis to their data and we have discovered two main dimensions, not just one as believed by the FT or USN. It is impossible to add those dimensions/ factors because they are uncorrelated (they share nothing in common). We are now going to explain them and extract the correct corresponding rankings.

The first factor includes the following variables (in decreasing order of importance): Alumni recommend rank, Weighted salary (US$), Placement success rank, FT research rank, Aims achieved rank, and Value for money rank. This factor reflects what we traditionally mean by business school quality: added value or return on investment (ROI) but also the reputation of the school and the quality of its research.

Statistical Ranking of the Financial Times MBA 2011 (in decreasing order of ROI score)

School name

ROI score

ROI rank

FT rank

ROI-FT rank difference

USN rank

ROI-USN rank difference*

Stanford University GSB

71,55

1

4

-3

1

0

University of Pennsylvania: Wharton

70,13

2

1

1

3

-1

Northwestern University: Kellogg

68,92

3

21

-18

5

-2

Columbia Business School

68,77

4

7

-3

9

-5

University of Chicago: Booth

67,74

5

12

-7

5

0

Dartmouth College: Tuck

66,53

6

18

-12

7

-1

New York University: Stern

66,25

7

15

-8

10

-3

MIT Sloan School of Management

65,82

8

9

-1

3

5

University of California at Berkeley: Haas

65,63

9

25

-16

7

2

London Business School

65,38

10

1

9

n/a

n/a

Duke University: Fuqua

65,35

11

20

-9

12

-2

Harvard Business School

64,06

12

3

9

2

9

Yale School of Management

63,30

13

15

-2

10

2

University of Michigan: Ross

62,91

14

24

-10

14

-1

Insead

62,75

15

4

11

n/a

n/a

IMD

62,24

16

14

2

n/a

n/a

Cornell University: Johnson

61,63

17

30

-13

16

-2

Iese Business School

60,48

18

9

9

n/a

n/a

University of Virginia: Darden

60,37

19

41

-22

13

2

UCLA: Anderson

59,82

20

31

-11

14

2

Carnegie Mellon: Tepper

58,45

21

41

-20

18

-1

Vanderbilt University: Owen

57,93

22

51

-29

28

-10

Georgetown University: McDonough

57,54

23

38

-15

25

-6

University of Texas at Austin: McCombs

57,43

24

49

-25

17

3

Emory University: Goizueta

57,26

25

38

-13

23

-2

HEC Paris

57,26

26

18

8

n/a

n/a

Indian Institute of Management, Ahmedabad (IIMA)

56,16

27

11

16

n/a

n/a

University of Southern California: Marshall

55,87

28

64

-36

21

1

Rice University: Jones

55,37

29

44

-15

34

-11

Indian School of Business

54,77

30

13

17

n/a

n/a

Pennsylvania State University: Smeal

54,44

31

59

-28

40

-16

Indiana University: Kelley

54,18

32

73

-41

23

2

University of Cambridge: Judge

54,11

33

26

7

n/a

n/a

University of North Carolina: Kenan-Flagler

53,92

34

62

-28

19

7

Hong Kong UST Business School

53,59

35

6

29

n/a

n/a

Ceibs

53,01

36

17

19

n/a

n/a

Georgia Institute of Technology

52,81

37

97

-60

28

-1

University of Western Ontario: Ivey

52,57

38

46

-8

n/a

n/a

IE Business School

51,53

39

8

31

n/a

n/a

University of Oxford: Saïd

50,89

40

27

13

n/a

n/a

SMU: Cox

50,67

41

88

-47

57

-29

Rotterdam School of Management, Erasmus University

50,34

42

36

6

n/a

n/a

Cranfield School of Management

50,10

43

34

9

n/a

n/a

University of Washington Business School: Foster

49,91

44

86

-42

37

-8

Boston College: Carroll

49,57

45

74

-29

34

-4

University of Toronto: Rotman

49,47

46

46

0

n/a

n/a

Arizona State University: Carey

49,29

47

64

-17

27

4

Esade Business School

49,19

48

21

27

n/a

n/a

Wisconsin School of Business

49,05

49

63

-14

28

4

Australian School of Business: AGSM

48,96

50

35

15

n/a

n/a

Manchester Business School

48,69

51

29

22

n/a

n/a

Ohio State University: Fisher

48,58

52

72

-20

25

8

University of Maryland: Smith

48,43

53

40

13

45

-11

Brigham Young University: Marriott

48,43

54

91

-37

32

3

Imperial College Business School

48,13

55

37

18

n/a

n/a

Texas A & M University: Mays

48,06

56

44

12

32

4

Boston University School of Management

48,03

57

68

-11

34

3

University of British Columbia: Sauder

47,96

58

80

-22

n/a

n/a

University of Notre Dame: Mendoza

47,52

59

80

-21

37

1

Babson College: Olin

47,28

60

84

-24

52

-13

University of Illinois at Urbana-Champaign

47,04

61

46

15

37

3

City University: Cass

47,03

62

32

30

n/a

n/a

National University of Singapore School of Business

46,95

63

23

40

n/a

n/a

York University: Schulich

46,84

64

49

15

n/a

n/a

Thunderbird School of Global Management

46,68

65

68

-3

75

-34

Kaist College of Business

46,32

66

99

-33

n/a

n/a

University of Rochester: Simon

45,61

67

52

15

45

-3

University of Iowa: Tippie

45,55

68

64

4

40

3

Warwick Business School

45,47

69

58

11

n/a

n/a

Ipade

45,02

70

64

6

n/a

n/a

McGill University: Desautels

44,68

71

57

14

n/a

n/a

Wake Forest University: Babcock

44,35

72

78

-6

47

-3

Melbourne Business School

44,27

73

53

20

n/a

n/a

Purdue University: Krannert

44,24

74

74

0

49

-4

Incae Business School

44,03

75

77

-2

n/a

n/a

University of South Carolina: Moore

43,77

76

80

-4

54

-8

SDA Bocconi

43,16

77

28

49

n/a

n/a

University of California at Irvine: Merage

42,69

78

53

25

40

7

University College Dublin: Smurfit

41,66

79

78

1

n/a

n/a

Lancaster University Management School

41,64

80

41

39

n/a

n/a

University of Florida: Hough

41,01

81

94

-13

47

1

University of California: Davis

40,39

82

83

-1

28

21

Nanyang Business School

40,09

83

33

50

n/a

n/a

University of Georgia: Terry

40,08

84

93

-9

57

-7

Hult International Business School

39,10

85

61

24

n/a

n/a

EM Lyon Business School

39,08

86

100

-14

n/a

n/a

University of Cape Town GSB

38,63

87

60

27

n/a

n/a

College of William and Mary: Mason

38,29

88

86

2

83

-32

Pepperdine University: Graziadio

37,80

89

92

-3

n/a

n/a

Durham Business School

37,61

90

55

35

n/a

n/a

Leeds University Business School

37,39

91

94

-3

n/a

n/a

IAE Business School

36,38

92

98

-6

n/a

n/a

University of Edinburgh Business School

36,13

93

88

5

n/a

n/a

Vlerick Leuven Gent Management School

34,98

94

55

39

n/a

n/a

Birmingham Business School

33,60

95

68

27

n/a

n/a

Bradford School of Management/TiasNimbas BS

33,10

96

90

6

n/a

n/a

SP Jain Center of Management

32,64

97

68

29

n/a

n/a

Eada

31,50

98

84

14

n/a

n/a

University of Strathclyde Business School

29,78

99

74

25

n/a

n/a

Politecnico di Milano School of Management

29,02

100

96

4

n/a

n/a

* ROI-USN rank difference is computed from the rankings of US business schools only (once all non-US business schools have been removed from the FT database).

Rank difference is obtained by subtracting the original FT or USN ranks from the ROI ranks. A positive number indicates that a school has been overrated by the FT or USN whereas a negative number indicates underrating.

The second factor includes the following variables (in decreasing order of importance): International mobility rank, International students (%), International board (%), International faculty (%), Career progress rank, and International experience rank. It is clearly an international factor that plays an essential role in our changing global economy.

Statistical Ranking of the Financial Times MBA 2011 (in decreasing order of International score)

School name

International score

International rank

FT rank

International - FT rank difference

USN rank

Insead

71,78

1

4

-3

n/a

IMD

71,74

2

14

-12

n/a

Hong Kong UST Business School

70,54

3

6

-3

n/a

London Business School

68,58

4

1

3

n/a

Iese Business School

67,55

5

9

-4

n/a

IE Business School

64,95

6

8

-2

n/a

SP Jain Center of Management

64,15

7

68

-61

n/a

City University: Cass

64,06

8

32

-24

n/a

Esade Business School

64,04

9

21

-12

n/a

HEC Paris

63,68

10

18

-8

n/a

Hult International Business School

63,42

11

61

-50

n/a

University of Oxford: Saïd

62,50

12

27

-15

n/a

University of Cambridge: Judge

62,12

13

26

-13

n/a

Australian School of Business: AGSM

60,61

14

35

-21

n/a

Cranfield School of Management

60,57

15

34

-19

n/a

Durham Business School

60,32

16

55

-39

n/a

Nanyang Business School

60,10

17

33

-16

n/a

Manchester Business School

59,93

18

29

-11

n/a

Imperial College Business School

59,81

19

37

-18

n/a

Eada

59,67

20

84

-64

n/a

Melbourne Business School

59,62

21

53

-32

n/a

Rotterdam School of Management, Erasmus University

59,28

22

36

-14

n/a

Warwick Business School

59,00

23

58

-35

n/a

Birmingham Business School

58,48

24

68

-44

n/a

Lancaster University Management School

58,45

25

41

-16

n/a

National University of Singapore School of Business

58,38

26

23

3

n/a

SDA Bocconi

57,79

27

28

-1

n/a

University of Edinburgh Business School

57,63

28

88

-60

n/a

Vlerick Leuven Gent Management School

57,44

29

55

-26

n/a

York University: Schulich

57,37

30

49

-19

n/a

Ceibs

56,38

31

17

14

n/a

University of Strathclyde Business School

55,70

32

74

-42

n/a

EM Lyon Business School

55,42

33

100

-67

n/a

Incae Business School

55,29

34

77

-43

n/a

University of Cape Town GSB

55,28

35

60

-25

n/a

MIT Sloan School of Management

54,95

36

9

27

3

Bradford School of Management/TiasNimbas Business School

54,48

37

90

-53

n/a

Stanford University GSB

54,23

38

4

34

1

University College Dublin: Smurfit

53,72

39

78

-39

n/a

University of Pennsylvania: Wharton

53,72

40

1

39

3

Columbia Business School

53,70

41

7

34

9

University of British Columbia: Sauder

53,62

42

80

-38

n/a

University of Toronto: Rotman

53,53

43

46

-3

n/a

Harvard Business School

53,14

44

3

41

2

McGill University: Desautels

52,84

45

57

-12

n/a

Politecnico di Milano School of Management

51,78

46

96

-50

n/a

Thunderbird School of Global Management

50,92

47

68

-21

75

Leeds University Business School

50,73

48

94

-46

n/a

IAE Business School

49,74

49

98

-49

n/a

New York University: Stern

48,92

50

15

35

10

Yale School of Management

48,76

51

15

36

10

Indian School of Business

48,44

52

13

39

n/a

Dartmouth College: Tuck

48,22

53

18

35

7

Northwestern University: Kellogg

48,20

54

21

33

5

University of California at Berkeley: Haas

47,97

55

25

30

7

University of Chicago: Booth

47,94

56

12

44

5

University of Western Ontario: Ivey

47,62

57

46

11

n/a

Indian Institute of Management, Ahmedabad (IIMA)

47,11

58

11

47

n/a

University of Michigan: Ross

47,06

59

24

35

14

Cornell University: Johnson

46,82

60

30

30

16

Duke University: Fuqua

46,52

61

20

41

12

Babson College: Olin

45,27

62

84

-22

52

UCLA: Anderson

44,93

63

31

32

14

University of Rochester: Simon

43,91

64

52

12

45

University of North Carolina: Kenan-Flagler

43,89

65

62

3

19

Georgetown University: McDonough

43,21

66

38

28

25

Carnegie Mellon: Tepper

43,10

67

41

26

18

Ipade

42,88

68

64

4

n/a

University of Virginia: Darden

42,58

69

41

28

13

University of South Carolina: Moore

42,46

70

80

-10

54

Boston University School of Management

42,43

71

68

3

34

Pennsylvania State University: Smeal

41,73

72

59

13

40

University of Maryland: Smith

41,67

73

40

33

45

Emory University: Goizueta

41,26

74

38

36

23

Ohio State University: Fisher

41,19

75

72

3

25

University of Texas at Austin: McCombs

41,03

76

49

27

17

Texas A & M University: Mays

40,82

77

44

33

32

Indiana University: Kelley

40,37

78

73

5

23

University of Washington Business School: Foster

40,22

79

86

-7

37

University of California at Irvine: Merage

40,05

80

53

27

40

Vanderbilt University: Owen

39,80

81

51

30

28

Purdue University: Krannert

39,75

82

74

8

49

University of Southern California: Marshall

39,40

83

64

19

21

Wisconsin School of Business

38,77

84

63

21

28

Rice University: Jones

38,21

85

44

41

34

Pepperdine University: Graziadio

37,57

86

92

-6

n/a

University of Illinois at Urbana-Champaign

37,48

87

46

41

37

Georgia Institute of Technology

37,46

88

97

-9

28

Arizona State University: Carey

37,10

89

64

25

27

SMU: Cox

37,08

90

88

2

57

Boston College: Carroll

37,03

91

74

17

34

Kaist College of Business

36,88

92

99

-7

n/a

University of California: Davis

36,88

93

83

10

28

University of Notre Dame: Mendoza

36,28

94

80

14

37

University of Iowa: Tippie

36,26

95

64

31

40

College of William and Mary: Mason

35,69

96

86

10

83

University of Florida: Hough

35,65

97

94

3

47

University of Georgia: Terry

34,91

98

93

5

57

Wake Forest University: Babcock

34,78

99

78

21

47

Brigham Young University: Marriott

33,76

100

91

9

32

International scores are T score transformations of the factor scores obtained by Principal Component Analysis with Varimax rotation.

Rank difference is obtained by subtracting the original FT ranks from the International ranks. A positive number indicates that a school has been overrated by the FT whereas a negative number indicates underrating.

3- Discussion

As it appears quite noticeably, if a sound statistical methodology is applied, neither the FT nor the USN rankings are very accurate. When we compute the average rank differences between the FT ROI ranks and our own we obtain a value of 17. This means that the average ranking mistake made by the FT is 17! The average rank difference between the USN ROI ranks and our own is 12 (this last value was corrected for range restriction because the US sample is twice as small as the world sample).

Nevertheless, our ROI results seem to match better with the results of the USN ranking. It makes sense because USN focuses mainly on the ROI sub dimensions and does not include any international information or schools. This US specificity of the USN rankings has been largely discussed and criticized, and has contributed to the success of the FT method.

The world economy is less and less US centric and international schools have an increasingly important competitive advantage over their US counterparts. This is confirmed by the very low scores of US universities on the International dimension. Indeed, the first US school - MIT Sloan School of Management – is ranked # 36 on this dimension, and this should be a reason for concern. US schools have become increasingly aware of the issue, and are working to catch up with the leading international institutions, but it seems there is still a long way to go.

Lastly, a fatal problem with the FT rankings remains the addition of two factors that have nothing in common. The FT combination of ROI elements with international elements is a Frankenstein’s statistical monster that should not exist: the ROI results are polluted by the international results and vice versa.

In terms of ROI, we have shown that Stanford is number 1 and London Business School number 10. London Business School is not number 1 as believed by the FT. In terms of Internationalism, we have shown that Insead is number 1, London Business School number 4, and Stanford number 38.The combination of a rank of 10 on ROI and a rank of 4 on Internationalism does not qualify London Business School for a 1 on anything. If it were so, then all journalists could be statisticians, and pigs could fly.

As we already underlined in our first article, the Financial Times’ ranking is biased by a fatal error, i.e. the combination of the ROI and International dimensions, which is statistically incorrect, however good their intentions were. A strong quantitative tool called Factor Analysis has allowed us to compile rigorous rankings and to validate that the two above mentioned dimensions can NOT be added.

This article has focused on the ranking of MBA courses, applying a sound methodology to the existing Financial Times 2011 data. The main outcome of the present study seems to confirm our expectations, both on publishers’ ranking methodologies and the characteristics of business school strategies. Regarding the publishers, the results of our statistical enquiry show that the methodologies used by the journalists to compile their rankings are flawed and unreliable.

As for the results of the schools, our expectations have also been met, with the US-based MBA courses being the most profitable (the first non-US school on the ROI table ranks number 10, and 13 out of the top 15 are based in the US), but seriously lagging behind when it comes to the criteria measuring the international orientation of the teachings. The majority of international schools are located in Europe (9 out of the top 10) and the first US school is ranked 36th. This should be a serious matter of concern for US schools, as we live in an increasingly multipolar world, and adaptation to other cultures and management techniques becomes imperative to thrive in different business environments.

Support independent research

Followers

Blog Archive

About Me

Philippe Ruiz is a professor of Statistics and Quantitative Research Methodology at the LSMRC.
Benjamin Gourmel is a PhD candidate at the LSMRC, his main research interests are in the field of strategy and business education.