National Research Council (NRC) Finally Releases Doctoral Program Standings

The results of the National Research Council (NRC) assessment of U.S. research doctoral programs were released in September. The much-anticipated results were based on a new approach, which produced ranges of rankings for programs in each discipline rather than a single ranking. Overall, MIT’s participating doctoral programs fared well as can be seen from the chart on the next page.

The approach of the current study addressed criticisms of the NRC’s previous study (1995) that was viewed as overly simplistic and inadequate in capturing the uncertainty associated with measurement of program quality.

In tending to these concerns, however, the new approach has elicited a new wave of criticisms, directed at both the project’s extended timeline and its methodological complexity.

The study, which was to be more heavily reliant on quantitative, objective measures, was conducted using data that was in many cases from the 2005-06 academic year. From an expansive data set compiled after institutions invested a great deal of time and effort in preparing submissions, the NRC study committee selected 20 key program characteristics (variables) and developed ranges of rankings from these data using two distinct approaches. The ranges were intended to represent – at a 90% confidence level – the range within which the program’s true ranking falls. Click here for an explanation of the 2010 NRC Methodology.

The first approach, for the survey-based (S) rankings, was based on how faculty weighted – or assigned importance to – the 20 characteristics that the study committee determined to be factors contributing to program quality. The weights of characteristics varied by field, based on faculty survey responses in each of those fields.

The second approach, for the regression-based (R) rankings, depended on the weights calculated from faculty ratings of a sample of programs in their field. These ratings were related, through multiple regression and principal components analysis, to the 20 characteristics that the committee determined to be factors of program quality.
In both approaches, the resulting weights were then applied to data corresponding to those characteristics for each of the programs in the field.

Taking the first MIT program from the chart as an example, Aeronautics & Astronautics was ranked between 7th and 16th out of 31 total participating programs using the survey-based (S) approach, and between 2nd and 5th using the regression-based (R) approach.

In total, 44% of MIT’s participating programs were ranked as high as 1st in either the R or S rankings (see M.I.T. Numbers [1]), and fully 81% of MIT’s participating programs were ranked 3rd or higher in either the R or S rankings (see M.I.T. Numbers [2]).