Rankings Math Questioned

A Hong Kong university is accused of underreporting enrollment numbers to boost its faculty-student ratio and ranking. The university says it is commissioning an independent audit but emphasizes that there are differences in data definitions.

By

The City University of Hong Kong has been accused of underreporting its enrollment to the British ranking company QS, resulting in a lower student-to-faculty ratio and potentially a higher ranking. The number of City University students reported by QS is more than 30 percent less than the number reported by the Hong Kong government.

In response to the allegations, which were first reported in the Hong Kong press and reportedly made by officials at other Hong Kong universities, City University is commissioning an audit. However, the university’s director of institutional research, Kevin Downing -- who is listed on QS’s website as a consultant to QS and member of the ranking company’s advisory board -- said via email that variations in enrollment figures reported to governmental agencies and rankings companies are “a common phenomenon in rankings exercises” and a “natural consequence of the different definitions of student number required for data submission by institutions for the different purposes of these separate exercises.”

Hong Kong's University Grants Committee (UGC), a governmental body, reports that the City University of Hong Kong enrolled a total of 14,325 students in 2016-17 in UGC-funded programs -- the agency reports the same 14,325 figure for both head count and full-time-equivalent enrollment -- while QS reports the university enrolls 9,240 students. City University’s enrollment as reported by QS has decreased in recent years -- an archived version of the QS site from 2015 shows a reported enrollment of 10,245 -- while the UGC figures show increases in the enrollment of students in government-funded programs (the UGC figures only reflect students who are enrolled in programs funded by the UGC).

Faculty-student ratio accounts for 20 percent of a university’s QS rankings, and is one of six indicators -- along with academic reputation, employer reputation, citations per faculty member, and the ratios of international students and staff, respectively -- that go into calculating the ranking. The City University of Hong Kong has risen quickly in the QS rankings, climbing from No. 108 to its current position of No. 49 in three years. Its score on the faculty-student ratio dimension has improved from 75.4 to 83.6 in that time, out of a maximum score of 100, though much of the recent gain in the university's ranking would seem to be attributable to an approximately 40-point gain on the citations-per-faculty-member metric posted by City University after QS made a methodology change to account for different publishing expectations across fields in 2015.

The City University of Hong Kong, like many universities, boasts of its positions in the QS and other major world university rankings systems on its website. The university has also participated in a rating system QS offers universities for a fee, called QS Stars, and received the maximum five stars.

In an email, Downing said that QS’s definition for student enrollment data is different than that of the UGC “because of its different purposes.” He noted that the UGC reports the same figure for City University's head count and FTE “because the UGC define student number to include everything, including subdegree students.”

By contrast, he said, “QS does not include subdegree numbers in its definition. This can have a significant impact on the FTE [full-time equivalent] number submitted by institutions, depending on the size of their subdegree operations, and whether institutions are expanding or shrinking these programs. It is well known that City University of Hong Kong has been progressively and continuously reducing its subdegree programs alongside reducing its self-financed student enrollment under the requirements of its University Strategic Plans and Academic Development Plans, and also specifically to free up space on a chronically overcrowded campus.” UGC data shows that the number of subdegree students in government-funded programs at the City University has been relatively flat for the past four years at a little more than 900 students. The UGC reports that City University enrolled 903 subdegree students,12,424 undergraduate students and 998 graduate students in government-funded programs in 2016-17.

Downing added, “The different academic approach and operating modes (e.g., part-time/full-time) adopted by different institutions to cater to different student needs or study modes is yet another factor explaining the variation in the data on student numbers. All data submitted to QS by universities is thoroughly audited by the QS ranking body and the QS rankings are themselves audited by a specialist ranking agency.”

“City University is now commissioning its own independent audit using one of the big four companies,” said Downing. A psychologist and the editor in chief of the journal Educational Studies, Downing said that he has served on QS’s Academic Advisory Board in his personal capacity and is chair of the QS MAPLE International Academic Advisory Committee, which organizes an annual conference, roles that are noted on his City University biography. “It is part of the responsibility of any academic to engage with outside bodies and serve on committees in areas related to their particular academic or professional interests,” Downing said in regard to his QS affiliations.

Asked whether QS is taking any action in regard to the allegations that City University underreported its enrollment data, Ben Sowter, QS’s research director, replied via email, “QS invites institutions to submit data directly into our proprietary data collections systems. This goes through a thorough validation check where we compare numbers against our historical records, the university’s public records and any third-party or government sources, like the UGC. Where we see substantial divergence from any or all of these validating records we seek clarification. It is not unusual for definitions to vary, either in terms of what counts or does not count as a student, or as to how FTE is calculated.

“We have repeatedly explored divergences between official data and that supplied by universities and interrogated the universities in question, including City U. We have always been satisfied with their robust and sophisticated explanations of how our data definitions differ from those used elsewhere,” Sowter said.

“We have learnt that the president of City U has commissioned an independent audit from one of the Big Four accounting firms to further reassure their stakeholders about the validity of their data submission to QS,” Sowter continued.

“It is only natural that these divergences would prompt question[s] and QS always takes such enquiries very seriously. QS has a zero-tolerance policy on manipulation of our outcomes; if any institution is revealed to be deliberately supplying incorrect or inaccurate data with a view to improving their outcome, they would be removed from the ranking altogether for that year and identified in a public statement.”