This excellent article describes in detail a new three-tiered rating system of teaching quality at universities introduced by the U.K. government, as well as a thoughtful discussion. As I have a son and daughter-in-law teaching in a U.K. university and grandchildren either as students or potential students, I have more than an academic interest in this topic.

How are the rankings done?

Under the government’s Teaching Excellence Framework (TEF), universities in England and Wales will get one of three ‘awards’: gold, silver and bronze (apparently there are no other categories, such as tin, brass, iron or dross for those whose teaching really sucks). A total of 295 institutions opted to participate in the ratings.

Universities are compared on six quantitative metrics that cover:

retention rates

student satisfaction with teaching, assessment and academic support (from the National Student Survey)

rates of employment/post-graduate education six months after graduation.

However, awards are relative rather than absolute since they are matched against ‘benchmarks calculated to account for the demographic profile of their students and the mix of programs offered.’

This process generates a “hypothesis” of gold, silver or bronze, which a panel of assessors then tests against additional evidence submitted for consideration by the university (higher education institutions can make up to a 15-page submission to TEF assessors). Ultimately the decision of gold, silver or bronze is a human judgment, not the pure product of a mathematical formula.

What are the results?

Not what you might think. Although Oxford and Cambridge universities were awarded gold, so were some less prestigious universities such as the University of Loughborough, while some more prestigious universities received a bronze. So at least it provides an alternative ranking system to those that focus mainly on research and peer reputation.

What is the purpose of the rankings?

This is less clear. Ostensibly (i.e., according to the government) it is initially aimed at giving potential students a better way of knowing how universities stand with regard to teaching. However, knowing the Conservative government in the UK, it is much more likely to be used to link tuition fees to institutional performance, as part of the government’s free market approach to higher education. (The U.K. government allowed universities to set their own fees, on the assumption that the less prestigious universities would offer lower tuition fees, but guess what – they almost all opted for the highest level possible, and still were able to fill seats).

What are the pros and cons of this ranking?

For a more detailed discussion, see the article itself but here is my take on it.

Pros

First this is a more thoughtful approach to ranking than the other systems. It focuses on teaching (which will be many potential students’ initial interest in a university) and provides a useful counter-balance to the emphasis on research in other rankings.

Second it has a more sophisticated approach than just counting up scores on different criteria. It has an element of human judgement and an opportunity for universities to make their case about why they should be ranked highly. In other words it tries to tie institutional goals to teaching performance and tries to take into account the very large differences between universities in the U.K. in terms of student socio-economic background and curricula.

Third, it does provide a simple, understandable ‘award’ system of categorizing universities on their quality of teaching that students and their parents can at least understand.

Fourth, and most important of all, it sends a clear message to institutions that teaching matters. This may seem obvious, but for many universities – and especially faculty – the only thing that really matters is research. Whether though this form of ranking will be sufficient to get institutions to pay more than lip service to teaching remains to be seen.

Cons

However, there are a number of cons. First the national student union is against it, partly because it is heavily weighted by student satisfaction ratings based on the National Student Survey, which thousands of students have been boycotting (I’m not sure why). One would have thought that students in particular would value some accountability regarding the quality of teaching. But then, the NUS has bigger issues with the government, such as the appallingly high tuition fees (C$16,000 a year- the opposition party in parliament, Labour, has promised free tuition).

More importantly, there are the general arguments about university rankings that still apply to this one. They measure institutional performance not individual department or instructor performance, which can vary enormously within the same institution. If you want to study physics it doesn’t help if a university has an overall gold ranking but its physics department is crap or if you get the one instructor who shouldn’t be allowed in the building.

Also the actual quantitative measures are surrogates for actual teaching performance. No-one has observed the teaching to develop the rankings, except the students, and student rankings themselves, while one important measure, can also be highly misleading, based on instructor personality and the extent to which the instructor makes them work to get a good grade.

The real problem here is two-fold: first, the difficulty of assessing quality teaching in the first place: one man’s meat is another man’s poison. There is no general agreement, at least within an academic discipline, as to what counts as quality teaching (for instance, understanding, memory of facts, or skills of analysis – maybe all three are important but can how one teaches to develop these diverse attributes be assessed separately?).

The second problem is the lack of quality data on teaching performance – it just isn’t tracked directly. Since a student may take courses from up to 40 different instructors and from several different disciplines/departments in a bachelor’s program, it is no mean task to assess the collective effectiveness of their quality of teaching. So we are left with surrogates of quality, such as completion rates.

So is it a waste of time – or worse?

No, I don’t think so. People are going to be influenced by rankings, whatever. This particular ranking system may be flawed, but it is a lot better than the other rankings which are so much influenced by tradition and elitism. It could be used in ways that the data do not justify, such as justifying tuition fee increases or decreased government funding to institutions. It is though a first systematic attempt at a national level to assess quality in teaching, and with patience and care could be considerably improved. But most of all, it is an attempt to ensure accountability for the quality of teaching that takes account of the diversity of students and the different mandates of institutions. It may make both university administrations and individual faculty pay more attention to the importance of teaching well, and that is something we should all support.

So I give it a silver – a good try but there is definitely room for improvement.

Thanks to Clayton Wright for drawing my attention to this.

Next up

I’m going to be travelling for the next three weeks so my opportunity to blog will be limited – but that has been the case for the last six months. My apologies – I promise to do better. However, a four hour layover at Pearson Airport does give me some time for blogging!

No institution can now afford to ignore demographics in its strategic planning, and in no area is this more true than in plans for online learning. Traditional distance education (print-based and until very recently, online distance education) has mainly attracted older students with already some experience of higher education. It has been seen as an ideal area for continuing professional education, and the growth of online professional masters degrees is evidence in support of this belief.

However, there is now a significant shift in the overall demographics, particularly in North America. Millennials (those born in the 1980s and 1990s) are now the largest living generation in the USA.

A recent survey by Connections Education (a Pearson company) found that 55% of millennials have taken an online course (higher than any previous generation) and a majority of millennial parents (51 percent) think that high school students should be required to take at least one online course.

This may be behind the continued growth in demand for online learning at a post-secondary level. Students coming into university or college, whether millennials or post-millennials, have grown up in a world where the Internet is part of life and to whom online learning is not an exotic or marginal activity but the natural order of things.

Institutions who want to attract the best and brightest students need to be aware of this and plan for it, not just for professional education programs but also for undergraduate and two year vocational programs. This is particularly important, as I have argued many times, at a program level, where every program needs to have a rational and evidence-based policy that determines the best balance between face-to-face, blended and fully online learning within the program. In particular, to what extent can courses and programs fully exploit the benefits and avoid the pitfalls of social media in support of the learning goals?

So, does your program or institution have a plan or policy for online learning? Is it a good one, and if so what makes it good?

This article presents a short written account and a video of the adaptive learning program being introduced at the University of Central Florida (which is one of the pioneers of blended learning). It is being used particularly in large classes to personalize the learning.

There’s not much detail in this article, and the program is in its early stages, but it is useful to know that UCF is experimenting with adaptive learning, given its long history of carefully evaluating its online learning initiatives.

Those of you with Netflix may have been watching the wonderfully funny Australian series, Dreamland (called ‘Utopia’ in Australia), about a (fictitious) Australian federal government agency, the National Building Authority, that never gets any of its infrastructure projects actually started (never mind completed). Sometimes reality though is even more absurd than fiction.

In a move heavily criticised by Australia’s Electrical Trades Union, the Australian Federal government has recently shut down a two year e-learning apprenticeship program which had doubled the completion rate to 93%. Instead, the Australian government has introduced a new program that reverts to the previous model by removing the online component and now requiring apprentices to use a printed textbook. It has also substantially reduced the online mentoring role of the previous program.

Ms Daley-Boorn, one of the students enrolled in the new program, commented:

We all agree on the same things – the online training with Energise Oz made it easy for us….now going to paper-based, it’s not more work, just a different style of learning and doesn’t suit what we do in schools now. People are not going to sit down and read a textbook, that’s just not how apprentices learn.

It’s not just Australia though that goes backward with e-learning for apprenticeship training. The Industry Training Authority of British Columbia (a Crown agency funded by the provincial government) commissioned a strategy in 2008 for the expansion of flexible learning in the trades in BC, which secured both ‘hard’ provincial and ‘soft’ Federal funding commitments amounting to over $13 million.

However, after the ITA Board accepted the recommendations in the report, the project was cancelled, and the provincial funding was diverted by the ITA to ‘other projects’ – despite the fact that completion rates for traditional, campus-based apprenticeship programs were a miserable 42%. It was left to BC’s two year community colleges to fund blended learning apprenticeship programs themselves – which some, such as Vancouver Community College and Camosun College in Victoria, have done quite successfully. Nevertheless they are small pickings compared with the province-wide program recommended to the ITA.

These are the harsh realities of online learning, which still suffers from prejudice and ignorance from both government and employers alike.

This is an article on a project by the University of Mary Washington, Virginia, that enables all students to create their own academic web presence through the provision of a university-wide blogging platform. The article provides some good examples of student work done through this project, particularly in history. A recent development at UMW has been the creation of a community site that aggregates the activity of the project, including sites created and content published. The article also provides links to similar projects at Emory University and Davidson College.

It should be noted that the University of British Columbia here in Vancouver established UBC Blogs and UBC Wiki several years ago.

UBC Blogs currently has 22,785 members. Go to http://blogs.ubc.ca/support/about/ to see the many different ways UBC Blogs are being used. Choosing any one is invidious, but the first one I came across is an excellent example, UBC student Matthew Kyriakides’ essay on gentrification and social movements in Vancouver’s downtown east side.

While the blog service is aimed at individual students and faculty members, i.e. anyone with a CWL (campus-wide log-in), the wiki enables group contributions:

The UBC Wiki is a shared space for use by students, staff, and faculty at the University of British Columbia. It serves as a course repository, a personal and collaborative work space, a documentation depository, and a growing guide to everything and anything UBC. The information, resources, and links that it contains are created, expanded, and annotated by its users. It is constantly evolving and changing because every member can add to it and edit any page.

a community project started in March 2012 by graduate students at the UBC Math Department [that] features hints and worked out solutions to past math exams. The goal of the project is to provide an open and free educational resource to undergraduate students taking math courses, with a strong emphasis for first and second year courses. The provided solutions do not simply provide what the answer is, but instead focus on the processes that it takes to solve the problem. The Math Exam Resources wiki offers:

High quality content written by math graduate students. The content is reviewed and can be updated on the fly with your comments and feedback.

The ability to use the discussion pages of any page in this wiki to dialogue with us or with other students about mathematics!

I strongly recommend that you browse the UBC Blogs and Wiki sites in particular to see how social media are being integrated fully with credit-based online learning at UBC. Most UBC courses still use a learning management system that allows for ‘private’ or ‘course only’ communications, but the blogs and wikis open up the courses to the general public who can comment on blogs or participate in wikis. Linking blogs and wikis to particular courses and controlling access through the use of passwords enables a degree of quality control. Usually it is UBC students who are ‘in control’. This is a development of open education that deserves more attention.

DISCLAIMER: The views and opinions expressed by Tony Bates, and all other participants in this blog, are the individual's views and opinions and do not necessarily reflect the views and opinions of Contact North | Contact Nord.