19 September 2009

Sometimes Dr Grumble's patients seem very surprised that he is not aware of some latest miracle medical breakthrough. This is usually because Dr Grumble does not read the Daily Mail.

Recently Dr G was asked to comment on a story about a wonderful new operation being conducted on the other side of the globe for a common problem. A journalist wanted to know if the operation was available in the UK. The hack was unable to say what the operation was. He hadn't read the scientific publication and seemed incapable of accessing it. So Dr Grumble read it for him. The truth was nothing like the stories being reported by other journalists who, presumably, also cannot have read the original publication. Sometimes journalists do not want the truth to get in the way of a good story but in this case the journalist listened to Dr Grumble and no article appeared. The Daily Mail has now gone up in the Grumble estimations.

It is very unfair to the public to tell them some wonderful new operation is available when the reality is something different. Desperate patients are vulnerable. When it comes to matters of medicine, people tend to believe newspaper articles which is unwise. They are even more likely to believe their friends which is equally unwise. It is something to do with trust. Building up trust takes time. Perhaps that's why patients over and over again say they like to see the same doctor. Doctors certainly find it easier to see the same patient. The trust that has been built up is part of this.

Knowing the very latest about scientific research is not essential to the practice of medicine and is even less important to patients. How many times have you read about a cancer breakthrough on the front pages? Things that initially appear to be wonderful, on reflection, can turn out to be not quite what they seem. Altering practice on the basis of the very latest publication is not always wise. The most junior of doctors may want to do this. Their problem is that they have not lived long enough to have seen things go in full circle.

Some journalists overestimate the importance of scientific news. Here's a sentence from a recent editorial in New Scientist:

HOW would you feel if you discovered that your child's doctor was unaware of recent findings in neuroscience? It's likely you'd be worried.

Hmmm. The article goes on to have a go at teachers who are apparently pretty ignorant when it comes to recent findings in neuroscience. Dr Grumble is a teacher and recently he has become interested in research into teaching methods. Being initially very ignorant himself on the topic he was surprised at the lack of an evidence base for the different ways you can teach. At first he thought that medicine must be way ahead of teaching with the concept of controlled trials forming an evidence base to support best practice. But the reality is that most medical decision making cannot be read off from an evidence-based protocol. And when it comes to exploring different methods of teaching it turns out that it is very difficult to do a controlled trial. The reason is that if you compare two teaching methods the students that are given the poorer method tend to plug knowledge gaps with work outside the classroom making it very difficult to show a difference between groups.

Journalists can be a bit harsh on professionals. "I blame the teachers" is their populist clarion call. It is a shame New Scientist should stoop this low. You can drive a car from A to B without knowing what is under the bonnet. You can be a good teacher without being familiar with the latest research. And your doctor could look after your child while being unaware of "recent findings in neuroscience."

Posted by
Dr Grumble

21 comments:

I'm a final year medical student and my partner is a teacher and I'm often taken aback at the lack of evidence used in teaching methods. I understand the difficulties in controlled trials, but many of those difficulties exist in medical trials. Several solutions are also possible. It's not really acceptable to say that the trials are too hard to do therefore we shouldn't bother.

I agree that New Scientist is stooping low with it's hyperbolic copy but it's general point is a fair one. Speaking to teachers it's clear that hardly any read research journals. Doctors risk being found negligent if they don't keep abreast of the latest important developments. It seems fair that teachers should be held to a similar standard.

The GTC should actively encourage teachers to subscribe to a research journal (they could even be provided by the unions, much like the BMJ comes with BMA membership).

Michael, Dr Grumble was similarly shocked at the lack of research in teaching methods and, despite the difficulties, it is possible to do trials. There have been some on medical students. There are though papers saying that it is essentially not possible. I agree that it is for us to prove them wrong.

The New Scientist article does not really address this which is the real issue. Understanding the brain will only guide you. What is really required if it is possible is proper controlled research into teaching methods.

Dr Grumble, indeed, the New Scientist solution, that teachers should better understand the brain, could prove more harmful than beneficial. How many drugs were used before the days of controlled trials because pharmacologically, it made some theoretical sense, but in practice either didn't work or had some horrendous side effects (see Cocaine as treatment for “a furred tongue in the morning, flatulence, [and] whitening of the teeth.”).

Expecting teachers to learn neuroscience and make assumptions about how to apply it could be equally dangerous. As you rightly concluded, trials are what is really required, not abstract theory.

"Michael, Dr Grumble was similarly shocked at the lack of research in teaching methods and, despite the difficulties, it is possible to do trials."

.........

There's loads of current research into teaching and learning methods. Perhaps you are looking in the wrong place. Educational Psychology journals would be a good place to start. Also check out publications from the Institute of Education.

People learn by reading the book in the month before the exam - all the educational theory and lectures and "PBL sessions" and seminars are largely for show and to keep said educationalists in a well-paid job!

Working academics generally get rather frustrated with the educationalists, since a vast amount of the research into teaching methods all seems to reach the same conclusion, "different teaching methods work best for different students/pupils".

As the typical group taught includes people who would probably (if one accepts the conclusion above) be best suited by different teaching methods, what exactly is one supposed to do? The pragmatic answer (at least in the Univs) is usually to try and vary the style a bit so that there is a bit of something for everyone.

It is generally a recurring complaint of working teachers (in both schools and Univs) that the educationalist academics rarely ask the question "How useful will the results of this study actually be to those doing the teaching?" The much-missed Ted Wragg was a bit of an exception to this in the pre-18 education area, but I am not sure he has ever had an equivalent for University education, at least in the UK.

Another difficulty with studies into teaching methods is that they tend (often by necessity) to be quasi-epidemiological and retrospective rather than interventional, prospective and randomised. Thus there are always endless confounding factors, plus the usual "Does correlation equal causation?" problem.

Finally, talking of teaching methods questions that one would love to have an answer to but which (I predict) will never get answered, how about that medical hardy perennial de jour:

"Are traditional didactic lecture-based courses, or problem based learning, a better way to teach medical students?"

About 10 years ago I was told that we all had to be trained in PBL because it was the future. I dutifully turned up and learnt how to do it. My first question, given the pressure that we were under to teach in this way, was whether or not there was any evidence that this was a better way to learn. Of course there wasn't and it seems there still isn't. So I then wondered why it was we were wasting all that time learning the method. As I recall, it came from Canada or was it the US and, as you know, what they do over there we always have to do a bit later. Whether there is any evidence that they are right or not never enters into it.

I dutifully taught the students using PBL. Except that I didn't do the teaching. The method seems to involve those that don't know a subject teaching others who don't know a subject. For the teacher it seemed extremely easy because all you did was sit back and facilitate.

Because I had to do very little work and thinking myself it gave me a lot of time to observe the students and the one thing I did find out very readily from the students talking to each other was the depth of their ignorance - exactly what they did and did not know - which sometimes surprised me and it also became very obvious which were the best students.

After initial very great enthusiasm with teachers like me being forced into PBL the powers-that-be rather let it drop. In some medical schools it was even more widely adopted and the feedback I had from graduates from these places was extremely negative.

There may or may not have been lots of "research" published on PBL but it would not meet the standards of the methods used in even a moderately good clinical trial yet these things are forced upon us.

I know I can teach well at the bedside. I know I can deliver things to the students that they cannot learn from books or each other. That is obviously the best way for somebody like me to use my time and I greatly resent those that told me otherwise without any more evidence than I have for my own belief in what I have to offer as a teacher.

Unfortunately there are a lot of these ex cathedra policies in the world in which we live. One knows not who instigates them and why we all follow like sheep when we all know the policies are misguided. If only those in charge had more common sense and an understanding of scientific method.

"As I recall, it came from Canada or was it the US and, as you know, what they do over there we always have to do a bit later."

You are probably thinking of McMaster Univ in Canada, Dr G. They were certainly one of the first PBL schools. Maastricht in the Netherlands was another pioneer, and Newcastle (I think) in Australia also springs to mind, but I think McMaster started it all.

Maastricht are still using PBL, I'm pretty sure, but there were rumours out of Canada few yrs back that McMaster were re-thinking because their grads were having problems getting into residency programmes in the US. Could be just a nasty slur, though.

Your commments about whether (largely) "silent" PBL tutoring is a sensible use of the time of folk like yourself are, er, widely echoed by other clinicians I know.

In fact I strongly suspect that a number of the PBL schools which started out telling the "facilitators" to "say nothing" have shifted over the years to "intervene, guide and correct" - though this is not always obvious officially. And of course if the students seem to want teaching, and the staff member in the PBL room would prefer to teach, then you can guess what is actually likely to happen.

I think even the PBL-lies would mostly accept that PBL was "an idea whose time had come" (as they see it) by the early to mid 90s, rather than it being backed by anything much in the way of actual evidence.

As I remember it from my basic scientist's perspective the key "steer" for PBL in the UK schools that adopted it was widely understood to be the original "Tomorrow's Doctors", together with, or encompassing:

- the perceived need for students to do more communicating and group work as part of the learning;

- the idea that there needed to be less fact-loaded curricula (often interpreted as less lectures, esp. in basic sciences);

- the idea that exposure to clinical stuff should start in year 1 (again, widely interpreted as "no more of this old first-2-yrs-is-basic-sci model");

- the idea that curricula should emphasise "skills to acquire" from very early on, widely interpreted to mean OSCE training type stuff, which replaced the old lab classes that went with the old basic science rich yrs 1+2.

Now, some cynics might say that the loss of "mainly basic science" yrs 1+2 was attractive to medical schools as it meant they would no longer have to make sure they had enough physiologists on staff to teach the physiology, enough pharmacologists to teach the pharmacology, enough microbiologists to teach the microbiology etc etc. For a jaundiced view of this type from one medic I know, see here (sorry, largish PDF)

Well, the way PBL is done at my place has certainly "adapted" over the years the course has been running, Dr G - at least in my opinion. There have been some designed changes, but there has also been a more general "accommodation" of practise to what the students seemed to need. In fact, if you get the people who actually run the courses together and then compare what is actually in the courses, and taught (using the word a bit loosely) by what methods, the difference between notionally full-on PBL, and notionally "integrated" (like at King's) is actually very little. Of course, King's has a good dose of PBL these days.

I recall one of the people who ran a graduate entry PBL course at St Georges Hospital saying in the BMJ a few years ago that he thought PBL was a good method for graduate students, but not really suitable for undergrads. I don't think I would go that far, but many tutors would say the method works better for the better students - or if you prefer, the ones who are highly self-motivated and pro-active, pretty organised, who have already got (or been inculcated with) good work habits, and who are confident enough to be prepared to speak up.

Now, one sometimes gets told that any entrant for medicine should be all of those things I just described, so therefore any medical school entrant should be able to cope with a PBL course. And the PBL courses do not have a higher drop-out rate than the trad ones, as far as I know, so the students do cope. But I have always had the sense that the better (as in most intellectually gifted and more "engaged") students get more out of PBL curricula than the people at the lower end of the year group. Of course, one of the arguments you sometimes hear for PBL is that the less switched-on should be able to see, in the group sessions, what they need to do by seeing what the rest of the students are doing!

Perhaps I am being old-fashioned, but I think it is unrealistic to ask every single 18 or 19 yr old entrant to a medical programme to be a hyper-motivated mini-swot. They are a bunch of (mainly middle class) kids straight out of school. Some are really into the work. Some are less so. Whether that really predicts which ones will ultimately be good doctors I rather doubt. Like many older University staff, I see University as a place where students' sense of "vocation" presumably develops along with their knowledge AND their gradual realisation of what they have signed up for. Expecting everyone to have all this in place when barely out of school strikes me as wholly unrealistic.

Going back to the courses, there is also an argument one sometimes hears that graduate entrants with science backgrounds get a lot more out of PBL. Partly it is greater maturity and "ability to think", but obviously they also already have a basic knowledge framework to "build" the clinical stuff, and the basic science-clinical knowledge connections, on to. In some ways I think graduate entry PBL courses rather resemble the "standard" US medical degree system, which of course is exclusively grad entry. The Univ of Sydney is another example of a place that went to PBL but at the same time went to graduate-only entry.

Sorry for mammoth screed - part 2 follows, separate due to HTML character limit (!)

-------------------------------------

It is hard to generalise about the PBL medical systems in the UK, and also to say what the overall experience has been, because many things have changed over the years the courses have been running - another thing that would make "trials" a bugger to construct. Apart from "teaching practice" adapting to what seems to work best (which is a good thing), there have been changes in the composition of the teaching staff and the student body.

Most medical schools probably have less "subjects related to medicine" type teaching staff, and more molecular cell biology "genes and proteins" types, in the bioscience departments now than in the mid 90s. The argument about whether tutors in PBL should be "experts" (which in the early-years PBL context really means "able to teach and understand broad-brush human biology") is another long-standing one, though again most tutors think that the more you know, the better it all works. A good "crib book" helps, of course. Some PBL schools (not ours, I should say) apparently used people like medical librarians and soc sci / humanities people as PBL tutors, which I think sounds crackers.

There have also been changes to the types of students recruited. The first two or three (or maybe even five) years of the PBL programme where I work had students who were effectively “fully informed consented volunteers” for a big experiment. In those first three years, every sixth student was a graduate entrant, and nearly all of them were deeply committed to PBL. But now we have year groups where up to 20% of the students (sic) will tell you that they didn't really want to do a PBL medical course but "this was where I got offered a place". This presents all kinds of new problems. [Not to mention making one want to ask "So why did you apply for a PBL school?"].

The GMC like to make approving noises about "diversity" (in course style), and it is a standard view among staff that students should be doing a course that they like and that suits them if they are to get the most out of it. But here we have people doing PBL unwillingly, which is hardly good for them, or the other students, or for the medical school they are at. What we do about this I don't really know. I am not sure whether "trad" medical courses contain similar numbers of people who wish they had done a PBL course. If they don't, one might think this was saying that the overall proportion of UK medical school places that are currently in PBL-heavy schools is too high.

PS I had forgotten the debate about PBL and related issues in the BMJ comments thread that followed an article by the Dean from Bristol (a very trad-minded school) a few years back.

The most interesting bits are not the article itself, but the comments thread where many of the pro and con arguments are raised, including the one about whether there is any research that actually bears on all this.

Anyway, I'll shut up now as I'm sure most people have stopped reading.

You made some very valid points. As an undergrad applicant straight out of an under performing state college, when I applied to uni I had no idea what PBL was! (As it happens I ended up at Newcastle which only uses PBL sparingly.) The idea that all applicants know in advance the format and are able to make a decision in advance about which format suits them is absurd.

At Newcastle, PBL is used in the later years of the course and our facilitators are generally teching fellows or consultants which means we rarely miss any important bits of information. However, from a personal perspective, I generally end up very knowledgeable about the bit of the subject I teach, and less so about what is taught to me. At least with normal teaching my knowledge is even.

Most students I know hate PBL, but it isn't easy to tell if that's just because you have to do more prep work! I see the difficulties with trialling the different methods but there will always be a way. Given the investment into PBL I think students are owed some evidence that their tuition fee money is being spent reasonably.

Hi Michael. Good point about school leavers not really knowing what is what in terms of course teaching style, thanks. I don't think the Univs always help as much as they could, as some are a little "opaque" about what teaching methods they use, at least in the prospectus. Though one would hope that after visiting the Univ for interview a student would have been given a good idea what the course was going to be.

I suppose your point could be read as another argument for PBL-heavy set-ups being graduate entry, since then there would likely be no confusion and hence less/no students ending up on "the wrong kind of course".

I agree 100% about it being sensible to want to trial / test the different curriculum styles as far as possible. I was just noting that it would be difficult to do - especially in a "head to head comparison" way. More cynically, one often hear jokes to the effect that a lot of people don't want to know the answer (to what works best) in case it isn't the answer they want. I couldn't possibly comment.

As a general point, PBL is not ALL that happens in PBL-based medical curricula. A typical "early years" week in a PBL system might be:

At a more general level, I think there are two real key differences between PBL and "total old school" curricula, but only one of them relates specifically to PBL. The first difference is that PBL systems are of necessity "integrated" (i.e. you can't separate the preclinical basic science from the clinical stuff) - but this is also true of many non-PBL "systems-based" curricula.

The second, and more critical difference, is this. In a lecture-based system, most (or even all) things that the curriculum designers thought were important will get mentioned in a lecture somewhere in the course. In a PBL system, most/all things that the designers thought were important will get "flagged up" by things (cues) in the PBL cases.

As you allude to, one of the things that causes angst in PBL systems is whether students "miss stuff" in the cases, which is obviously relevant to what I have just said.

I have heard some of the harder-line educationalist people argue that this "missing stuff" is a key feature (sic), the idea being that the students need to learn to be "self-policing" about what they miss. You certainly do get times when someone will say "I think we need to cover XYZ... group 6 did it last week and I think we must have missed the cues to it". On the other hand, most PBL tutors will give students a strong push towards something if it is important and it is being missed in the discussion, as you say happens in your PBL sessions.

Re. early years PBL (say yrs 1 and 2), I don't think clinical staff are actually the people you want in there as tutors. If you want the focus in yrs 1 and 2 to be on underlying basic science, which is what most systems hope to do, then you want scientists as tutors, in my opinion. But they do need to be widely knowledgeable about the basic sciences, and clued-up as to how bits of basic science "connect" to medicine. They also need to be interested in the material and in teaching students.

Now, these things may be hard to fulfil nowadays, given the research focus in the biosciences on "reductionist" cell/molecular stuff. But I am not really convinced they would be fulfilled easily by getting clinical people to do all the PBL. Not to mention that it would cost more, given the difference between clinical and non-clinical salaries.

I am a student at Peninsula, which is a 'modern' pbl based medical undergraduate program.

I didnt think that pbl particularly helped me learn, and your right Dr Grumble, learning from equally ignorant collegues can be like the blind leading the blind. The small group and communication skills were invaluble though.

What i did particularly benefit from was Peninsula's focus on clinicially relevant learning, in which learning was based around clinical scenarios. The focus on learning clinical skills from day 1 was also helpful in relating your phsyiology etc learning to clinical practice. Although the links were occasionally somewhat tenuous, it was a far better way of learning than sitting in lecutures and making notes. It also allowed us to link in public health, and the patient perspectives into our learning which when you go onto our clinical years is really helpful.