We use cookies to improve our service and to tailor our content and advertising to you. More infoClose You can manage your cookie settings via your browser at any time. To learn more about how we use cookies, please see our cookies policyClose

The errr-portfolio

All rapid responses

Rapid responses are electronic letters to the editor. They enable our users to debate issues raised in articles published on thebmj.com. Although a selection of rapid responses will be included online and in print as readers' letters, their first appearance online means that they are published articles. If you need the url (web address) of an individual response, perhaps for citation purposes, simply click on the response headline and copy the url from the browser window. Letters are indexed in PubMed.

I thank Dr Spence for his entertaining and timely column: On the
evidence base of recent discussions on the doctors.net forum and personal
communications, large numbers of trainees echo his sentiments.

It is mandatory for surgical speciality trainees to engage with the
online Intercollegiate Surgical Curriculum Project (ISCP) and to do so,
trainees must pay £125 each year. Assessment is via multiple electronic
forms (only slightly satirised by Dr Spence); six meetings with an
educational supervisor; time consuming documentation of "Learning
Agreements", ticked off electronically when the supervisor has seen
evidence that the trainee has completed these; in addition to yearly ARCP,
at which the trainee need not be present. Old style specialist registrars
did not (do not) use the system and were (are) assessed once or twice
yearly by RITA which is documented on a short, free, paper form.

In response to arguments that the thorough electronic documentation
is A Very Good Thing, I refer readers to a survey of ISCP users, n=539,
recently published in J R Soc Med (Pereira and Dean, 2009). The survey
found 49% described the performance of ISCP online assessments as poor or
very poor. 79% rated the ISCP website’s user friendliness as average or
worse. 76% respondents reported needing to carry out paper assessments due
to practical difficulties using the online forms. 94% did not consider the
trainee fee good value. The authors conclude "The performance of the ISCP
leaves large numbers of British
surgeons unsatisfied." In its current format, I agree with the majority of
respondents on ISCP and with Dr Spence's judgment on "uniform validated
assessment".

Competing interests:
S Price is a modern medical trainee in Specialty Training having transferred from the "old system" after 18 months as a LAT.

Dr Spence is indeed a brave man.
Or perhaps he is unaware that for several years now it has been clear to
those doctors properly qualified to hold or express an opinion that
carefully measured well documented competency-based training to a well-
structured curriculum is vastly superior?
The old haphazard laissez-faire acquisition of random experiences during
poorly planned and often opportunistic self-managed series of jobs is what
led to international assumptions about the quality of British Medical
Training, based on its products..the Consultant, and the GP.

The esteem they seemed to enjoy everywhere was founded on sand: the
most superficial look at their documentation would have proved they were
winging it.

I'm sure we'll quickly see replies from many better qualified than I
who will be able to clearly demonstraate the evidence to prove all this.
(I do seem to recall there was one paper in about 2001 which might be
quoted?)

The McNamara Fallacy is named after the Robert McNamara, the US
Secretary of Defence in the 1960s who was obsessed with quantifying the
Vietnam War in a way that has many parallels with medical education today.
Needless to say the Vietnam War was not the US administrations finest
hour.
It can be summarised as follows-

1. Measure whatever can be easily measured.

2. Disregard that which cannot be measured easily.

3. Presume that which cannot be measured easily is not important.

4. Presume that which cannot be measured easily does not exist.

The crux of the issue for all of us, Des included, is the fact that
the formalisation of any assessment system often leads to intuition and
clinical and academic judgement (phronesis) being viewed as of secondary
importance.

A simple problem is to bake a cake, a complicated problem is fly a human
to the moon and back, a complex problem is to toilet train a child. A
skilful doctor needs to be deal with a lot of complicated problems but
also deal with a lot of much more complex issues, some of which are
always changing. This applies to both technical and non technical,
clinical and non clinical skills. It beggars belief that these assessment
tools, full of word salads of “unspeak”, can actually measure the gamut of
clinical skills, discretionary judgement and interpersonal skills to deal
with complex problems. This illusory belief that we can convert the
intangible into to the tangible makes things worse and my impression is
that most trainees view these tools as a hindrance to actual learning.
Emperors clothes. Is it time that someone was brave enough to mention
sherry, grey suits and even heaven forbid, honest references. Did a
luddite like me just say that?

Competing interests:
These ideas are based on a talk given by Dr R Glavin