Demystifying Medicine One Week at a Time

An interesting conversation recently took place among residency program directors in my field of Internal Medicine.

Testing the old-fashioned way.

At issue was the declining pass rate of first-time test takers of the ABIM Certification Exam.

It’s a mouthful to say, but the ABIM exam is the ultimate accolade for internists; one is only eligible to take the exam after having successfully completed a three-year residency training period (the part that includes “internship,” right after medical school).

An easy analogy is to say that the board exam is for a doctor what the bar exam is for a lawyer. The difference is that a doctor can still practice if s/he does not pass–they might be excluded from certain jobs or hospital staffs; but certification, while important, is a bit of gilding the lily. [Licensure to practice comes from a different set of exams.]

There’s no doubt it’s a hard test. I was tremendously relieved to have passed it on my first try. Over the last few years, the pass rate for first time takers has fallen from ~90% to a low of 84%.

It may not seem significant, but for 7300 annual test takers, the difference in pass rates affects about 365 people–or one additional non-passing doctor for every day of the year.

In any event, we program directors have taken note. And the falling pass rate has raised questions:

Are the study habits of millennials not up to the level of Baby Boomers and Gen X’ers? Now you may be on to something.

One concern that has a ring of truth to it is that young doctors have become great “looker-uppers,” and have lost the sense of what it’s like to actually read and study medicine. While doctors enter the profession with a commitment to lifelong learning, some of us fear that the young folk only go far enough to commit to lifelong googling.

Another key point: in today’s era of restricted work hours, something has to give. Too often, when residents must complete the same amount of work in a limited amount of time, what’s sacrificed is the didactic portion of the education: the stuff we do by running through case after case, discussing subtleties and action plans. When time is limited, the work’s simply gotta get done.

8 Comments

In such succinct fashion, you’ve articulated a highly valid point. While reduced work hours have made residency training more manageable there appears to be little doubt that our medical knowledge has been compromised as a result. In some cases, residents seem to look at the work hour rules as a race to see who can leave the hospital the earliest. Residency training is supposed to be about mastering the complex interactions happening in the human body and the effect of disease on this system. No small task.

I would just add that I wouldn’t be surprised if previous generations of doctors actually feel that they didn’t have to read as much medicine on their own since the broad clinical experience gave them lessons that textbooks and the Internet can’t provide. Furthermore, looking up material online provides the further distraction of receiving a Facebook notification or stumbling upon a series of funny cat videos on YouTube.

Maybe you are looking at work hour restrictions in the wrong way. First-time passers have been declining since resident work hours have become fully apart of the resident culture. I was the first class (2006) of mandatory work hour restrictions yet my second and third-year residents were still apart of the old guard and many times I heard about “when I was a intern…..” Faculty may have stressed the restrictions but impressing my residents and my reputation amongst them was far more important than the nephrology attending who was doing his two weeks of mandatory general medicine wards. Spending time with my residents and learning from them or being called out by an attending for not knowing the answer being forced into arguing my side were invaluable and still stick with me to this days. There was pressure to stay and not live by the clock. Liberties were taken with clocking in and out. By 2008, these were the interns working with my class. At this point, we and the 2007 class were the last groups to work with the old guard and that pressure to stay and work was waning. The class of 2009 never had to deal with those pressures because work hours and clocking your times were the new norm.

Studying is not enough. Maybe classic medical education of spending more time on the wards or in clinics experiencing AND then reading about your patients repeatedly helped ingrained the information and is why pass rates were higher. Spending more times in teaching rounds and learning from the residents and attendings are so important but because time is of the essence, those little pearls of wisdom are not being passed down. If passing exams was about studying, then why have residencies at all? Because on-the-job learning is just as important as studying.

So…who needs to change? the Millenials or the ABIM exam and its Boomer/Gen-X writers/administrators? Maybe the Millenials are on to a skill set that is undervalued by the ABIM. I sit right on the border of Gen-X and Mellenials, and I am not sure of the answer. This year, as a third year FP resident, I was surprised when on rounds I saw a 3rd year medical student candidly use a smartphone while getting pimped. No one said anything about this but I can’t imagine just 4 years ago having done this on rounds as a student and gotten away with it.

in about 2005, while the PD for the first two years of surgical residents (called the foundational program now in Canada), I finally realized that the “way it was done” in education was not working, and that we had to ask them to tell us how they wanted to learn. that is, many had trouble concentrating for long periods, had a problem sitting in a classroom or auditorium, and wanted digital info that they could save somewhere to get to at some later date. look or listen briefly, catalog and move on was the general methodology. adoption of anything digital was the way. it helped me to realize that although more efficient, the lecture, if not accepted, was not useful. in the end, the learner rules the game. the process of sitting on our past experiences and methodologies just does not work, and we need to be dynamic with them, and to develop new ways of getting to understanding and educational maturity with new methodologies.

A COUNTERPOINT FROM A MILLENIAL
Thank-you very much for opening this dialogue about this topic. Too often, we are having this conversation muttering underneath our breaths instead of out in the open. Boomers/GenXers are often muttering about us ‘not paying attention’ or ‘not studying enough’…. And we in turn are muttering about how our exams are ‘just about memorizing a bunch of bull$#!* that will be out of date in 2 years, if it isn’t already plainly out of date now’.

PROTESTING TOO MUCH?
As a millenial, I am starting as an attending now this year, and I have to say, I still passed my exams, and still had to deal with learning the ‘old fashioned way’. In the end, memorizing factoids and endless lists were the way to beating the exam – but these do not translate well into my daily practice as a physician. I think that a big problem underlying the current examinations systems in most specialties and jurisdictions is that they ask antiquated questions that often have not changed with the times, nor advanced with the fields on which they test. Most importantly, they value the lower levels of learning (e.g. Bloom’s Taxonomy level = ‘Remember’, and perhaps the higher level of ‘Apply’; or Miller pyramid’s ‘Knows’, ‘Knows how’) rather than critical reasoning and problem solving.

EGO VS. SUPEREGO
My expertise in “googling”, however, has prevented medical errors and possibly saved lives. There has been at least once when posed with a difficult patient encounter and two MDs facing a quandry, we turned to the wisdom of the internet to resolve the issue. I did, however, not just take the first random website… but actually used resources like Cochrane or other guidelines to support my decision. My ego (or dare I say, my Id) might have had me just shouting at my colleague to their face, having an unsupported argument, and then someone would pull rank and decide. But instead, both of us used our superegos, and with the assistance of the internet, arrived at the best answer for the patient. And yes, that day, I was right – but I could have just as easily been wrong. And being wrong might have caused morbidity or mortality.

The days of holding all of medical knowledge in your head are going the ways of the dinosaurs. The lawyers have long known that it is not your actual memorization of the law that makes one a great lawyer, but what you do with it. Most other professions do not kid themselves into thinking that their experts can remember it all. Instead, they allow for open book exams (e.g. the Bar exam, the Canadian General Accounting exams) – because it is your REASONING and ARTICULATION of your thoughts that shows the true knowledge. The vast sum of medical knowledge is easily now much larger in volume than any criminal code, and yet, why are we so stuck on the idea that our doctors should just ‘know it all’ (and off the top of their heads)?

TEACHING AND STRIVING FOR A HIGHER LEVEL (OF LEARNING & BEST PRACTICES)
When we ‘pimp’ our medical students… we ask them core knowledge. Something easily accessible by google now. But the fact that they can look it up quickly does not a great physician make. In fact, by making memorizing factoids and statistics the mainstay of medical ‘expertise’, we have historically relegated other important aspects of our job (e.g. interpersonal skills, leadership, etc.) to the wayside. Certification exams like the ABIM can not measure whether a finishing physician can convince a patient to be adherent to a regime of medications or lifestyle changes. It can only measure that he or she knows that the patient could benefit from that intervention.

But what use is that knowledge without effective action that follows?
What are our exams not about clinical reasoning on a patient that does not make clear cut sense?
Why not about negotiating with a standardized patient or nurse over a difficult situation?

Likely because these are difficult to examine… And so, as educators, we have chosen the path of least resistance – giving our learners a scanton sheet and a #2 pencil.

NOT ‘DUMBER’, JUST THE HARBINGERS OF CHANGE
I would like to stand up for my generation in saying that we are likely not DUMBER. We are just different. We have been taught with frameworks like CanMEDs and the ACGME Competencies and we see the practice of medicine as more than just bubbles to be filled in, or lists to be recited.

If work hours restrictions are to be blamed, then shouldn’t exam scores have gone up?
Much of that self-directed memory work we call ‘studying’ is not really best done at work. Have you ever tried to read a chapter of Rosen’s Emergency Medicine in between seeing 30-something patients in a busy Emergency Department? Is it even possible to memorize approaches to malaria and other parasitic diseases from Harrison’s while on call for Internal Medicine and admitting your 4th patient with a COPD exacerbation? If you truly think about it, if ‘reduced work hours’ has ANYTHING to do with pass rates on exams then it should have translated into more off-the-clock studying time and have INCREASED pass rates… since you can’t really study for MCQ questions and Short Answer Questions while writing orders on call, interviewing a patient, or holding a retractor in the OR…

To think that we residents (or in my case, former residents) learned ONLY at work is somewhat naive and teacher-centered.

A CALL FOR CHANGE
As we Millenials are a group with little agency in the matter of how we write exams, however, might I also posit that our whole examinations system need radical change. Most recently while studying for my final examination in residency, I was advised to answer questions as a ‘expert doctor from 2008’ instead of an expert and up-to-date doctor of 2013 – because my examiners and the exam-setters will not have read the 2013 update on STEMI care, nor the 2012 IDSA guidelines… etc.. Many other colleagues and friends have recounted similar quandaries when sitting for their exams.

Not only do these examinations test lower levels of knowledge because those are the easiest to test, but possibly it has a hidden and more serious underpinning… Possibly, a far more insidious implication for professionalism and patient safety.

When you look at it from a distance, the current system from medical school onward highly values guessing and intellectual bravado (or as I like to call it, the ‘My Brain is Bigger than Yours’ syndrome). The current system encourages and, in fact, REWARDS ‘guess work’… in a field where ultimately, we should value those who double check when unsure, or evoke institutional guidelines and checklists to PREVENT errors rather than resting on the perceived intellectual prowess of the know-it-all physician. So too does the public act of ‘Pimping’, especially when the ‘gold standard’ is the ‘Pimper’ and not an actual referenced source.

In a culture where surgical checklists and multidisciplinary teams are the norm – why must the doctor be the pillar of all knowledge, standing alone, unsupported by the vast amount of medical knowledge that is literally at our fingertips?

I dare say that most of us want to train physicians who are humble enough to look things up on our smartphones. If we want our clinical clerks and residents to know their limits – why MUST I choose “D” or randomly guess on items I don’t know? With modern computers, should we not be able to allow learners to say ‘I don’t know, but I know where to look it up”? Would that not be a better habit to form? Our hidden curriculum and assessment suggests that if you’re a “real doctor” you should just bluff and guess if you’re not sure – lest you risk looking like a fool. Instead, should we not be cultivating a culture that encourages each person to become more self-aware, knowing one’s limits, and how to overcome them?

When I ask questions (i.e. ‘pimp’) my students and junior residents, I have always rewarded them for saying ‘I don’t know’. I am happy if they are able to recognize that they have their limits of knowledge – as medicine is rapidly advancing and changing readily. I reward them further, too, when they ask for 2 minutes to look it up… because then I can watch them reason through their thinking, vet the course with which they navigate the vast knowledge base of our discipline, and possibly guide them better towards the right answer. In fact, I have used this as a teaching technique, with 4 learners collaborating via smartphone to make a quick learning guide after being posed a question they could not answer.

Some day when I’m NOT there, and they are 10 years out into practice, I will feel successful if they retain ONLY the skill of being able to look up and vet the answers in front of them. With the rate of medical research and scientific discovery, I have no doubt that the ‘medical fact’ I teach them now will be vastly out of date and possibly viewed as ‘medical myth’ by then. The ONLY thing that will allow our learners to continue to change and grow with our discipline will be those skills of point-of-care research and evaluation.

The way forward is not necessarily to repeat history. Let’s challenge ourselves to lead our field into the future instead of regressing into or repeating the past. It was not always better ‘back in the day’.

Beautifully written! I am 10 years out of my surgical residency (thus trained mostly without hour restrictions) and couldn’t agree more. Medicine is so vast and so rapidly changing, that it is unreasonable to ask anyone to “know it all”. I look up information almost daily to make sure I am providing the best, most current care for my patients. And, I have learned many things ( “bedside skills”, leadership) that help me every day and have never tested. Why do we cling to the traditional way of testing when we no longer practice that way?